fair_drop.suspicious

Note

We currently only support collections with incrementing IDs
Only Ethereum NFTs can be scraped

This module can be used to scrape a NFT collection on OpenSea for tokens flagged as suspicious. The scraped data is stored in data/suspicious_nfts/ as a JSON file with extra metadata. The scraped_on field is a UNIX timestamp (epoch) for when the file was created.

{
 "contract": "0xbc4ca0eda7647a8ab7c2061c2e118a18a936f13d",
 "name": "Bored Ape Yacht Club",
 "scraped_on": 1659948199,
 "data": [
     {
         "token_id": 0,
         "url": "https://opensea.io/assets/ethereum/0xbc4ca0eda7647a8ab7c2061c2e118a18a936f13d/0",
         "is_suspicious": false
     },
     {
         "token_id": 1,
         "url": "https://opensea.io/assets/ethereum/0xbc4ca0eda7647a8ab7c2061c2e118a18a936f13d/1",
         "is_suspicious": false
     },
     {
         "token_id": 2,
         "url": "https://opensea.io/assets/ethereum/0xbc4ca0eda7647a8ab7c2061c2e118a18a936f13d/2",
         "is_suspicious": false
     }
   ]
}

Example

To scrape the BAYC collection from OpenSea, you only need to supply the contract address. All other CLI arguments are optional.

$ python3 fair_drop/suspicious.py --contract 0xbc4ca0eda7647a8ab7c2061c2e118a18a936f13d

Command Line

usage: suspicious.py [-h] -c CONTRACT [-r RETRIES] [--backoff BACKOFF]
                     [-b BATCH_SIZE] [--lower_id LOWER_ID]
                     [--upper_id UPPER_ID] [--total_supply TOTAL_SUPPLY]
                     [--log {DEBUG,INFO,WARNING,ERROR,CRITICAL}]
                     [--keep-cache [{True,False}]]

-h, --help: show this help message and exit

-c <contract>, --contract <contract>: Collection contract address

-r <retries>, --retries <retries>: Number of retry attempts

--backoff <backoff>: Retries backoff parameter for failed requests

-b <batch_size>, --batch-size <batch_size>: Batch size of NFT URLs to be processed

--lower_id <lower_id>: Lower bound token ID of the collection

--upper_id <upper_id>: Upper bound token ID of the collection

--total_supply <total_supply>: Total supply of the collection

--log {DEBUG,INFO,WARNING,ERROR,CRITICAL}: Set the desired log level

--keep-cache {True,False}: Keep the cache file after scraping

Internal functions

fair_drop.suspicious.get_upper_lower_total(contract_address)

Get the upper and lower bound, and the total supply of the NFTs in a collection (on-chain).

Parameters: contract_address (str) – Contract address of the collection
Return type: Dict[str, int]
Returns: A dictionary with the lower and upper bound token id and the total supply

fair_drop.suspicious.get_collection_name(contract_address)

Get the name of the collection from the contract.

Parameters: contract_address (str) – Contract address of the collection
Return type: str
Returns: Name of the collection

fair_drop.suspicious.is_nft_suspicious(nft_url, session)

Download and parse the NFT page to check if it is flagged as suspicious

Parameters

nft_url (str) – URL of the NFT page
session (Session) – A requests.Session object

Return type

Optional[Dict]

Returns

A dict with relevant information about the NFT and whether it is suspicious or not

fair_drop.suspicious.list_collection_nfts_urls(contract_address, lower_id, upper_id)

List all OpenSea URLs for the NFTs in a collection

Parameters

contract_address (str) – Contract address of the collection
lower_id (int) – Lower bound token id
upper_id (int) – Upper bound token id

Return type

List[str]

Returns

list of the OpenSea URLs of NFTs

fair_drop.suspicious.main(contract_address, total_retries, backoff_factor, batch_size, lower_id, upper_id, total_supply, keep_cache)

Main function to scrape all NFTs in a collection and check if they are suspicious

Parameters

contract_address (str) – Contract address of the collection
total_retries (int) – Total number of retries to allow.
backoff_factor (int) – A backoff factor to apply between attempts after the second try.
batch_size (int) – Batch size of NFT URLs to be processed
lower_id (int) –
upper_id (int) –
total_supply (int) –
keep_cache (bool) –

Return type

None

fair_drop.suspicious.load_scrape_cache(contract_address)

Loads cache of previously scraped collections, based on CSV files saved.

Parameters: contract_address (str) – Contract address of the collection
Return type: DataFrame
Returns: Either a DataFrame with the scraped NFTs or an empty DataFrame if no cache is found