Product listing archiver and image downloader
This tool archives product listings on various asset / 3D marketplaces. It accepts a marketplace and user to be crawled, then crawls and downloads product listings: images, descriptions, and links. See the list below for supported marketplaces:
Please note: this tool is not made to steal art, only archive product listings. Please also be responsible when using this tool, as modifying the degree of parallelism set by the program can cause a denial of service attack. For example, CubeBrush is a very poorly hosted website and crashes with 500 errors very often; therefore, the tool slows down for CubeBrush so multiple clients won’t crash it.
Compile and run the tool
You can find a link to the source code below. Please feel free to build and run the tool yourself, or extend it to support other marketplaces.
ABSTRACT marketplace website username [(--destination | -d) path] [(--token | -t) access_token] [(--verbose | -v)] DESCRIPTION Crawls the specified marketplace website for names, descriptions, and images from product listings belonging to the specified user. The output folder will be created in the current directory, named after the specified user, if the output path left unspecified. SOURCE https://gitlab.com/spirited/marketplace ARGUMENTS website The website to crawl (artstation, cgtrader, cubebrush, envato) username The username of the marketplace author to be crawled OPTIONS -d, --destination string Destination to save images to (current directory) -t, --token string Access token for connecting to some marketplaces -v, --verbose bool Enables verbose logging (false) EXAMPLES marketplace artstation spirited marketplace artstation spirited --destination ./marketplace