Product listing archiver and image downloader
This tool archives product listings on various asset / 3D marketplaces. It accepts a marketplace and user to be crawled, then crawls and downloads product listings: images, descriptions, and links. See the list below for supported marketplaces:
Please note: this tool is not made to steal art, only archive product listings. Please also be responsible when using this tool, as modifying the degree of parallelism set by the program can cause a denial of service attack. For example, CubeBrush is a very poorly hosted website and crashes with 500 errors very often; therefore, the tool slows down for CubeBrush so multiple clients won’t crash it.
Compile and run the tool
You can find a link to the source code below. Please feel free to build and run the tool yourself, or extend it to support other marketplaces.
marketplace website username [(--destination | -d) path]
[(--token | -t) access_token] [(--verbose | -v)]
Crawls the specified marketplace website for names, descriptions, and images
from product listings belonging to the specified user. The output folder
will be created in the current directory, named after the specified user, if
the output path left unspecified.
website The website to crawl (artstation, cgtrader, cubebrush, envato)
username The username of the marketplace author to be crawled
-d, --destination string Destination to save images to (current directory)
-t, --token string Access token for connecting to some marketplaces
-v, --verbose bool Enables verbose logging (false)
marketplace artstation spirited
marketplace artstation spirited --destination ./marketplace