Rust Scraper avatar

Rust Scraper

Try for free

No credit card required

View all Actors
Rust Scraper

Rust Scraper

lukaskrivka/rust-scraper
Try for free

No credit card required

Speed of light scraping with Rust programming language! This is an early alpha version for experimenting, use at your own risk!

Start URLs

urlsarrayRequired

URLs that will be scraped. Must be an array of objects with "url" property.

Extraction config

extractarrayRequired

Array that defines what and how should be scraped from a page HTML. See readme for more info.

Proxy configuration

proxy_settingsobjectOptional

Select proxies to be used by your crawler. For most use cases we recommend the default Apify automatic proxy.

Max concurrency

max_concurrencyintegerOptional

Sets the maximum concurrency (parallelism) for the crawl. Keep this is reasonable level because this scraper can go really fast.

Default value of this property is 50

Max request retries

max_request_retriesintegerOptional

Sets the maximum number of retries for each request(URL).

Default value of this property is 3

Debug log

debug_logbooleanOptional

Shows when each URL starts and ends scraping with timings. Don't use for larger runs as the log gets filled quickly.

Default value of this property is false

Push data buffer size

push_data_sizeintegerOptional

Buffers results into vector (array) before pushing to a dataset. This prevents overwhelming Apify API. The default number is usually a good choice.

Default value of this property is 500

Force cloud

force_cloudbooleanOptional

This allows local runs to use cloud storage, mainly for testing. On Apify platform this has no effect.

Default value of this property is false

Developer
Maintained by Community

Actor Metrics

  • 1 monthly user

  • 3 stars

  • >99% runs succeeded

  • Created in Feb 2019

  • Modified 4 years ago

Categories