Improvement Update
- Resolved issues with Website Traffic Search due to recent changes to upstream source. It is available for users on a paid Apify plan due to increased upkeep costs.
- General stability improvements and optimizations
Improvement Update
- Performance on Domain Authority Search is now 10x faster (3s per Domain) than before, allowing for quicker lookups and improved efficiency when processing large datasets
- General stability improvements and optimizations
Improvement Update
- Added new Website Traffic Search to extract website traffic data for given domains, similar to the old Website Traffic search, but with improved accuracy and reliability
- Added new Keyword Difficulty Search to extract keyword difficulty data for given keywords and country, providing insights into the competitiveness of specific keywords in search engine results
- Overhauled logs and messages to provide clearer guidance and information to users, enhancing the overall user experience
- Improved documentation to provide more clarity and consistency
- General stability improvements and optimizations
- Removed unavailable search options SEO Audit, Competitors, Keyword Rankings, AI Visibility. They will stay
Improvement Update
- Improved log messages to provide more clarity on the process and
expected outputs
- Improved documentation to provide more clarity on the available
search types and their expected outputs
- General stability improvements and optimizations
- Deprecated unavailable search types (Website Traffic, SEO Audit,
Competitors, Keyword Rankings, AI Visibility). They will stay
unavailable until further notice due to heavy bot detection on
upstream sources.
Maintenance Update
- General stability improvements and optimizations
Hotfix Update
- Updated Overview Output View to support newly added fields
- Resolved some networking issues
Maintenance Update
- Updated Output View to support new formatting and features
- Updated documentation to reflect the changes
- Resolved issues with Website Traffic & Backlinks searches
Improvement Update
- Added support for limited Actor permissions to enhance security and
control
- Added new Dataset Schema features to improve data validation and
consistency
- Increased Max URL(s) per run for paid users from 3 to over 100K to
allow large-scale data extraction
- Updated documentation to reflect the changes
Maintenance Update
- Resolved parsing issues with Domain Authority Search on some
queries with incomplete data
Maintenance Update
- Domain Authority Search is available with Semrush Domain Authority
Score and new MOZ metrics (Domain Authority, Spam Score). We'll try
to bring back old Domain Authority search data as soon as possible.
- Website Traffic Search is also temporarily unavailable due to
increased bot protection on the upstream source. We are working hard
to resolve the issues and restore full functionality as soon as
possible.
Hotfix Update
- All search types except Website Overview, Domain Authority, Website
Traffic and Top Websites are temporarily unavailable due to
increased bot protection on the upstream source. We are working hard
to resolve the issues and restore full functionality as soon as
possible.
Hotfix Update
- Added Competitors Search as a new search type to extract top
competitors for given domains and search countries
- Decreased max URLs per run for paid users from unlimited to 3 per
run (Free Users: 1) to ensure optimal performance and reduce chances
of failure
- Updated Documentation to reflect the changes
- General stability improvements and optimizations
- Mitigated issues with SEO Audit Searches, Domain Authority
Search and Website Traffic Search due to heavy bot detection
on upstream sources
Improvement Update
- Updated Documentation to reflect the changes
- General stability improvements and optimizations
- Resolved issues with Domain Authority Search and Website
Traffic Search due to recent changes to upstream source
Improvement Update
- Added all new AI Visibility Search to extract AI visibility
metrics for given domains / brands
- Updated Documentation to reflect the changes
- General stability improvements and optimizations
Hotfix Update
- Mitigated issues with our backend, which caused runs to hang
indefinitely in some cases.
Hotfix Update
- Domain Authority Search is under maintenance due to recent
upstream api changes, it is temporarily unavailable until further
notice.
- Resolved issues with SEO Audit Search due to recent changes to
upstream source
Hotfix Update
- Resolved domain validation issues, newer TLDs are now fully
supported.
Maintenance Update
- We renabled Backlinks Search for free users again for testing
purposes
- Updated Documentation to reflect the changes
- Resolved issues with SERP Search exiting early with missing
domain info but without a domain/URL needed for this search. It will
now run properly with keyword and country only.
Maintenance Update
- Due to recent changes, SEO Audit Search is now only available to
paid users.
- Due to recent changes, free usage limits are reduced significantly.
Free users can now only run 2 URLs per run instead of 5.
- Updated Documentation to reflect the changes
- Resolved issues with Domain Authority Search, Website Traffic
Search, Keyword Rankings Search and several others due to
recent changes to upstream source
- Resolved issues with SEO Audit Search due to recent changes to
upstream source
Improvement Update
- Added all new SEO Audit Search and Keyword Rankings Search
- Updated Documentation to reflect the changes
Improvement Update
- Revamped Documentation to provide better readability and clarity
Improvement Update
- Added
inbound_traffic_sourcesand outbound_traffic_sources to
Website Overview output to provide more insights into traffic
sources
- Changed which searches are enabled by default for new users. Now,
only Domain Authority, Traffic and Backlinks searches are enabled by
default. Website Overview is disabled by default as it only return
results for high-traffic websites (Top 25K). Top Websites searches
is now also opt-in. Users can enable them if needed.
- Due to recent changes, we now only make Backlink Search
available to paid users.
- We revamped output view to make it more user-friendly and easier to
access key metrics
- Updated Documentation to reflect the changes and provide more
clarity
- Fixed Website Overview Search and Backlinks Search returning
no data due to recent changes on Semrush
Improvement Update
- Fixed Keyword Volume CPC Search to ensure it runs also when no
Domain / URL is provided as it is not needed for this search type.
- Fixed SERP Search to ensure it runs also when no Domain / URL is
provided as it is not needed for this search type.
- Fixed Top Trending Websites Search to ensure it runs also when
no Domain / URL is provided as it is not needed for this search
type.
Improvement Update
- Added Backlinks Search as a new search type to extract backlink
data for given domains
- Added Keyword Volume CPC Search as a new search type to extract
keyword data including search volume, cost-per-click (CPC) and more
for given keywords and country
- Added SERP Search as a new search type to extract search engine
results page (SERP) data for given keywords and country
- Added a new field
domain to Domain Authority Search to store the
domain name so that it can be easily referenced in the output.
- Improved logs by providing more detailed messages to the URL
validation process. Now, each URL's validation status is logged
- We ristricted the number of URLs per run to 5 for free users to
ensure fair usage and optimal performance. Users on a paid plan have
unlimited usage.
Release Version
- We released our Semrush Scraper, which allows users to extract data
from Semrush, including website authority, traffic, and top websites
in various industries and countries.
- There will be additional search types like keyword rankings and serp
searches and many features coming very soon
- Users are encouraged to provide feedback for further improvements.
- Future updates will focus on continuing to improve and enhance searching capabilities.
For any issues or suggestions, please open up an issue on the platform.
This changelog entry documents the initial release of the Semrush Scraper actor, detailing its core features, functionalities, and initial bug fixes.
