Actor picture

RequestList Bridge

pocesar/request-list-bridge

Allows you to filter (thus cleaning up your list) and append new data to those requests before sending to your target task. Also enables a workaround to provide requestsFromUrl to existing actors that don't support it natively.

RequestList Bridge

Allows you to filter (thus cleaning up your list) and append new data to those requests before sending to your target task.

Also enables a workaround to provide requestsFromUrl to existing actors that dont't support it natively.

Example

Target task accepts a startUrls but as a plain string array:

{
    "taskId": "x02LumCy0WXdGP7nt",
    "requestListSources": [
        {
            "requestsFromUrl": "https://apify-uploads-prod.s3.amazonaws.com/Hhewqx4YNnyWz2qGk-data.csv"
        }
    ],
    "plainArray": true,
    "targetStartUrlsProperty": "startUrls"
}

Target task accepts RequestQueue only:

{
    "taskId": "x02LumCy0WXdGP7nt",
    "requestListSources": [
        {
            "requestsFromUrl": "https://apify-uploads-prod.s3.amazonaws.com/Hhewqx4YNnyWz2qGk-data.csv"
        }
    ],
    "targetStartUrlsProperty": "startUrls"
}

Filter out and transform the requests:

{
    filter: (req) => /apify/i.test(req.url),
    map: (req) => {
        req.userData.isMapped = true;
        return req;
    },
}

License

Apache 2.0

  • Updated
  • Last run
  • Used22 times
  • Used by3 users
Categories