API Response Watcher
No credit card required
API Response Watcher
No credit card required
Checks if some API endpoint's response has changed. Works by creating and storing a JSON schema from the endpoint's response and using it to validate the next response. Depending on the configuration, the stored JSON schema can be updated every time the response changes.
A utility to check if some API endpoint's response has changed.
Purpose
This Actor allows to to test a list of endpoints to check if their response change over time. For example, you may want to test some API you are using to scrape the content of a website and be notified if some data has been added, changed, or removed to a specific endpoint.
How it works, and how to use it
The first time you test an endpoint, the Actor creates a JSON schema out of its response and stores it in the Key-value store. You can specify the name or the ID of the store you want to use for this, into "Next Key-value store", in the Actor's input. For more information about Key-value stores, see here.
The next time you want to test the same endpoint, you can move the Key-value store ID from "Next Key-value store" to "Previous Key-value store": in this way, the Actor will use the previously generated JSON schema to validate the new response. If the validation fails, it will output the differences.
Finally, the Actor will regenerate the JSON schema out of the new answer, merge it with the old schema and save it in the Key-value store pointed by "Next Key-value store". If the previous and next stores are the same, the old schema will be overwritten.
Input
Here is a sample input:
1{ 2 "endpoints": [ 3 "https://dummyjson.com/products", 4 "curl 'https://dummyjson.com/carts' -H 'Accept: application/json'" 5 ], 6 "prevKvsName": "abcde12345", 7 "nextKvsName": "schemas", 8 "noRequired": false, 9 "doMergeSchemas": true, 10 "reportEmail": "my.email@apify.com", 11 "reportSlackChannel": "#api-watcher", 12 "reportSlackToken": "*******" 13}
Some notes
- The Actor will test two endpoints:
https://dummyjson.com/products
andhttps://dummyjson.com/carts
:- the first endpoint is a simple public endpoint's URL, which will be contacted through a
GET
request; - the second endpoint is actually a cURL command, which will be parsed. You can use the cURL syntax to describe more complex requests, for example ones using a method different from
GET
, custom headers, and custom payload.
- the first endpoint is a simple public endpoint's URL, which will be contacted through a
- The Actor will validate their responses against some JSON schema from the Key-value store
abcde12345
, if found. - The Actor will store the updated schemas in a named Key-value store called
schemas
. Each endpoint will have a record key in the Key-value store based on its URL. - While generating and updating the JSON schema, the Actor will mark all the JSON properties as "required", because
noRequired
is set tofalse
. - The Actor will merge the old and new schemas before storing it into the Key-value store, because
doMergeSchema
istrue
. - At the end of the Run, if some differences were found:
- an email will be sent to
my.email@apify.com
, calling the Actorapify/send-mail
, with a link to the Run default dataset, where those differences were stored; - a message will be written to
#api-watcher
, calling the Actorkaterinahronik/slack-message
and using the given token.
- an email will be sent to
Further considerations
- The Actor is able to automatically distinguish between store names and IDs.
- If you don't specify a "Next Key-value store" in the input, the Actor won't perform validation.
- If you don't specify a "Previous Key-value store" in the input, the Actor will still store the generated schema into the default, unnamed storage. Unnamed storage has an expiration date: for more information, see here.
Output
Here is a sample output:
1{ 2 "url": "https://some-api/data/1", 3 "data": { 4 "id": 1 5 }, 6 "prevSchema": { 7 "type": "object", 8 "properties": { 9 "id": { 10 "type": "string" 11 } 12 } 13 }, 14 "nextSchema": { 15 "type": "object", 16 "properties": { 17 "id": { 18 "type": "integer" 19 } 20 } 21 }, 22 "mergedSchema": { 23 "type": "object", 24 "properties": { 25 "id": { 26 "type": [ 27 "integer", 28 "string" 29 ] 30 } 31 } 32 }, 33 "validationErrors": [ 34 { 35 "instancePath": "/id", 36 "schemaPath": "#/properties/id/type", 37 "message": "must be string" 38 } 39 ] 40}
The id
in the response, which was previously a string, is now an integer.
The old and new schemas were merged, because doMergeSchema
in the input was true
.
The merged schema admits bot a string and an integer as id
, so, if it will be used to validate the next Run, both types will pass the validation.
How to monitor some endpoints
You can leverage Apify's schedules. Just create a Task with the desired input and run it periodically: you can set it up to receive a notification when some changes are detected.
If you set the same previous and next Key-value stores, the reference schema will be updated every time, so that you will be notified just once when a change is detected.
Or, if you prefer, you can set two different values for the two stores, even leaving the next Key-value store blank, and the change will be detected every time, until you manually update the reference schema.
Actor Metrics
1 monthly user
-
2 stars
Created in May 2024
Modified 6 months ago