Results Checker

  • lukaskrivka/results-checker
  • Modified
  • Users 24
  • Runs 5.2k
  • Created by Author's avatarLukáš Křivka

Check the results of your scrapers with this flexible checker. Just supply a dataset or key-value store ID and a few simple rules to get a detailed report.

Apify Storage ID

apifyStorageId

Optional

string

Id of dataset or key value store where the data are located.

Record Key

recordKey

Optional

string

Only specify if Apify storage id points to a key value store! This is the record key under which the data are.

Raw Data

rawData

Optional

array

Raw JSON array you want to check.

Functional Checker

functionalChecker

Required

string

Function that returns an object with item fields as keys and values as predicates (functions that return true/false)

Context

context

Optional

object

Custom object where you can put any value that will be accessible in functional checker functions as third parameter. Useful for dynamic values coming from other actors.

Identification Fields

identificationFields

Optional

array

You can specify which fields should display in the debug OUTPUT to identify bad items. By default it shows all fields which may make it unnecessary big.

Minimal success rate

minimalSuccessRate

Optional

object

You can specify minimal success rate (0 to 1) of any field. If the success rate will be higher than this, the field will not be count as bad field. This needs t obe an object with fields as keys and success rate as values.

No extra fields

noExtraFields

Optional

boolean

If checked on, the actor will consider extra fields as bad fields. Uncheck if you don't mind them

Limit

limit

Optional

integer

How many items will be checked. Default is all

Offset

offset

Optional

integer

From which item the checking will start. Use with limit to check specific items.

Batch Size

batchSize

Optional

integer

You can change number of loaded and processed items in each batch. This is only needed if you have really huge items.

Max bad items saved

maxBadItemsSaved

Optional

integer

Sets how big report you get for each unique combination of bad fields. Keeping this small and running it again after fixing some is the best approach. It speeds up the actor and reduces memory need.