Zip Download of KV Store
Try for free
No credit card required
View all Actors
Zip Download of KV Store
vaclavrut/downloadkvstorezip
Try for free
No credit card required
Creates a zip file from all items in the key-value store, zips them, and downloads them in a unique file. On input, you can specify the number of the files or keep it null to download them all. The key-value store must be under your Apify account.
Dockerfile
1# This is a template for a Dockerfile used to run acts in Actor system.
2# The base image name below is set during the act build, based on user settings.
3# IMPORTANT: The base image must set a correct working directory, such as /usr/src/app or /home/user
4FROM apify/actor-node-basic:v0.21.10
5
6# Second, copy just package.json and package-lock.json since it should be
7# the only file that affects "npm install" in the next step, to speed up the build
8COPY package*.json ./
9
10# Install NPM packages, skip optional and development dependencies to
11# keep the image small. Avoid logging too much and print the dependency
12# tree for debugging
13RUN npm --quiet set progress=false \
14 && npm install --only=prod --no-optional \
15 && echo "Installed NPM packages:" \
16 && (npm list --all || true) \
17 && echo "Node.js version:" \
18 && node --version \
19 && echo "NPM version:" \
20 && npm --version
21
22# Copy source code to container
23# Do this in the last step, to have fast build if only the source code changed
24COPY . ./
25
26# NOTE: The CMD is already defined by the base image.
27# Uncomment this for local node inspector debugging:
28# CMD [ "node", "--inspect=0.0.0.0:9229", "main.js" ]package.json
1{
2 "name": "apify-project",
3 "version": "0.0.1",
4 "description": "",
5 "author": "It's not you it's me",
6 "license": "ISC",
7 "dependencies": {
8 "apify": "0.21.10",
9 "apify-client": "0.6.0",
10 "adm-zip": "latest"
11 },
12 "scripts": {
13 "start": "node main.js"
14 }
15}main.js
1const Apify = require('apify');
2const ApifyClient = require('apify-client');
3const AdmZip = require('adm-zip');
4
5async function asyncForEach(array, max, callback) {
6 let itemCount = max !== null ? max : array.length;
7 for (let index = 0; index < itemCount; index++) {
8 console.log("Solving " + index + " from " + array.length);
9 await callback(array[index], index, array)
10 }
11}
12
13Apify.main(async () => {
14 const input = await Apify.getValue('INPUT');
15 const storeId = input.storeId;
16 const maximumItems = input.numberOfItemsInZipFromKv ? input.numberOfItemsInZipFromKv : null;
17
18 console.log("Loading data from KV with ID: " + storeId);
19
20 if(maximumItems !== null){
21 console.log("Going to download only:" + maximumItems + " items from the KV store.")
22 }else{
23 console.log("Going to download whole KV store");
24 }
25 const zip = await new AdmZip();
26
27 const apifyClient = new ApifyClient();
28 const keyValueStores = apifyClient.keyValueStores;
29 apifyClient.setOptions({storeId: storeId});
30 const keys = await keyValueStores.listKeys();
31
32 console.log("Total number of items in store: " + keys.items.length);
33
34 await asyncForEach(keys.items, maximumItems, async (item) => {
35 if (item.key !== "INPUT") {
36 const itemData = await keyValueStores.getRecord({key: item.key});
37 await zip.addFile(item.key, new Buffer(itemData.body));
38 }
39 });
40
41 const willSendthis = await zip.toBuffer();
42 console.log("Saving the zip file.")
43
44 await Apify.setValue("zip_file.zip", willSendthis, {contentType: 'application/zip'});
45 const enviroment = await Apify.getEnv();
46
47 console.log("Download the file here.");
48 console.log("https://api.apify.com/v2/key-value-stores/" + enviroment.defaultKeyValueStoreId + "/records/zip_file.zip?disableRedirect=true")
49});Developer
Vaclav RutMaintained by Community
Actor metrics
- 1 monthly users
- 0.0% runs succeeded
- 48.9 days response time
- Created in Jul 2018
- Modified almost 2 years ago
Categories
Build new tools
Enterprise solutions