Changelog

All notable changes to this project will be documented in this file. See standard-version for commit guidelines.

4.15.2 (2024-11-12)

Bug Fixes

• 🐛 invalid url when scraping subreddit search (963c64e)

4.15.1 (2024-11-06)

4.15.0 (2024-11-06)

Features

• 🎸 prevent images front beeing loaded by actor (741458d)

4.14.2 (2024-10-30)

Bug Fixes

• 🐛 missing post body for some cases (1db9be2)

4.14.1 (2024-10-29)

4.14.0 (2024-10-29)

Features

• 🎸 scrape subreddit rules (c35cf55)

4.13.10 (2024-10-03)

4.13.9 (2024-10-03)

4.13.8 (2024-10-03)

4.13.7 (2024-10-03)

4.13.6 (2024-10-03)

Bug Fixes

• 🐛 scraping duplicated comments (2f11d2b)

4.13.5 (2024-09-17)

Bug Fixes

• 🐛 not tracking startUrls for limiting posts (1f12aa1)

4.13.4 (2024-09-17)

Bug Fixes

• 🐛 limit number os posts for each search or startUrl (2867763)

4.13.3 (2024-09-13)

Bug Fixes

• 🐛 not getting user karma (ed5b21b)

4.13.2 (2024-09-12)

Bug Fixes

• 🐛 user limit on search is not working (ee0a6a5)

4.13.1 (2024-09-12)

4.13.0 (2024-08-29)

Features

• 🎸 display message when limit of communities is reached (33a47f9)

4.12.0 (2024-08-28)

Features

• 🎸 add new sort option (f262e49)

4.11.10 (2024-08-27)

4.11.9 (2024-08-27)

Bug Fixes

• 🐛 not getting community posts after search (2a8263f)

Bug Fixes

• 🐛 not seaching for communities (dd5e735)

4.11.7 (2024-08-20)

Bug Fixes

• 🐛 not paginating user posts (c2c0bb1)

4.11.6 (2024-08-16)

Bug Fixes

• 🐛 comments scraped without dataType on search (c8609ca)

4.11.5 (2024-08-14)

Bug Fixes

• 🐛 duplicating comments when scraping post (033a980)

4.11.4 (2024-08-08)

Bug Fixes

• 🐛 not limiting number of posts when using start URL (3266163)

4.11.3 (2024-08-01)

4.11.2 (2024-08-01)

Bug Fixes

• 🐛 invalid url error when performing search (11938d1)

4.11.1 (2024-07-31)

Bug Fixes

• 🐛 no results when searching for posts (ddab654)

4.11.0 (2024-07-10)

Features

• 🎸 log last page used while paginating (f1220ea)

4.10.0 (2024-07-04)

Features

• 🎸 new field to limit post result by date (26f9b70)

4.9.8 (2024-06-05)

4.9.7 (2024-05-28)

Bug Fixes

• 🐛 error parsing community search url (91e46d9)

4.9.6 (2024-05-20)

4.9.5 (2024-05-20)

Bug Fixes

• 🐛 search community by category does not return results (853d7ea)

4.9.4 (2024-05-19)

Bug Fixes

• 🐛 not scrapping subreddit (e5cae45)

4.9.3 (2024-05-07)

4.9.2 (2024-05-07)

4.9.1 (2024-05-07)

Bug Fixes

• 🐛 search inside subreddit return outside results (6b883a8)

4.9.0 (2024-04-05)

Features

• 🎸 save comment postId for tracking nested comments origin (e7b9fcb)

4.8.1 (2024-04-04)

4.8.0 (2024-03-27)

Features

• 🎸 add videoURL to post (0dd7d5b)

4.7.0 (2024-02-21)

Features

• 🎸 get user id for post and comments (bf3d62f)

4.6.18 (2024-02-21)

4.6.17 (2024-02-21)

4.6.16 (2024-02-21)

4.6.15 (2024-02-21)

4.6.14 (2024-02-21)

4.6.13 (2024-02-21)

4.6.12 (2024-02-18)

4.6.11 (2024-02-18)

4.6.10 (2024-02-18)

Bug Fixes

• 🐛 not handling post url query parameters correctly (61dced2)

4.6.9 (2024-02-17)

4.6.8 (2024-02-17)

4.6.7 (2024-02-17)

Bug Fixes

• 🐛 error while trying to get comment replies (9cfc513)

4.6.6 (2024-02-17)

4.6.5 (2024-02-17)

4.6.4 (2024-02-17)

4.6.3 (2024-02-17)

4.6.2 (2024-02-17)

4.6.1 (2024-02-11)

4.6.0 (2024-02-11)

Features

• 🎸 improve blocked session handling (a75b5ad)

4.5.12 (2024-02-10)

Bug Fixes

• 🐛 not getting all comments (6fd69ed)

4.5.11 (2024-02-10)

4.5.10 (2024-02-10)

4.5.9 (2024-02-08)

4.5.8 (2024-02-08)

4.5.7 (2024-02-08)

4.5.6 (2024-01-04)

4.5.5 (2024-01-04)

4.5.4 (2024-01-04)

Bug Fixes

• missing documentation about lite version (6bc904d)

4.5.3 (2023-12-23)

4.5.2 (2023-12-14)

4.5.1 (2023-12-14)

4.5.0 (2023-12-14)

4.4.0 (2023-12-14)

4.3.1 (2023-12-13)

4.3.0 (2023-12-12)

4.3.0-beta.1 (2023-12-12)

4.3.0-beta.0 (2023-12-12)

Features

• 🎸 include NSFW filter to any url (0a72764)

4.2.1 (2023-12-11)

4.2.0 (2023-11-12)

4.1.0 (2023-11-10)

4.0.9 (2023-10-09)

4.0.8 (2023-10-07)

4.0.7 (2023-08-01)

4.0.6 (2023-07-28)

4.0.5 (2023-07-17)

4.0.4 (2023-07-17)

4.0.3 (2023-07-14)

4.0.1 (2023-07-14)

4.0.1 (2023-07-14)

Bug Fixes

• 🐛 search for comments not working (#100) (4280aea)

4.0.1 (2023-07-13)

4.0.0 (2023-06-26)

3.5.1 (2023-06-19)

3.5.0 (2023-06-02)

3.4.13 (2023-05-29)

3.4.12 (2023-05-26)

Bug Fixes

• 🐛 error scrapping community (813f992)

3.4.11 (2023-05-25)

3.4.10 (2023-05-18)

3.4.9 (2023-05-18)

3.4.8 (2023-05-18)

3.4.7 (2023-05-18)

3.4.6 (2023-05-18)

3.4.5 (2023-05-18)

3.4.4 (2023-05-18)

3.4.3 (2023-05-09)

3.4.2 (2023-05-09)

3.4.1 (2023-04-21)

3.4.0 (2023-04-13)

3.3.0 (2023-04-10)

3.2.0 (2023-01-27)

Features

• 🎸 enable r/popular parsing (c2128fc)

3.1.2 (2022-10-19)

Bug Fixes

• 🐛 scrapping less then the maxItems limit (1cc7125)

3.1.1 (2022-10-19)

3.1.0 (2022-10-19)

3.0.9 (2022-10-17)

Bug Fixes

• 🐛 treat scrollTimeout value bigget than 32 bit number (0e50923)

3.0.8 (2022-10-13)

3.0.7 (2022-08-24)

Bug Fixes

• 🐛 Docker image error (3b60421)

3.0.6 (2022-08-24)

Bug Fixes

• 🐛 search is not returning posts (07c3379)

3.0.5 (2022-08-21)

3.0.4 (2022-08-18)

Bug Fixes

• 🐛 error while scraping comments (8910242)

3.0.3 (2022-08-18)

3.0.2 (2022-08-18)

3.0.1 (2022-08-18)

3.0.0 (2022-08-18)

Bug Fixes

Features

2.2.3 (2022-07-05)

Bug Fixes

• 🐛 broken css selectors for url (fe18b95)

• cannot scrape comments (5f03e7e)

• community members count throws error (d784733)

• community posts are not being scraped (5cc553c)

• getUrlType no working for POST and COMMENT (95d339d)

• 🤖 disable routes that are not working (c06000d)

2.2.2 (2022-06-09)

2.2.1 (2022-05-16)

2.2.0 (2022-04-11)

Features

• 🎸 notify when page requests login (b1a49bd)

2.1.3 (2022-03-15)

2.1.2 (2022-03-15)

2.1.1 (2022-03-08)

2.1.0 (2022-02-23)

Features

• 🎸 add support for content gate on communities (f761cc0)

2.0.4 (2022-02-22)

Bug Fixes

• 🐛 remove maximum parameter from INPUT_SCHEMA (4bc789b)

2.0.3 (2022-02-22)

Bug Fixes

• 🐛 scrapping posts in a search returned an error (bf83b9c)

2.0.2 (2022-02-17)

2.0.1 (2022-02-17)

2.0.0 (2022-02-17)

⚠ BREAKING CHANGES

• All post and comments are saved as a separated JSON file instead of be inside the community or post they are from.

Bug Fixes

• version (0323f7b)

1.2.3 (2022-01-27)

Features

• 🎸 get community category when scraping category topic (#48) (bba616b)
• 🎸 scrape user and communities trophies (#62) (fbbca1b)
• 🎸 use crawler.autoscalePool.abort() (91289ed)

Bug Fixes

• 🐛 different types of url inputs (#45) (f93f954)
• 🐛 empty comments (#44) (895b33b)
• 🐛 humanized date convertion (#52) (68e8da3)
• 🐛 parse comment votes (#53) (da0de4c)
• 🐛 proxy validation (#50) (386a884)
• 🐛 relative date convertion (#49) (7fdef07)
• 🐛 remove unused field from communityParser (96de7f7)
• 🐛 return total members count (#43) (00a7d7e)
• 🐛 save username as user (1223d4a)
• 🐛 toogle search if no startUrls are provided (#36) (90b4c83)
• 🐛 update comment description selector (b52baf5)

1.2.2 (2021-03-02)

Bug Fixes

• 🐛 toogle search if no startUrls are provided (#36) (90b4c83)

1.2.1 (2021-02-26)

⚠ BREAKING CHANGES

• 🧨 Change the startUrl input format

Features

• 🎸 always use search if present (#29) (eb808c0)
• 🎸 save community url (#33) (e7648c7)

Bug Fixes

• 🐛 missing INPUT_SCHEMA fields (7fbfcbe)

• 🤖 use requestList editor on input schema (#28) (955641d)

Changelog

All notable changes to this project will be documented in this file. See standard-version for commit guidelines.