Allow proxy setting

It’s planned , I moved it up from my todo list , will be done tonight

All my actors would need a bit of cleaning so they have almost a standard input

Thx for raising the issue , will ping you on this post when ready

Done, have a try and tell us if you need anything else

I gave it a try using inputs that had worked before and they're all return zero results when using the proxy.

sorry my bad, fixed a typo

Try it now

Still getting an error:

2023-11-23T06:44:51.460Z Traceback (most recent call last): 2023-11-23T06:44:51.462Z File "/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 776, in urlopen 2023-11-23T06:44:51.464Z self._prepare_proxy(conn) 2023-11-23T06:44:51.466Z File "/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 1045, in _prepare_proxy 2023-11-23T06:44:51.468Z conn.connect() 2023-11-23T06:44:51.469Z File "/usr/local/lib/python3.11/site-packages/urllib3/connection.py", line 625, in connect 2023-11-23T06:44:51.471Z self._tunnel() # type: ignore[attr-defined] 2023-11-23T06:44:51.473Z ^^^^^^^^^^^^^^ 2023-11-23T06:44:51.475Z File "/usr/local/lib/python3.11/http/client.py", line 926, in _tunnel 2023-11-23T06:44:51.477Z raise OSError(f"Tunnel connection failed: {code} {message.strip()}") 2023-11-23T06:44:51.479Z OSError: Tunnel connection failed: 590 UPSTREAM503 2023-11-23T06:44:51.480Z 2023-11-23T06:44:51.482Z The above exception was the direct cause of the following exception: 2023-11-23T06:44:51.484Z 2023-11-23T06:44:51.486Z urllib3.exceptions.ProxyError: ('Unable to connect to proxy', OSError('Tunnel connection failed: 590 UPSTREAM503')) 2023-11-23T06:44:51.488Z 2023-11-23T06:44:51.490Z The above exception was the direct cause of the following exception: 2023-11-23T06:44:51.491Z 2023-11-23T06:44:51.494Z Traceback (most recent call last): 2023-11-23T06:44:51.496Z File "/usr/local/lib/python3... [trimmed]

did you provide your own proxies ?

Send me a google meet invite : sousalopes.thomas@gmail.com or if you are not available , you can send me the input you used by mail, (only proxy used, i will use my own cookie session) So that i can emulate your problem and find a solution

I'm using proxies provided by apify. I tried both the residential and datacenter. Here's the output using the default datacenter proxy: 2023-11-27T23:34:33.994Z proxyConfiguration {'useApifyProxy': True, 'apifyProxyGroups': ['BUYPROXIES94952']} 2023-11-27T23:34:34.103Z proxy url http://groups-BUYPROXIES94952:*********@10.0.33.153:8011 2023-11-27T23:34:34.109Z Traceback (most recent call last): 2023-11-27T23:34:34.111Z File "/usr/src/app/src/main.py", line 536, in main 2023-11-27T23:34:34.112Z linkedin.run() 2023-11-27T23:34:34.113Z File "/usr/src/app/src/main.py", line 357, in run 2023-11-27T23:34:34.114Z self.linkedin(url) 2023-11-27T23:34:34.115Z File "/usr/src/app/src/main.py", line 349, in linkedin 2023-11-27T23:34:34.116Z self.call_linkedin_comments(url, type_url, ugc) 2023-11-27T23:34:34.117Z File "/usr/src/app/src/main.py", line 253, in call_linkedin_comments 2023-11-27T23:34:34.118Z sufix_urn_li = url.split('-%s-'%prefix_urn_li)[1].split('-')[0] 2023-11-27T23:34:34.120Z ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^ 2023-11-27T23:34:34.120Z IndexError: list index out of range

can you provide your input , will try emulate and understand why it's failing to parse the post url

I've tried it with and without the proxy. What I'm seeing now at the end of the log is: 023-11-27T20:40:45.645Z Transforming anonymous people urls to urls with universalnames 2023-11-27T20:40:45.657Z 2023-11-27T20:44:12.935Z skiping author 2023-11-27T20:44:12.938Z no results to be saved

Here is an example input that worked before the change but returns zero results now: https://www.linkedin.com/posts/tomvarghesejr_heforshe-lifelonglearning-mentorship-activity-6475554808718860288-VJlB This also worked before: "days_since_post": 6, "url": "https://www.linkedin.com/in/navneet-singh-160012/" And this is a new link format that I just started getting from linkedin: https://www.linkedin.com/feed/update/urn:li:activity:7134940395557330944/

I've had to stop because I was testing without the proxy and got my account temporarily blocked.