Added to the todo for tomorrow morning , But we found that text saved in apify dataset isn't always the same encoding as the one you print for logs before saving (if this makes sense to you)

We will check if it's code related or apify related when we push to the storage

Thank you for quick response, as always. I understand that the encoding can get broken in various places. Admittedly, we haven't checked the run logs to investigate. In the end, we, of course, just want to grab a dataset file so I hope this can be fixed. I don't think we had this issue with other actors but it could be because the results there were within ASCII charset.

If it's any help, one low-level example of broken encoding: This character "♻️" was encoded as C3 A2 C2 99 C2 BB C3 AF C2 B8 C2 8F instead of E2 99 BB EF B8 8F. My not very educated guess is that this could be double-encoding but I don't think this explains all the symptoms.

I just checked and it's indeed double-encoding. We can probably do some post-processing on our side but it's better to fix the root cause of course.

We have updated the actor, have a try

Probably related to the way we were decoding linkedin text content, we removed the decoding step and return what linkedin returns

Thank you for looking into it. I tried re-running one of the tasks with the latest build (0.0.119) and the issue still persists unfortunately. Seems to be on the same level as before.

If it didn't help we can't do much.

At this point we return what linkedin returns us as content

I know the encoding in dataset is not always the same as the data we initial want to save (we faced this kind of issue with another actor)

Do you want us to handle unicode cleaning (ignore out of scope char / emojis ) ?