No credit card required
AI Web Agent
No credit card required
Use natural language prompts to browse the web, click on elements, fill and submit forms, extract data, and take screenshots using the OpenAI API.
Exceeding maximum context length will crash the flow
Closed
When the AI web crawler finds a website which is bigger (html most likely) than the window which is allowed on the Open AI - it will just crash.
What we would expect is no crash, the data should be capped. Maybe just warning that the data are lowered.
Thank you
2023-11-30T16:09:50.188Z file:///home/myuser/node_modules/openai/error.mjs:31 2023-11-30T16:09:50.191Z return new BadRequestError(status, error, message, headers); 2023-11-30T16:09:50.192Z ^ 2023-11-30T16:09:50.194Z 2023-11-30T16:09:50.196Z BadRequestError: This model's maximum context length is 16385 tokens. However, your messages resulted in 18178 tokens (17820 in the messages, 358 in the functions). Please reduce the length of the messages or functions. 2023-11-30T16:09:50.198Z at APIError.generate (file:///home/myuser/node_modules/openai/error.mjs:31:14) 2023-11-30T16:09:50.199Z at OpenAI.makeStatusError (file:///home/myuser/node_modules/openai/core.mjs:263:21) 2023-11-30T16:09:50.201Z at OpenAI.makeRequest (file:///home/myuser/node_modules/openai/core.mjs:303:24) 2023-11-30T16:09:50.203Z at process.processTicksAndRejections (node:internal/process/task_queues:95:5) 2023-11-30T16:09:50.204Z at async file:///home/myuser/node_modules/langchain/dist/chat_models/openai.js:514:29 2023-11-30T16:09:50.206Z at async RetryOperation._fn (/home/myuser/node_modules/p-retry/index.js:50:12) { 2023-11-30T16:09:50.207Z status: 400,
Hi, thank you for reaching out, we will check it out and let you know.
Hi again, we will adjust the status message on this one, so it's clear that the maximum content length was reached on a first look. Thanks for pointing it out. I will keep the issue opened till it's ready.
petrvecera
Hi, I think the issue is not the "error message" but the crash of the agent. It can't continue. If I am not wrong I can't affect the length of the content length right? I think the right solution would be to "cut" the content and throw warning that content was cut short.
Hi, thanks for the comment, we'll take a look at it. Unfortunately it's not a top priority at the moment, but once someone gets back to it, we'll let you know.
recycling_data
+1 this has been happening for me as well. It would be great if it could return an error code that could then be handled in a workflow. Adding that given the expense of the newer models with larger context window, I would prefer to try it with GPT 3.5 and if it fails because of the context window, send it to GTP 4.
Thanks for upvoting this, we hope to get back to it soon.
Hey,
sorry for the inconvenience. We needed to reprioritized our workload and we will get back to these use cases in about 3 months. We are checking the issues for bugs but this case seems to be more of a situation which simply doesnt work for AI scraping use case. We will test it and try to tweak it when the Actor is based on our roadmap, but even than it may happen that this use case will not be doable with AI scraping tool.
We are going to close the issue now, but 1. will come back to it later and 2. feel free to reopen to ask further questions.
Apify- 86 monthly users
- 60.4% runs succeeded
- 10 days response time
- Created in Oct 2023
- Modified 6 months ago
