[Bug]: Context-related error after the condensation observation?

[enyst]

Is there an existing issue for the same bug?

• I have checked the existing issues.

Describe the bug and reproduction steps

This looks strange, recording it here quick:

This looks like an error in the controller, where it did handle the exception (because we have the condensation observation), but it still failed.

OpenHands Installation

app.all-hands.dev

OpenHands Version

No response

Operating System

None

Logs, Errors, Screenshots, and Additional Context

No response

[mamoodi]

Wasn't this fixed in main and still not released? So OpenHands Cloud won't have this fix? Or is this something else? :D

[enyst]

Maybe... It's strange that it has "trimming prompt" message right before. I thought there is no execution path through the code that could do these one after the other. 🤔 (except there must be lol)

Anyway I also saw it succeed, repeatedly, after this! 🤷‍♂️ If no one else sees this, and we're getting the fix released anyway, I may or not return to it with low priority.

[enyst]

I think I got what happens. I saw this one more time, on the remote app, like:

Trimming prompt to meet context window limitations
Trimming prompt to meet context window limitations
{"type":"invalid_request_error","message":"input length and max_tokens exceed context limit: 204289 + 8192 > 204648, decrease input length or max_tokens and try again"}}...

I think for this succession of messages to happen, it needs several factors:

1. there was a runtime error, bad one, and State wasn't saved. So it didn't have truncation_id to restore only the reduced context; it restored it fully
2. first call via litellm got a ContextLimitExceeded, we caught it and trimmed the prompt (first message)
3. second call via litellm got also a ContextLimitExceeded, we caught it and trimmed the prompt again (second message)
4. I thought Anthropic changed their error message; maybe, but only in a particular case: when input tokens < limit, but input tokens + output tokens > limit. 3rd call via litellm: litellm and us, we didn't catch this one.

Point 1. is worth a better solution IMHO. State is a bit too fragile for something like this.

Point 4. was fixed by @csmith49 and should be in the upcoming release.