Tokens And Task Managment - Automatically Keep Tasks and Tokens to a Sane Length

[GaryJurgens]

Background

I am using the VS Code LLM with Copilot for GitHub Models, primarily Sonnet 3.5, and I plan to transition to Sonnet 3.7 once it becomes available, and Stable and GitHub Enables Everything Again.

This Proposal, I don't believe Just Tackles a Sonnet Problem, or a VS Copilot Problem, but it could be Extremely Beneficial to other Models, Especially Reducing Cost of Tokens and Token Usage.

Even Simple Prompts, can often lead to large Tasks, that balloons, this is just the way this works, if you change 1 thing, related things need to also change.

Issue 1: Task Token Limit & API Restrictions

I have observed that when task tokens exceed 500,000, the model begins to hallucinate and encounters API limit errors. This issue is not due to actual request frequency but seems to be tied to the acceptable use policy.

Through testing, I have correlated response quality with token/task size:

• Best responses occur at ~300,000 tokens per task.
• Performance degrades above 500,000 tokens, leading to slower responses and inconsistent output.

Would it be possible to implement a feature that automatically splits large tasks into subtasks while maintaining continuity using
Memory Bank or another method? The goal is to retain thought process and coherence while switching tasks dynamically.

Issue 2: Loss of Context in New Tasks

When a new task is created, Roo does the following:

• Reanalyses the entire project rather than building on previous work.
• Develops a different thought process, sometimes generating conflicting ideas.
• Rewrites code unnecessarily, leading to redundant work.
• Loses track of the original intent, creating inefficiencies and inconsistencies.

As an experienced developer, I don’t use Roo to build entire programs, but rather as an enhancement tool to streamline my workflow, make me more efficient.

However, the current behaviourism increases workload instead of reducing it.

Feature Request

• Implement a task management system that allows for automatic task segmentation when token limits are approached.
• Ensure that previous thought processes, decisions, and context are preserved when switching tasks.
• Introduce a continuity mechanism (e.g., Memory Bank or another method) to prevent unnecessary rework and maintain logical consistency across tasks.

I have Attempted Global Definitions, Task Prompts, Memory Bank Modification and so on. To restrict and isolate this, to no effective solution.

I don't know if i am Missing Something, An MCP Server that Potently does this, already developed ?

This feature would significantly improve efficiency, reduce redundant work, and enhance the developer experience when using Roo for complex projects.

Would love to hear thoughts on feasibility and potential roadmap plans for this! 🚀