gptme often tries to use python instead of ipython to execute python code

[Miyou]

I feel like this could be solved by adding some extra prompting around this

[ErikBjare]

We used to just use ```python and ```bash for python and shell tool, but had to change it since it kept trying/asking to run non-runnable code snippets/examples it showed using the same markup. I later changed it to just ipython/shell respectively, and it works much better, but it sometimes messes it up still. Tried different prompts, but hard to measure what has impact since it's somewhat rare/random.

See #67 for backstory

[Miyou]

We used to just use python and bash for python and shell tool, but had to change it since it kept trying/asking to run non-runnable code snippets/examples it showed using the same markup. I later changed it to just ipython/shell respectively, and it works much better, but it sometimes messes it up still. Tried different prompts, but hard to measure what has impact since it's somewhat rare/random.

See #67 for backstory

Perhaps not using ``` for tool execution could be a solution. Maybe use XML tags instead?

[ErikBjare]

@Miyou you can try that with the --tool-format xml parameter

[bjsi]

Maybe changing the name of the tool from python -> ipython?

[ErikBjare]

@bjsi Yeah I think that might help, could eval some small model which struggles and see if it helps.

[ErikBjare]

@Miyou Try using latest gptme master with --tool-format tool now that we merged #306

Since it skips the markdown format altogether, it shouldn't suffer from this issue.

[ErikBjare]

I haven't seen this in master branch since merging #361, just bumped into it when trying uvx gptme (for a show-off tweet) which uses the latest v0.25.0 version which doesn't have the change.

I should make a new release, will hopefully fix the issue for good.

[ErikBjare]

Just released https://github.com/ErikBjare/gptme/releases/tag/v0.26.0

Let me know if it still happens!

[ErikBjare]

Just happened to me again :(

Although it had just mentioned the still-not-renamed python.py file:

- gptme/tools/python.py (handle completion signal)

Might still be a lot more rare after the tool rename, should perhaps rename the tool file too.

[bjsi]

Would it work better by formatting tool calls using xml?

…

On Tue, 14 Jan 2025 at 18:04, Erik Bjäreholt ***@***.***> wrote: Just happened to me again :( — Reply to this email directly, view it on GitHub <#327 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AN3UCA32HUT7J6QKVUFLMNT2KVGURAVCNFSM6AAAAABTP654HOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKOJQG42DOMZQGY> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[ErikBjare]

@bjsi Possibly, I haven't tested it extensively. I assume different models struggle to different extents as well (but I almost only run Claude 3.6).