Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

469: agent profiler, waterfall chart, step summary table #587

Conversation

paulpalmieri
Copy link
Contributor

Notebook usage only for now:

image

@paulpalmieri paulpalmieri linked an issue Aug 26, 2024 that may be closed by this pull request
@paulpalmieri
Copy link
Contributor Author

@adeprez would you be available to discuss implementation ? I've used globals to be able to decorate any functions but ideally I would avoid that.

@paulpalmieri
Copy link
Contributor Author

Important: currently limited to Jupyter notebooks as it returns a plot and a dataframe.

Usage

Use this code after an agent completes.

plot, table = agent.get_summary()

image

To decorate a function:

from lavague.core.utilities.profiling import time_profiler

with time_profiler("event_name"):
    your_function_call()

Currently tracked steps:

  • world model inference
  • retrieval
  • navigation engine inference
  • execute code

@paulpalmieri paulpalmieri requested a review from adeprez August 26, 2024 15:56
@paulpalmieri paulpalmieri marked this pull request as ready for review August 26, 2024 15:57
@paulpalmieri paulpalmieri changed the title 469 waterfall chart to understand agent bottleneck latency issues 469: agent profiler, waterfall chart, step summary table Aug 26, 2024
@paulpalmieri paulpalmieri merged commit 18d6756 into main Aug 27, 2024
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Waterfall chart to understand Agent bottleneck latency issues
2 participants