Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Document configuration for running with Ollama (for local models) #148

Closed
jwmatthews opened this issue Apr 5, 2024 · 2 comments · Fixed by konveyor/editor-extensions#390
Closed

Document configuration for running with Ollama (for local models) #148

jwmatthews opened this issue Apr 5, 2024 · 2 comments · Fixed by konveyor/editor-extensions#390
Assignees
@JonahSussman
Labels
documentation Improvements or additions to documentation
Milestone
v0.1.0

Comments

@jwmatthews
Copy link
Member

We would like to allow usage with a local LLM deployed via Ollama (https://ollama.com/)

Tasks:

  • Configure ollama on a local machine to run ~2+ local LLMs
  • Update provider code to be able to connect and work with ollama
  • Update config.toml to contain an example configuration stanza to ease usage for others
  • Update README.md to show example of how to use ollama
@jwmatthews jwmatthews added the priority/nextup Issues we want to address soon label Apr 5, 2024
@JonahSussman JonahSussman self-assigned this Apr 5, 2024
@jwmatthews jwmatthews mentioned this issue Apr 11, 2024
Closed
@jwmatthews
Copy link
Member Author

Related links that may help:

@dymurray dymurray added this to the v0.1.0 milestone Jan 21, 2025
@jwmatthews jwmatthews added documentation Improvements or additions to documentation and removed priority/nextup Issues we want to address soon labels Jan 22, 2025
@jwmatthews
Copy link
Member Author

This would be a good documentation task to ensure we have coverage for this AND for https://podman-desktop.io/docs/ai-lab

@jmontleon originally created this documentation to help guide the configuration with local LLMs
https://github.com/konveyor/kai/blob/main/docs/llm_selection.md#openai-api-compatible-alternatives

I assume we will need a volunteer to re-run with latest Kai and update docs.

@shawn-hurley shawn-hurley modified the milestones: v0.1.0, v0.2.0 Feb 14, 2025
@fabianvf fabianvf changed the title Update configuration ability to run with Ollama (for local models) and document example Document configuration for running with Ollama (for local models) Feb 19, 2025
@fabianvf fabianvf modified the milestones: v0.2.0, v0.1.0 Feb 19, 2025
@djzager djzager mentioned this issue Feb 20, 2025
Merged
djzager pushed a commit to konveyor/editor-extensions that referenced this issue Feb 20, 2025
@JonahSussman
📖 Update sample-provider-settings.yaml with Ollama example (#390)
d8fea9c 
Closes konveyor/kai#148

Signed-off-by: Jonah Sussman <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@JonahSussman JonahSussman

Labels
documentation Improvements or additions to documentation
Projects
Kai Tech Preview Release
Status: Done
Milestone
v0.1.0
Development

Successfully merging a pull request may close this issue.

📖 Update sample-provider-settings.yaml with Ollama example konveyor/editor-extensions
5 participants
@jwmatthews @shawn-hurley @fabianvf @dymurray @JonahSussman