Repositories list

• llmaz

  Public
  ☸️ Easy, advanced inference platform for large language models on Kubernetes. 🌟 Star to support our work!

  kuberneteshuggingfacellm+ 7modelscopellamacppvllmllm-inferencetext-generation-inferenceollamasglang

  Go

  •

  Apache License 2.0

  •11•61•34•0•Updated Feb 11, 2025Feb 11, 2025

• PUMA

  Public
  A lightweight, high-performance inference engine for heterogeneous devices. WIP.

  llmllm-inference

  Rust

  •

  Apache License 2.0

  •1•0•1•0•Updated Feb 10, 2025Feb 10, 2025

• llmchat

  Public
  ChatUI for LLMs on llmaz.

  dashboardllmllmops

  Python

  •

  Apache License 2.0

  •5•3•1•0•Updated Feb 8, 2025Feb 8, 2025

• Awesome-LLMOps

  Public
  🎉 An awesome & curated list of best LLMOps tools.

  awesome

  10•40•0•0•Updated Feb 7, 2025Feb 7, 2025

• website

  Public
  Website and Blogs.

  blogwebsite

  HTML

  •

  MIT License

  •1•1•2•0•Updated Feb 5, 2025Feb 5, 2025

• scheduler-plugins

  Public
  Scheduler plugins designed for smart scheduling with llmaz.

  Go

  •1•3•0•1•Updated Jan 27, 2025Jan 27, 2025

• Manta

  Public
  💫 A lightweight p2p-based cache system for model distributions on Kubernetes. Reframing now to make it an unified cache system with POSIX promise 🎯

  kubernetesdistributed-systemscache+ 2p2p-networkllm

  Go

  •

  Apache License 2.0

  •2•20•14•1•Updated Dec 6, 2024Dec 6, 2024

• omnistore

  Public
  🎯 An unified python client to communicate with various kinds of object-store providers.

  pythonosss3+ 3gcsminioobject-store

  Python

  •

  MIT License

  •2•2•3•0•Updated Nov 11, 2024Nov 11, 2024

• .github

  Public
  1•0•0•0•Updated Oct 23, 2024Oct 23, 2024

• community

  Public
  The InftyAI community.

  Apache License 2.0

  •1•0•0•0•Updated Oct 23, 2024Oct 23, 2024

• template-repo

  Public
  A template repo.

  Apache License 2.0

  •1•0•0•0•Updated Sep 18, 2024Sep 18, 2024

• llmaz-helm

  Public
  Helm installation for llmaz.

  inferencehelm-chartllm+ 2llmopsllmaz

  Smarty

  •1•0•1•0•Updated Sep 11, 2024Sep 11, 2024

• llmlite

  Public
  🌵 A library helps to communicate with all kinds of LLMs consistently.

  llmops

  Python

  •

  MIT License

  •2•17•18•0•Updated Jul 5, 2024Jul 5, 2024

• PR-Copilot

  Public
  Your AI pair programmer 🤖️ specialized in code review, code summary and even code completion. 🧑‍💻🐛

  Python

  •

  MIT License

  •3•2•2•0•Updated Jun 29, 2024Jun 29, 2024