Skip to content

Issues: mozilla-ai/lumigator

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

121 Open 171 Closed
121 Open 171 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

[FEATURE]: Secure access from Ray enhancement New feature or request
#863 opened Feb 13, 2025 by javiermtorres
1 task done
3
Extend our eval library / workflow code to support LLM-as-judge api Changes which impact API/presentation layer backend
#862 opened Feb 13, 2025 by aittalam
@aittalam
Align on architecture for running LLM-as-judge evaluation api Changes which impact API/presentation layer backend
#861 opened Feb 13, 2025 by aittalam
@aittalam
Run a judge with the current inference pipeline, both with rubric as system prompt and with sample-wise rubric api Changes which impact API/presentation layer backend
#860 opened Feb 13, 2025 by aittalam
@aittalam
[FEATURE]: Lumigator UI Analytics & Data Collection enhancement New feature or request epic
#859 opened Feb 13, 2025 by jularase
1 task done
Expose in the UI usage of custom models running locally (ollama, llamafile, vllm...) enhancement New feature or request frontend
#858 opened Feb 12, 2025 by ividal
1 task done
Update user guide for local LLM usage via ollama/vllm/llamafile with new experiment/workflow endpoints documentation Improvements or additions to documentation
#857 opened Feb 12, 2025 by ividal
[FEATURE]: Migrate to the new experiments and workflow apis enhancement New feature or request frontend
#855 opened Feb 12, 2025 by khaledosman
1 task done
@khaledosman
1
[FEATURE]: Decouple components ui data from api responses and backend data structures enhancement New feature or request frontend
#854 opened Feb 12, 2025 by khaledosman
1 task done
@khaledosman
Refactor file and folder structure on the frontend enhancement New feature or request frontend
#853 opened Feb 12, 2025 by khaledosman
1 task done
1
@khaledosman
1
Add e2e and screenshot tests enhancement New feature or request frontend
#852 opened Feb 12, 2025 by khaledosman
1 task done
@khaledosman
Refactor component architecture enhancement New feature or request frontend
#851 opened Feb 12, 2025 by khaledosman
1 task done
@khaledosman
[FEATURE]: Cache api requests and properly handle loading and error states for all api requests enhancement New feature or request frontend
#850 opened Feb 12, 2025 by khaledosman
1 task done
@khaledosman
Mention datasets in the README documentation Improvements or additions to documentation good first issue Good for newcomers
#849 opened Feb 12, 2025 by agpituk
[FEATURE]: Improvements in Dataset provision enhancement New feature or request epic
#846 opened Feb 11, 2025 by jularase
1 task done
1
[FEATURE]: Expand Multilingual Model Support & Language Pair Evaluation enhancement New feature or request epic use-case
#841 opened Feb 11, 2025 by jularase
1 task done
[FEATURE]: Translation Specific Metrics enhancement New feature or request epic use-case
#840 opened Feb 11, 2025 by jularase
1 task done
Support for a job creating multiple datasets/multiple of an artifact type backend
#835 opened Feb 10, 2025 by njbrake
User guide: creating a Dataset for Lumigator evaluations (summarization) documentation Improvements or additions to documentation
#834 opened Feb 10, 2025 by ividal
@ividal
1
[FEATURE]: Support more than one lumigator install per kubernetes cluster enhancement New feature or request
#832 opened Feb 10, 2025 by chainlink
1 task done
1
[BUG]: Helm Support release names other than 'lumigator' bug Something isn't working
#831 opened Feb 10, 2025 by chainlink
1 task done
[BUG]: Helm Dependencies cannot be disabled bug Something isn't working
#828 opened Feb 9, 2025 by chainlink
1 task done
[FEATURE]: Bring Your Own Model (BYOM) to be Evaluated in Lumigator enhancement New feature or request epic initiative
#826 opened Feb 7, 2025 by jularase
1 task done
1
Simplify installation: Centralise YAML configuration for Docker
#812 opened Feb 5, 2025 by peteski22
1
@peteski22
Use Lumigator to run comparison of DeepSeek R-1 models with ACI-Bench Doctor-Patient summarization documentation Improvements or additions to documentation good first issue Good for newcomers use-case
#811 opened Feb 5, 2025 by njbrake
@njbrake
ProTip! What’s not been updated in a month: updated:<2025-01-13.