A Nvidia and LlamaIndex project on multi-agent reasoning with knowledge graph, focusing on human nutrition.
Run following commands from project root directory.
-
Supports Ubuntu LTS(2204/2404) host only.
-
Ensure Docker is installed on host machine.
-
Ensure Compose Plugin is installed on host machine.
-
Install python dependencies.
pip install -r requirements.txt
- Set API keys in '.env' file.
Initialize local instances.
docker compose --env-file=.env --profile pg16age --profile nim up -d
# Takes around >5mins to build model workspace in nim container.
docker exec -it contest-pg16age psql -U postgres -d postgres -f /tmp/load_kg.sql
# Meanwhile wait for 'contest-pg16age' to initialize before importing graph data from CSV files by running mounted SQL script.
bash scripts/run_vllm
# Wait for backend instances to be initialized before running next command.
Initialize UI.
streamlit run main.py
- Running local instances of llm models requires at least 24GB of VRAM.
- Initial inference require building of finite state machine(FSM) for structured output.
- Lacks data persistence on host filesystem.