LLama 3.2 3B AI Server implementation

This Project Is Under Development, Contributions Are Welcome.

---

To get the current env.template config (reach out to [email protected])

Server Stack

• ⚡ FastAPI for the Python backend API..
  • 💾 Supabase (Supabase)
• 🐋 Docker Compose for development and production.
• 🔒 Secure password hashing by default.
• 🔑 JWT (JSON Web Token) authentication.
• 📫 Email based password recovery.
• ✅ Tests with Pytest.
• 📞 Traefik as a reverse proxy / load balancer.
• 🚢 Deployment instructions using Docker Compose, including how to set up a frontend Traefik proxy to handle automatic HTTPS certificates.
• 🏭 CI (continuous integration) and CD (continuous deployment) based on GitHub Actions.

API Documentation

Service API (provides endpoints to use the model)

• Questionary See API Endpoint
• Summarize See API Enpoint

LLM implementation / initialisation

• LLama 3.2 3B

Deployment Config

Docker is used see here important files:

Dockerfile Docker Config

Configure

You can then update configs in the .env files to customize your configurations.

Before deploying it, make sure you change at least the values for:

• SECRET_KEY
• FIRST_SUPERUSER_PASSWORD
• POSTGRES_PASSWORD

You can (and should) pass these as environment variables from secrets.

Read the deployment.md docs for more details.

Backend Docs (not updated now: template config viewable)

Backend docs: backend/README.md.

Deployment

Deployment docs: deployment.md.