Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Demo: RAG/docling/llama-index service with an Instructlab frontend #287

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

nerdalert
Copy link
Member

RAG Demo:

  • Adds RAG APIs to docling-serve to create a RAG service.
  • llama-index for categorical document collections.
  • Instructlab quantized model is used for generations.
  • Two collections are created, one via URL and one via file
    uploading to via the Instructlab UI.
  • Collections are then queried and returned with the Answer
    to the query along with the sources and metadata from the
    vector DB.
  • Do a negative test to ensure if the document does not
    contain information matching the query it does not
    hallucinate an answer.

** All components are running locally on a MAC M1.
The PDF ingestions are fast forwarded as they take about 60s each.**

Backend RAG service code will be posted to the docling repo and link here.

Demo video:

rag-demo-v2-oct21.mp4

Signed-off-by: Brent Salisbury <[email protected]>
@nerdalert
Copy link
Member Author

The backend code is posted here DS4SD/docling-serve#9

@vishnoianil vishnoianil added the demo PR that contains Demo related changes label Nov 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
demo PR that contains Demo related changes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants