-
Notifications
You must be signed in to change notification settings - Fork 149
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fantastic. Enhancement ideas #24
Comments
Thanks for this write-up. Pretty spot-on in terms of things I'd like to tackle soon. Will address some thoughts for each:
|
I get your hesitance with summarisation. I think highlighting source information alongside a well written summary is pretty cool. My point is more to your point of keeping things local. And to get that well written "completion" using a local copy of Vicuna iso OpenAI Funny enough I found a repo that does it and demoed here: https://vault.pash.city/ |
The local LLMs idea is definitely intriguing. An additional layer you could potentially utilize with local models: visually showing the model's attention over parts of the summary. You could potentially click on parts of the summary and see which search result context is most utilized to inform each part of the summary. I think I'd only be interested in summarization if we could take it a step further like this with more explainability built-in 😃 |
@jasmeetchadha You might be interested in LocalGPT, which also feeds embeddings results to a local LLM. It requires more resources than my local machines can handle, though, and I don't trust the output of LLMs. Semantra looks more promising for the kinds of things I want to do, finding verbatim citations by concept, etc. I would sometimes want to feed the output to an LLM, though, to find an answer to a question that requires combining multiple result chunks of technical text, for instance. I also plan to use this to search documents in other languages or multiple languages in one file, so it would be good if I could see a translation of the citation into English alongside the quote instead of having to copy and paste into a translator. So some general thing that makes it easier to pipe the results into other tools would be nice. (But I haven't even tried Semantra, yet.) |
I like the idea of semantic search. To expand on this (and I am sure it has been said before). It would be good to add:
UI / Input
Processing
4) Specify where embeddings are stored from UI (or make it clear in readme)
5) Save embeddings to avoid repeat processing
5) Auto-update embedding at startup for any new files that may have been added to folder
6) Enable GPU mode
Summarization (additional)
7) Perhaps use Vicuna (quite good already) or another local LLM to process the results from the search query into a coherent reply (with sources, links to source document).
8) This reply can be adjacent to a "source window" (in case of multiple documents), keeping full visibility into source text for the user.
The text was updated successfully, but these errors were encountered: