Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update the chatbotcore modules #47

Merged
merged 24 commits into from
Jan 7, 2025
Merged

Update the chatbotcore modules #47

merged 24 commits into from
Jan 7, 2025

Conversation

ranjan-stha
Copy link
Collaborator

Changes

  • Update the retrieval system
  • Update the chunking method.
  • Update the prompts used for llm
  • Update the Hyper-parameters.

This PR doesn't introduce any:

  • temporary files, auto-generated files or secret keys
  • n+1 queries
  • flake8 issues
  • print
  • typos
  • unwanted comments

This PR contains valid:

  • tests
  • permission checks (tests here too)

ranjan-stha and others added 23 commits December 20, 2024 09:33
@ranjan-stha
Add contextual chunk module; Fix on llm prompts;
90e7757
@ranjan-stha
Update packages in lock file
9f33566
@ranjan-stha
Update the memory needed for embedding model;
6f1d861
@ranjan-stha
Update the configurations for worker and embedding model;
237eacc
@ranjan-stha
Use the env variables for embedding model; Update the payload for pos…
cd02b15 
…t request;
@ranjan-stha
Update the parameters;
27fb4bd
@ranjan-stha
Include the celery env;
e89f110 
Add the env values
@ranjan-stha
Update to handle numeric values
f038507
@sandeshit @ranjan-stha
small change to the prompt
fdf0e90
@sandeshit @ranjan-stha
filter for history added and prompts updated
219e4c9
@sandeshit @ranjan-stha
Only unique documents retrieval
76cc3fd 
fix pre-commit issues

removed unwanted logs

fixed flake8 issues
@sandeshit @ranjan-stha
chunk formatting
937d744
@ranjan-stha
refactor and cleanup;
841750d
@ranjan-stha
Use openai module
5673eaf
@ranjan-stha
Add openai api key env var
3353901
@ranjan-stha
Extract content when used with openai
071418d
@ranjan-stha
pass the llm type as argument
eeb4220
@ranjan-stha
Fix the data type
fce77c1 
Fix the import
@sandeshit @ranjan-stha
chunk size changes
af4daf9
@sandeshit @ranjan-stha
prompt fixed
1bf0501
@sandeshit @ranjan-stha
llmtype handler
0c65adf 
flake8 issues fix
@sandeshit @ranjan-stha
answer within 200 words
e996e8e
@sandeshit @ranjan-stha
ollamahandler issue fixed
1415f07
@ranjan-stha ranjan-stha changed the title Project/contextual chunks Update the chatbotcore modules Dec 20, 2024
sudan45
sudan45 requested changes Dec 20, 2024
View reviewed changes
chatbotcore/custom_embeddings.py Outdated Show resolved Hide resolved
chatbotcore/doc_loaders.py Show resolved Hide resolved
susilnem
susilnem approved these changes Dec 27, 2024
View reviewed changes
Copy link
Member

@susilnem susilnem left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@sudan45 sudan45 merged commit e066f2c into develop Jan 7, 2025
9 checks passed
@sudan45 sudan45 deleted the project/contextual-chunks branch January 7, 2025 05:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sudan45 sudan45 sudan45 approved these changes

@susilnem susilnem susilnem approved these changes

@thenav56 thenav56 Awaiting requested review from thenav56

@sandeshit sandeshit Awaiting requested review from sandeshit

Assignees

@Rup-Narayan-Rajbanshi Rup-Narayan-Rajbanshi

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@ranjan-stha @sudan45 @susilnem @Rup-Narayan-Rajbanshi @sandeshit