Added max_new_tokens as a config option to llm yaml block #1317
Conversation
|
Published docs preview URL: https://privategpt-preview-794b2550-d997-4a87-bcaa-373b49d55975.docs.buildwithfern.com |
There was a problem hiding this comment.
The default value should be defined in the code. settings.yaml should not be modified.
Putting default values in settings.yaml is making the file settings.yaml consistantly growing, and will make the creation of custom configuration harder for users - which is why we should try to keep the definition of default values from there.
Co-authored-by: lopagela <[email protected]>
|
Published docs preview URL: https://privategpt-preview-32590c32-8bac-4cca-9ab9-23dd70059514.docs.buildwithfern.com |
Add default value for max_new_tokens = 256 Co-authored-by: lopagela <[email protected]>
|
Published docs preview URL: https://privategpt-preview-d1b13aa7-97a6-4307-9c0d-320e67c3949e.docs.buildwithfern.com |
|
Published docs preview URL: https://privategpt-preview-c1147ddb-3a26-4a2a-b6b8-e4f2a8ddbc24.docs.buildwithfern.com |
|
Published docs preview URL: https://privategpt-preview-9ed065ba-e3f4-48a3-a35e-c51262762f26.docs.buildwithfern.com |
|
Published docs preview URL: https://privategpt-preview-e1fa1630-7e34-4827-bccd-dc4404693a2d.docs.buildwithfern.com |
) * added max_new_tokens as a configuration option to the llm block in settings * Update fern/docs/pages/manual/settings.mdx Co-authored-by: lopagela <[email protected]> * Update private_gpt/settings/settings.py Add default value for max_new_tokens = 256 Co-authored-by: lopagela <[email protected]> * Addressed location of docs comment * reformatting from running 'make check' * remove default config value from settings.yaml --------- Co-authored-by: lopagela <[email protected]>
Added max_new_tokens as a configuration option to the llm block in settings