-
Notifications
You must be signed in to change notification settings - Fork 9
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
how to use ollama #349
Comments
Thank you for opening an issue, sorry about that. |
I got another error after pulling your fix: INFO:root:process_url_with_metadata: Starting summarization with ollama... Thanks, |
Thanks for the update, sorry that its still not working, I refuse to use ollama, so I am doing this blind. If it doesn't please let me know and I'll continue to try and figure it out. |
It still doesn't work. To verify my ollama server is OK, I use curl (below) and it works OK: curl http://192.168.0.224:11434/api/generate -d '{ "model": "qwen2.5-coder:7b-instruct-q5_K_M", "prompt": "Why is the sky blue?", "stream": false }' Do you verify tldw with lamma.cpp server? If yes, I can try llama.cpp server. Thanks, |
I did not test it recently, just did, found bugs, fixed them, and pushed the changes. Llama.cpp should work for chatting now with no issues. Did you receive the same error as before? |
Here is the error from printouts: The response of my curl command: {"model":"qwen2.5-coder:7b-instruct-q5_K_M","created_at":"2024-10-13T01:49:45.978756425Z","response":"The sky appears blue because of a phenomenon called Rayleigh scattering, named after Lord Rayleigh who first described it in 1871.\n\nWhen sunlight enters Earth's atmosphere, it encounters gas molecules such as nitrogen and oxygen. These molecules are much smaller than the wavelength of visible light, so they scatter the light in all directions.\n\nThe amount of scattering depends on the wavelength of the light. Shorter wavelengths (such as blue) are scattered more easily than longer wavelengths (such as red). This is because shorter wavelengths interact more strongly with gas molecules and are therefore more likely to be scattered.\n\nAs a result, when sunlight enters Earth's atmosphere, it is scattered in all directions, but the shorter wavelengths of blue light are scattered more frequently. As a result, we see the sky as blue during the day.\n\nDuring sunrise and sunset, the sun is lower on the horizon, so its light has to travel through more of the Earth's atmosphere before reaching our eyes. This causes even more scattering of the shorter wavelength blue light, resulting in a reddish or orange hue in the sky during these times.","done":true,"done_reason":"stop","context":[151644,8948,198,2610,525,264,10950,17847,13,151645,198,151644,872,198,10234,374,279,12884,6303,30,151645,198,151644,77091,198,785,12884,7952,6303,1576,315,264,24844,2598,13255,62969,71816,11,6941,1283,10214,13255,62969,879,1156,7481,432,304,220,16,23,22,16,382,4498,39020,28833,9237,594,16566,11,432,33906,6819,34615,1741,438,46403,323,23552,13,4220,34615,525,1753,9155,1091,279,45306,315,9434,3100,11,773,807,44477,279,3100,304,678,17961,382,785,3311,315,71816,13798,389,279,45306,315,279,3100,13,10698,261,92859,320,20805,438,6303,8,525,36967,803,6707,1091,5021,92859,320,20805,438,2518,568,1096,374,1576,23327,92859,16282,803,16510,448,6819,34615,323,525,8916,803,4363,311,387,36967,382,2121,264,1102,11,979,39020,28833,9237,594,16566,11,432,374,36967,304,678,17961,11,714,279,23327,92859,315,6303,3100,525,36967,803,13814,13,1634,264,1102,11,582,1490,279,12884,438,6303,2337,279,1899,382,16014,63819,323,42984,11,279,7015,374,4722,389,279,34074,11,773,1181,3100,702,311,5821,1526,803,315,279,9237,594,16566,1573,18739,1039,6414,13,1096,11137,1496,803,71816,315,279,23327,45306,6303,3100,11,12942,304,264,62144,812,476,18575,39040,304,279,12884,2337,1493,3039,13],"total_duration":77062695133,"load_duration":73682358729,"prompt_eval_count":25,"prompt_eval_duration":34378000,"eval_count":222,"eval_duration":3343166000} |
Thank you, I broke down and installed ollama/set it up and verified it works with the chat and summary endpoints. Now Ollama works. As of Commit: #353 (Uses the OpenAI API endpoint) Thanks again for the heads up about this. |
I am not sure this new issue is related to the original one or not. ERROR:root:Error in enhanced_rag_pipeline: Required inputs (['token_type_ids']) are missing from input feed (['input_ids', 'attention_mask']). |
It seems it has nothing to do with ollama. Hope it helps you to find the rootcause Thanks |
Can you please file a new issue with the full output of the console while running the application with:
I'm unable to replicate the error you posted, as those error messages aren't in my code, and I don't see that pop up when testing either RAG pipeline with Ollama. |
My system has ollama server running. But I got the following error when I configure tldw to use ollama API
ollama_api_IP = http://127.0.0.1:11434/api/generate
ollama_api_key =
ollama_model = qwen2.5:7b-instruct-q5_K_M
WARNING:root:Summarization with ollama failed: Error: Invalid API Name ollama
WARNING:root:Failed to generate summary using ollama API
INFO:root:Summarization completed successfully.
ERROR:root:Summarization failed.
Any idea why this happens?
Thanks,
The text was updated successfully, but these errors were encountered: