Attach images to conversations as URLs

[krschacht]

No description provided.

[jlvallelonga]

from: https://docs.anthropic.com/en/docs/build-with-claude/vision#can-claude-read-image-urls

so just openai for now

[jlvallelonga]

fixes this: #398 (comment)

the problem is that the ActiveStorage::Current.url_options are set only during a request lifecycle (with ActiveStorage::SetCurrent) and workers don't run within the context of a request. This fix also allows you to get the url from within the rails console.

see: https://github.com/rails/rails/blob/main/activestorage/app/controllers/concerns/active_storage/set_current.rb
and: rails/rails#40855 (comment)
and: rails/rails#42520

[krschacht]

on dang, nice debugging on this. i couldn’t figure out what the difference was within the worker

[krschacht]

Oh dang, I didn’t realize that Claude actually requires base64 encoding. I thought when I saw in the OpenAI docs that they suggest passing in the URLs that this meant it was the preferred method and I just assumed that would be the same for the other models.

That decreases the priority of this then. If it’s not too much more work, I think it’s still worth getting this for openai since that’s the most used model. But if you run into any blockers then we can always elect to merge in the improvements you have and defer the rest. I’m just sharing this aloud since I’m not sure where you’re at with this or if you’ve hit a blocker.

[jlvallelonga]

Oh dang, I didn’t realize that Claude actually requires base64 encoding. I thought when I saw in the OpenAI docs that they suggest passing in the URLs that this meant it was the preferred method and I just assumed that would be the same for the other models.

That decreases the priority of this then. If it’s not too much more work, I think it’s still worth getting this for openai since that’s the most used model. But if you run into any blockers then we can always elect to merge in the improvements you have and defer the rest. I’m just sharing this aloud since I’m not sure where you’re at with this or if you’ve hit a blocker.

I think it's working with openai. just need to get these new env vars into github and the prod server (with appropriate values):

      APP_URL_PROTOCOL: "http"
      APP_URL_HOST: "localhost"
      APP_URL_PORT: "3000"

[krschacht]

@jlvallelonga our production setup is using the credentials file. I did a recent commit on the other repo to add that. You can edit it with bin/rails credentials:edit --environment production and it lives in the repo. Only the encryption key is in the production env.

For github actions we should just add env to the rubyonrails.yml file

[jlvallelonga]

ah I just merged this thinking it was approved. Let me know if you want any changes and I can make those in another branch @krschacht

[krschacht]

@jlvallelonga no problem, I'll give it a close look right now

[krschacht]

New PR: #532