Refresh lang models for all users #536

drnic · 2024-11-06T07:39:15Z

In this PR, we provide a rake task rails models:import to create/update LanguageModel for all User from the models.yaml file in the root of the repo.

This models.yaml file can be manually modified, or can be regenerated from the first User in the local DB with rails models:export

revert local ffi 1.17.0 change needed for macos dev
update create_initial_assistants_etc
going to wait until Dynamic "best" Language Model #538 merged

Update it from local DB with: rails models:export

krschacht

I know you're still working on this, but I did a quick pass through since I had some time and shared a few thoughts. It's looking good! Thanks so much for tackling this!

krschacht · 2024-11-07T03:20:52Z

config/options.yml

@@ -51,6 +51,7 @@ shared:
    default_openai_key: <%= ENV["DEFAULT_OPENAI_KEY"] %>
    default_anthropic_key: <%= ENV["DEFAULT_ANTHROPIC_KEY"] %>
    default_groq_key: <%= ENV["DEFAULT_GROQ_KEY"] %>
+    lang_models_url: <%= ENV.fetch("LANG_MODELS_URL", "https://raw.githubusercontent.com/AllYourBot/hostedgpt/refs/heads/main/models.json") %>


For all new features, I've been trying to support them within options.yml. I've really come to like the rails pattern of getting to store secrets and configs in the bin/rails credentials:edit. Is that too unconventional?

To continue support of it I think this line would need to be:
<%= ENV["LANG_MODELS_URL"] || default_to(:lang_models_url) || "https://raw.githubusercontent.com/AllYourBot/hostedgpt/refs/heads/main/models.json" %>

Actually, this idea of a remote URL didn't make it into this PR. Removing.

krschacht · 2024-11-07T03:22:00Z

models.yaml

+  supports_images: true
+  supports_tools: true
+  input_token_cost_cents: '0.00025'
+  output_token_cost_cents: '0.001'


Just like you left the quotes off the strings, I'm pretty sure you don't need quotes around floats in a yml file

TIL a while back - if you convert BigDecimal (.to_d) -> YAML, it stores it as a string so it can record large floats; and if its a normal float (.to_f) -> YAML, then you get a normal float value.

So these strings are perhaps because the original "floating point number" was a BigDecimal

From schema.rb:

t.decimal "input_token_cost_cents", precision: 30, scale: 15 t.decimal "output_token_cost_cents", precision: 30, scale: 15

(note, I didn't write this YAML file, it was generated from the rake tasks)

oh interesting, that’s a good lesson for me

krschacht · 2024-11-07T03:25:12Z

app/models/language_model.rb

@@ -47,11 +49,51 @@ def created_by_current_user?
    user == Current.user
  end

+  def api_service_name
+    api_service.name
+  end


How about delegate :name, to: :api_service, prefix: true and then pull it up to the other delegate a few lines above

app/models/language_model.rb

lib/tasks/models.rake

krschacht · 2024-11-07T03:26:28Z

README.md

@@ -29,14 +29,14 @@ This project is led by an experienced rails developer, but I'm actively looking
 - [Deploy the app on Heroku](#deploy-the-app-on-heroku)
 - [Deploy on your own server](#deploy-on-your-own-server)
 - [Configure optional features](#configure-optional-features)
-  - [Give assistant access to your Google apps](#configuring-google-tools)
+  - [Configuring Google Tools](#configuring-google-tools)


Thanks for catching the broken anchors below. But on this change, since our readme has gotten so long, I like the more descriptive "Give assistant access to your Google apps" on this one. It gives people a vague idea of what's possible whereas I don't the more succinct "Configuring Google Tools" hints at enough.

As always, sorry if my editor's markdown formatter/linter "fixes" things in a way that aren't to be fixed. I'll try to keep any eye out for any regressions in future PRs.

krschacht · 2024-11-07T03:30:45Z

README.md

+
+```plain
+rails models:export[tmp/models.json]
+```


Thanks for adding this explanation. I am wondering though if we can add some smart defaults so that a new user can safely ignore this and still get reasonable behavior. Can we make it so that bin/dev also runs models:import on startup? It already runs migrations and bundle upon startup so I've been leaning into it.

We have a fair number of people using this app who are only mildly technical, or are technical but don't know rails, so it's been nice to keep the day-to-day running of it super simple.

But actually, now that I write all this, I don't think people will reliably restart their server. It's not like fly, render, heroku do an automatic nightly restart or anything. Now I'm on the fence... I think I still learn towards doing it on startup, since it's a bit of a catch all. But we'll also add a button to the Settings at some point. And maybe later on we could even add a nightly sync of models:import in the queue. (Scope creep for now, I'm just thinking out loud.)

krschacht · 2024-11-07T03:31:21Z

app/models/language_model.rb

@@ -4,12 +4,14 @@ class LanguageModel < ApplicationRecord
  BEST_CLAUDE = "claude-best"
  BEST_GROQ = "groq-best"

+  # TODO: infer these from models.yaml


Agreed, it's nice to eliminate this config

krschacht · 2024-11-07T03:33:05Z

app/models/language_model.rb

  BEST_MODELS = {
    BEST_GPT => "gpt-4o-2024-08-06",
    BEST_CLAUDE => "claude-3-5-sonnet-20240620",
    BEST_GROQ => "llama3-70b-8192",
  }

+  # TODO: what are these for?


The app now tracks your usage $. Currently, when you click "..." on a conversation you see this:

Soon I want to add it so that when you click on your profile pic (button left) then it shows something equivalent for the month-to-date across all conversations.

But... (note to self) these two things should probably be moved to a separate concern of their own. They're not so important as to be so prominent in this model.

drnic · 2024-11-17T23:26:07Z

PR being recreated off main in #556

drnic added 8 commits November 7, 2024 13:11

I need ffi 1.17 locally

67418de

Introduce Setting.lang_models_url via $LANG_MODELS_URL

36f300b

LanguageModel.export_to_file(path:)

e7a1aec

Communal list of models/prices in models.yaml

3b22586

Update it from local DB with: rails models:export

Import new or updated models

5b47342

Document importing models

a4d7cac

models.yaml has models nested under models: key

dc40e91

Revert ffi bump

d1c84bb

drnic force-pushed the refresh-lang-models branch from 3e3f195 to d1c84bb Compare November 7, 2024 03:12

krschacht requested changes Nov 7, 2024

View reviewed changes

drnic closed this Nov 17, 2024

drnic mentioned this pull request Nov 18, 2024

Import LanguageModels from models.yaml #556

Open

2 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Refresh lang models for all users #536

Refresh lang models for all users #536

drnic commented Nov 6, 2024 •

edited

Loading

krschacht left a comment

krschacht Nov 7, 2024

drnic Nov 17, 2024

krschacht Nov 7, 2024

drnic Nov 17, 2024

drnic Nov 17, 2024

krschacht Nov 19, 2024

krschacht Nov 7, 2024

krschacht Nov 7, 2024

drnic Nov 17, 2024

krschacht Nov 7, 2024

krschacht Nov 7, 2024

krschacht Nov 7, 2024

drnic commented Nov 17, 2024

Refresh lang models for all users #536

Refresh lang models for all users #536

Conversation

drnic commented Nov 6, 2024 • edited Loading

krschacht left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

drnic commented Nov 17, 2024

drnic commented Nov 6, 2024 •

edited

Loading