Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Import LanguageModels from models.yaml #556

Open
wants to merge 7 commits into
base: main
Choose a base branch
from
+435 −196
Open

Import LanguageModels from models.yaml #556

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
12fe76a
Import LanguageModels from models.yaml
drnic Nov 17, 2024
a48a5d0
Clean up language of export task
drnic Nov 17, 2024
2e21113
Add supports_system_message to export
drnic Nov 17, 2024
dd55367
Add best to export; sort export models
drnic Nov 17, 2024
473c453
New users load models from models.yaml
drnic Nov 18, 2024
3d7523a
Remove unused data-only migrations
drnic Nov 18, 2024
b433bc3
Document new user behavior
drnic Nov 18, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 40 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -368,6 +368,46 @@ HostedGPT requires these services to be running:

Every time you pull new changes down, kill `bin/dev` and then re-run it. This will ensure your local app picks up changes to Gemfile and migrations.

## Language models

Each User has their own list of Language Models they can use.

When a new User is created (when a person registers for the first time), they are initialized with a long list of models. This list is loaded from `models.yaml`.

When an administrator upgrades their deployment of HostedGPT, they can update the available models for all users with a task `rails models:import`.

### Refreshing language models

There is a shared list of known LLM models for OpenAI, Anthropic, and Groq in `models.yaml` and a Rake task to import them into all users:

```plain
rails models:import
```

### Update models.yaml

The `models.yaml` file in the root of the project is used by HostedGPT applications to refresh their local list of models.

To refresh the `models.yaml` file using the models in local DB, run:

```plain
rails models:export
```

### Alternate export file

If you want to export the models in the local DB to another file, either `.json` or `.yaml`, pass in an argument:

```plain
rails models:export[tmp/models.json]
```

To import from another file, similarly provide the path as an argument:

```plain
rails models:import[tmp/models.json]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see that export checks the file extension for json vs yaml, but it does not look like import supports json so we should change this documented example to be yaml or add support for json (but I think it's also fine for us to support json export but not import)

```

### Running tests

If you're set up with Docker you run `docker compose run base rails test`. Note that the system tests, which use a headless browser, are not able to run in Docker. They will be run automatically for you if you create a Pull Request against the project.
Expand Down
55 changes: 55 additions & 0 deletions app/models/concerns/language_model/export.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
module LanguageModel::Export
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On the file path, I've always followed the convention that the concerns/ directory is only used for concerns that span multiple models. I just lifted that from what 37signals does in their code.

So to keep consistent with the rest of the models concerns, I'd move this to app/models/language_model/export.rb

extend ActiveSupport::Concern

def as_json(options = {})
options = options.with_indifferent_access
attrs = super(options)
attrs["api_service_name"] = api_service_name if options[:only].include?(:api_service_name)
attrs
end
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I got an exception in rails console when I did LanguageModel.last.as_json because I left off only. We could just add & before, but it technically won't work if we do ...as_json(only: :api_service_name)

We don't necessarily need to support all those different edge cases, but out of curiosity I was asking chatgpt if there was a more idiomatic way to support delegate attributes with as_json since this is a common pattern. I don't think there is, but ChatGPT wrote this code as a suggestion (I didn't test this):

  def as_json(options = {})
    super(options).tap do |hash|
      hash['api_service_name'] = api_service_name if include_api_service_name?(options)
    end
  end

  private

  def include_api_service_name?(options)
    options = options.with_indifferent_access
    return true unless options[:only] || options[:except]

    only = Array(options[:only])
    except = Array(options[:except])

    (only.empty? || only.include?('api_service_name')) && !except.include?('api_service_name')
  end


DEFAULT_EXPORT_ONLY = %i[
api_name
name
best
api_service_name
supports_images
supports_tools
supports_system_message
input_token_cost_cents
output_token_cost_cents
]

DEFAULT_MODEL_FILE = "models.yaml"

class_methods do
def export_to_file(path: Rails.root.join(LanguageModel::Export::DEFAULT_MODEL_FILE), models:, only: DEFAULT_EXPORT_ONLY)
path = path.to_s
storage = {
"models" => models.as_json(only:)
}
if path.ends_with?(".json")
File.write(path, storage.to_json)
else
File.write(path, storage.to_yaml)
end
end

def import_from_file(path: Rails.root.join(LanguageModel::Export::DEFAULT_MODEL_FILE), users: User.all)
users = Array.wrap(users)
storage = YAML.load_file(path)
models = storage["models"]
models.each do |model|
model = model.with_indifferent_access
users.each do |user|
lm = user.language_models.find_or_initialize_by(api_name: model[:api_name])
lm.api_service = user.api_services.find_by(name: model[:api_service_name]) if model[:api_service_name]
lm.attributes = model.except(:api_service_name)
lm.save!
rescue ActiveRecord::RecordInvalid => e
warn "Failed to import '#{model[:api_name]}': #{e.message} for #{model.inspect}"
end
end
end
end
end
3 changes: 3 additions & 0 deletions app/models/language_model.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# We don"t care about large or not
class LanguageModel < ApplicationRecord
include LanguageModel::Export
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When you move the concern file, this can be updated to include Export


belongs_to :user
belongs_to :api_service

Expand All @@ -18,6 +20,7 @@ class LanguageModel < ApplicationRecord
scope :best_for_api_service, ->(api_service) { where(best: true, api_service: api_service) }

delegate :ai_backend, to: :api_service
delegate :name, to: :api_service, prefix: true

def created_by_current_user?
user == Current.user
Expand Down
63 changes: 1 addition & 62 deletions app/models/user/registerable.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,68 +12,7 @@ def create_initial_assistants_etc
anthropic_api_service = api_services.create!(url: APIService::URL_ANTHROPIC, driver: :anthropic, name: "Anthropic")
groq_api_service = api_services.create!(url: APIService::URL_GROQ, driver: :openai, name: "Groq")

best_models = %w[gpt-4o claude-3-5-sonnet-20240620 llama3-70b-8192]
[
["gpt-4o", "GPT-4o (latest)", true, open_ai_api_service, 250, 1000],
["gpt-4o-2024-08-06", "GPT-4o Omni Multimodal (2024-08-06)", true, open_ai_api_service, 250, 1000],
["gpt-4o-2024-05-13", "GPT-4o Omni Multimodal (2024-05-13)", true, open_ai_api_service, 500, 1500],

["gpt-4-turbo", "GPT-4 Turbo with Vision (latest)", true, open_ai_api_service, 1000, 3000],
["gpt-4-turbo-2024-04-09", "GPT-4 Turbo with Vision (2024-04-09)", true, open_ai_api_service, 1000, 3000],
["gpt-4-turbo-preview", "GPT-4 Turbo Preview", false, open_ai_api_service, 1000, 3000],
["gpt-4-0125-preview", "GPT-4 Turbo Preview (2024-01-25)", false, open_ai_api_service, 1000, 3000],
["gpt-4-1106-preview", "GPT-4 Turbo Preview (2023-11-06)", false, open_ai_api_service, 1000, 3000],
["gpt-4-vision-preview", "GPT-4 Turbo with Vision Preview (2023-11-06)", true, open_ai_api_service, 1000, 3000],
["gpt-4-1106-vision-preview", "GPT-4 Turbo with Vision Preview (2023-11-06)", true, open_ai_api_service, 1000, 3000],

["gpt-4", "GPT-4 (latest)", false, open_ai_api_service, 3000, 6000],
["gpt-4-0613", "GPT-4 Snapshot improved function calling (2023-06-13)", false, open_ai_api_service, 1000, 3000],

["gpt-3.5-turbo", "GPT-3.5 Turbo (latest)", false, open_ai_api_service, 300, 600],
["gpt-3.5-turbo-0125", "GPT-3.5 Turbo (2022-01-25)", false, open_ai_api_service, 50, 150],
["gpt-3.5-turbo-1106", "GPT-3.5 Turbo (2022-11-06)", false, open_ai_api_service, 100, 200],

["claude-3-5-sonnet-20240620", "Claude 3.5 Sonnet (2024-06-20)", true, anthropic_api_service, 300, 1500],
["claude-3-opus-20240229", "Claude 3 Opus (2024-02-29)", true, anthropic_api_service, 1500, 7500],
["claude-3-sonnet-20240229", "Claude 3 Sonnet (2024-02-29)", true, anthropic_api_service, 300, 1500],
["claude-3-haiku-20240307", "Claude 3 Haiku (2024-03-07)", true, anthropic_api_service, 25, 125],
["claude-2.1", "Claude 2.1", false, anthropic_api_service, 800, 2400],
["claude-2.0", "Claude 2.0", false, anthropic_api_service, 800, 2400],
["claude-instant-1.2", "Claude Instant 1.2", false, anthropic_api_service, 80, 240],

["llama3-70b-8192", "Meta Llama 3 70b", false, groq_api_service, 59, 79],
["llama3-8b-8192", "Meta Llama 3 8b", false, groq_api_service, 5, 8],
["mixtral-8x7b-32768", "Mistral 8 7b", false, groq_api_service, 24, 24],
["gemma-7b-it", "Google Gemma 7b", false, groq_api_service, 7, 7],
].each do |api_name, name, supports_images, api_service, input_token_cost_per_million, output_token_cost_per_million|
million = BigDecimal(1_000_000)
input_token_cost_cents = input_token_cost_per_million/million
output_token_cost_cents = output_token_cost_per_million/million

language_models.create!(
api_name:,
api_service:,
name:,
best: best_models.include?(api_name),
supports_tools: true,
supports_system_message: true,
supports_images:,
input_token_cost_cents:,
output_token_cost_cents:,
)
end

# Only these don't support tools:
[
["gpt-3.5-turbo-instruct", "GPT-3.5 Turbo Instruct", false, open_ai_api_service, 150, 200],
["gpt-3.5-turbo-16k-0613", "GPT-3.5 Turbo (2022-06-13)", false, open_ai_api_service, 300, 400],
].each do |api_name, name, supports_images, api_service, input_token_cost_per_million, output_token_cost_per_million|
million = BigDecimal(1_000_000)
input_token_cost_cents = input_token_cost_per_million/million
output_token_cost_cents = output_token_cost_per_million/million

language_models.create!(api_name:, api_service:, name:, supports_tools: false, supports_system_message: false, supports_images:, input_token_cost_cents:, output_token_cost_cents:)
end
LanguageModel.import_from_file(users: [self])

assistants.create! name: "GPT-4o", language_model: language_models.best_for_api_service(open_ai_api_service).first
assistants.create! name: "Claude 3.5 Sonnet", language_model: language_models.best_for_api_service(anthropic_api_service).first
Expand Down
22 changes: 1 addition & 21 deletions db/migrate/20240624100000_add_groq.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,28 +7,8 @@ def up

User.all.find_each do |user|
groq_api_service = user.api_services.create!(url: APIService::URL_GROQ, driver: :openai, name: "Groq")
language_model = user.language_models.create!(position: 3, api_name: LanguageModel::BEST_GROQ, api_service: groq_api_service, name: "Best Open-Source Model", supports_images: false)
user.language_models.where("position >= 3").where.not(id: language_model.id).find_each do |model|
model.update(position: model.position + 1)
end

[
["llama3-70b-8192", "Meta Llama 3 70b", false, groq_api_service],
["llama3-8b-8192", "Meta Llama 3 8b", false, groq_api_service],
["mixtral-8x7b-32768", "Mistral 8 7b", false, groq_api_service],
["gemma-7b-it", "Google Gemma 7b", false, groq_api_service],
].each do |api_name, name, supports_images, api_service|
user.language_models.create!(api_name: api_name, api_service: api_service, name: name, supports_images: supports_images)
end

[ "GPT-3.5", "Claude 3 Opus" ].each do |name|
asst = user.assistants.find_by(name: name)
next if asst.nil?
asst.deleted! if asst.conversations.count == 0
asst.deleted! if asst.conversations.count == 1 && asst.conversations.first.messages.count <= 2
end

user.assistants.create!(name: "Meta Llama 3 70b", language_model: language_model)
user.assistants.create! name: "Meta Llama 3 70b", language_model: language_models.best_for_api_service(groq_api_service).first
end
end

Expand Down
58 changes: 0 additions & 58 deletions db/migrate/20240911114059_update_existing_language_models_with_prices.rb
View file
Open in desktop

This file was deleted.

55 changes: 0 additions & 55 deletions db/migrate/20241022053212_update_model_prices.rb
View file
Open in desktop

This file was deleted.

15 changes: 15 additions & 0 deletions lib/tasks/models.rake
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
namespace :models do
desc "Export language models to a file, defaulting to models.yaml"
task :export, [:path] => :environment do |t, args|
args.with_defaults(path: Rails.root.join(LanguageModel::Export::DEFAULT_MODEL_FILE))
models = User.first.language_models.ordered.not_deleted.includes(:api_service)
LanguageModel.export_to_file(path: args[:path], models:)
end

desc "Import language models to all users from a file, defaulting to models.yaml"
task :import, [:path] => :environment do |t, args|
args.with_defaults(path: Rails.root.join(LanguageModel::Export::DEFAULT_MODEL_FILE))
users = User.all
LanguageModel.import_from_file(path: args[:path], users:)
end
end
Loading
Loading