Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Import LanguageModels from models.yaml #556

Open
wants to merge 7 commits into
base: main
Choose a base branch
from

Conversation

drnic
Copy link
Contributor

@drnic drnic commented Nov 17, 2024

  • Update app/models/user/registerable.rb to import models
  • Clean out the migrations of explicit data creation

This PR does not update, remove, nor introduce new models.

@drnic drnic mentioned this pull request Nov 17, 2024
3 tasks
@drnic drnic marked this pull request as ready for review November 18, 2024 01:03
@drnic
Copy link
Contributor Author

drnic commented Nov 18, 2024

@krschacht this PR replaces #536

Copy link
Contributor

@krschacht krschacht left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I pulled it locally, ran the import script, and all works. I did a little poking around console and reviewed the code closely, just a few small comments.

The one thing I'm still a bit unsure of is the auto-updating. With the old approach of adding a migration for each new model is annoying, but one bonus is that everyone who forks off hostedgpt gets their branch auto-updated automatically each night (provided they haven't diverged from main). When their fork's main updates with the new migration, heroku and render automatically re-deploy (I'm not sure about Fly). This means that a lot of people have been noticing new models & assistants automatically appearing within their apps without doing anything.

It's not critical that we continue to support that, but is there an easy way? All 3 environments are configured to have a release step which runs bin/rails db:prepare so can we connect the models:import so it auto-runs after db:prepare is run?

To import from another file, similarly provide the path as an argument:

```plain
rails models:import[tmp/models.json]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see that export checks the file extension for json vs yaml, but it does not look like import supports json so we should change this documented example to be yaml or add support for json (but I think it's also fine for us to support json export but not import)

@@ -0,0 +1,55 @@
module LanguageModel::Export
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On the file path, I've always followed the convention that the concerns/ directory is only used for concerns that span multiple models. I just lifted that from what 37signals does in their code.

So to keep consistent with the rest of the models concerns, I'd move this to app/models/language_model/export.rb

@@ -81,6 +81,72 @@ class LanguageModelTest < ActiveSupport::TestCase
assert list.include?("alpaca:medium")
end

# Functionality in LanguageModel::Export concern
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These tests look good, but I've been putting tests for a concern into the corresponding test file so that would be: test/models/language_model/export_test.rb. Then you don't need this comment

attrs = super(options)
attrs["api_service_name"] = api_service_name if options[:only].include?(:api_service_name)
attrs
end
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I got an exception in rails console when I did LanguageModel.last.as_json because I left off only. We could just add & before, but it technically won't work if we do ...as_json(only: :api_service_name)

We don't necessarily need to support all those different edge cases, but out of curiosity I was asking chatgpt if there was a more idiomatic way to support delegate attributes with as_json since this is a common pattern. I don't think there is, but ChatGPT wrote this code as a suggestion (I didn't test this):

  def as_json(options = {})
    super(options).tap do |hash|
      hash['api_service_name'] = api_service_name if include_api_service_name?(options)
    end
  end

  private

  def include_api_service_name?(options)
    options = options.with_indifferent_access
    return true unless options[:only] || options[:except]

    only = Array(options[:only])
    except = Array(options[:except])

    (only.empty? || only.include?('api_service_name')) && !except.include?('api_service_name')
  end

@@ -1,5 +1,7 @@
# We don"t care about large or not
class LanguageModel < ApplicationRecord
include LanguageModel::Export
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When you move the concern file, this can be updated to include Export

@@ -0,0 +1,254 @@
---
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rails convention seems to be .yml rather than .yaml

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants