LLMStack fails to start when performing migration on fresh install

[acherla]

Describe the bug
LLmstack api service fails to start due to an incorrectly configured migration.py script for creating the backend tables against postgres. This is during a fresh install. Postgres version used is postgres 16

V2 Migration Guide at https://errors.pydantic.dev/2.8/migration/
  warnings.warn(
Operations to perform:
  Apply all migrations: account, admin, apiabstractor, app_store, apps, auth, authtoken, base, contenttypes, datasources, django_rq, flags, jobs, organizations, sessions, sheets, sites, socialaccount
Running migrations:
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 105, in _execute
    return self.cursor.execute(sql, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
psycopg2.errors.UndefinedColumn: column datasources_datasource.read_accessible_by does not exist
LINE 1: ...eated_at", "datasources_datasource"."updated_at", "datasourc...
                                                             ^


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/code/manage.py", line 22, in <module>
    main()
  File "/code/manage.py", line 18, in main
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python3.11/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python3.11/site-packages/django/core/management/__init__.py", line 436, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python3.11/site-packages/django/core/management/base.py", line 413, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python3.11/site-packages/django/core/management/base.py", line 459, in execute
    output = self.handle(*args, **options)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/core/management/base.py", line 107, in wrapper
    res = handle_func(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/core/management/commands/migrate.py", line 356, in handle
    post_migrate_state = executor.migrate(
                         ^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/migrations/executor.py", line 135, in migrate
    state = self._migrate_all_forwards(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/migrations/executor.py", line 167, in _migrate_all_forwards
    state = self.apply_migration(
            ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/migrations/executor.py", line 252, in apply_migration
    state = migration.apply(state, schema_editor)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/migrations/migration.py", line 132, in apply
    operation.database_forwards(
  File "/usr/local/lib/python3.11/site-packages/django/db/migrations/operations/special.py", line 193, in database_forwards
    self.code(from_state.apps, schema_editor)
  File "/code/llmstack/data/migrations/0006_add_pipeline_config.py", line 49, in add_legacy_pipeline_to_datasource_config
    for datasource in datasources:
  File "/usr/local/lib/python3.11/site-packages/django/db/models/query.py", line 400, in __iter__
    self._fetch_all()
  File "/usr/local/lib/python3.11/site-packages/django/db/models/query.py", line 1928, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/models/query.py", line 91, in __iter__
    results = compiler.execute_sql(
              ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/models/sql/compiler.py", line 1562, in execute_sql
    cursor.execute(sql, params)
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 122, in execute
    return super().execute(sql, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 79, in execute
    return self._execute_with_wrappers(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 92, in _execute_with_wrappers
    return executor(sql, params, many, context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 100, in _execute
    with self.db.wrap_database_errors:
  File "/usr/local/lib/python3.11/site-packages/django/db/utils.py", line 91, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 105, in _execute
    return self.cursor.execute(sql, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
django.db.utils.ProgrammingError: column datasources_datasource.read_accessible_by does not exist
LINE 1: ...eated_at", "datasources_datasource"."updated_at", "datasourc...

[d34dman]

I am able to reproduce the error on commit 98ffa1e. I reverted the code back to v0.2.5 and it works.

---

EDIT: I was able to get past the above error.

[gitwittidbit]

How to apply that solution? (I am facing the same issue as OP)

[HWiese1980]

Same here, please tell us how! @d34dman

https://xkcd.com/979/

E: Ah, just saw what you did. Overlooked...

pip install -U llmstack==0.2.5

[HWiese1980]

Doesn't help. Getting a different error message. Giving up, sorry...

[d34dman]

@HWiese1980 , I have stopped using this as the issues in GitHub is being ignored by maintainers. For example, after resolving this issue, I ran into another issue which prevented proper installation in local and created a PR here #290 which didn't get any response.

---

P.S.

I understand the priority of maintainers could be different and they are moving fast and exposing the source of the project is a huge contribution in itself. So me being skeptical to continue using this project is solely my choice.

[devalexandre]

@HWiese1980 , I have stopped using this as the issues in GitHub is being ignored by maintainers. For example, after resolving this issue, I ran into another issue which prevented proper installation in local and created a PR here #290 which didn't get any response.

P.S.

I understand the priority of maintainers could be different and they are moving fast and exposing the source of the project is a huge contribution in itself. So me being skeptical to continue using this project is solely my choice.

why not do a fork?

[d34dman]

why not do a fork?

Am currently evaluating other open-source projects.