Can't run docker without POSTGRES_DATABASE set (using mysql only)

[Skyedra]

Describe the issue

Hello,

I am trying to set up weblate via docker.

I see in the docker doc there are env vars for postgre but not mysql. So I try to set them in settings_override.py, but it does not seem this file is being included yet?

How can I run the docker install with mysql? (I will not use postgre)

Thank you!

I already tried

• I've read and searched the documentation.
• I've searched for similar issues in this repository.

Steps to reproduce the behavior

in docker-compose:

...
 weblate:
    container_name: weblate
    image: weblate/weblate
    tmpfs:
      - /run
      - /tmp
    volumes:
      - /home/weblate/weblate-data:/app/data
      - /var/cache/weblate:/app/cache
    env_file:
      - ./weblateEnvironment
    restart: always
    read_only: true
    ports:
      - "5400:80"
    user: "1003:1003"
...

in /home/weblate/weblate-data/settings-override.py:

import os
import platform
from logging.handlers import SysLogHandler

# I even tried putting garbage input here to force an error, but
# didn't work?  I think this file is not being executed yet?

DATABASES = {
    "default": {
        # Database engine
        "ENGINE": "django.db.backends.mysql",
        # Database name
        "NAME": "weblate",
        # Database user
        "USER": "weblate",
        # Database password
        "PASSWORD": "password-here",
        # Set to empty string for localhost
        "HOST": "host-here",
        # Set to empty string for default
        "PORT": "3306",
        # In case you wish to use additional
        # connection options
        "OPTIONS": {},
    }
}

Expected behavior

No response

Screenshots

No response

Exception traceback

Starting...
Traceback (most recent call last):
  File "/usr/local/bin/weblate", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/usr/local/lib/python3.11/site-packages/weblate/runner.py", line 19, in main
    utility.execute()
  File "/usr/local/lib/python3.11/site-packages/django/core/management/__init__.py", line 382, in execute
    settings.INSTALLED_APPS
  File "/usr/local/lib/python3.11/site-packages/django/conf/__init__.py", line 102, in __getattr__
    self._setup(name)
  File "/usr/local/lib/python3.11/site-packages/django/conf/__init__.py", line 89, in _setup
    self._wrapped = Settings(settings_module)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/conf/__init__.py", line 217, in __init__
    mod = importlib.import_module(self.SETTINGS_MODULE)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/usr/local/lib/python3.11/site-packages/weblate/settings_docker.py", line 54, in <module>
    "NAME": get_env_str("POSTGRES_DATABASE", required=True),
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/weblate/utils/environment.py", line 30, in get_env_str
    raise ValueError(f"{name} has to be configured!")
ValueError: POSTGRES_DATABASE has to be configured!
Failed to load configuration, please see errors above.

How do you run Weblate?

Docker container

Weblate versions

docker compose doesn't work since it can't start up, but it's docker image ID 95d3c152d7a2 created 10 days ago

Weblate deploy checks

Can't run since it's stuck restarting

Additional context

No response

[github-actions]

Thank you for your report; the issue you have reported has just been fixed.

• In case you see a problem with the fix, please comment on this issue.
• In case you see a similar problem, please open a separate issue.
• If you are happy with the outcome, don’t hesitate to support Weblate by making a donation.

[Skyedra]

Thank you for the quick fix! Can you tell me how I can pull the latest docker build to use it?

I tried updating my docker config to use image: weblate/weblate:bleeding-2023-09-26-e77e8c743c7c8889a0ace49abe65e79699670615 to pull the fix, and I added to my environment file WEBLATE_DATABASES=False and ran:

docker kill weblate
docker rm weblate
root@test:~# docker ps --all | grep weblate   # verified it is completely gone
docker-compose -f docker-compose.yml -f docker-production.yml up -d

Which did pull a new image. But it doesn't seem to respect the new variable:

Starting...
Traceback (most recent call last):
  File "/usr/local/bin/weblate", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/usr/local/lib/python3.11/site-packages/weblate/runner.py", line 19, in main
    utility.execute()
  File "/usr/local/lib/python3.11/site-packages/django/core/management/__init__.py", line 382, in execute
    settings.INSTALLED_APPS
  File "/usr/local/lib/python3.11/site-packages/django/conf/__init__.py", line 102, in __getattr__
    self._setup(name)
  File "/usr/local/lib/python3.11/site-packages/django/conf/__init__.py", line 89, in _setup
    self._wrapped = Settings(settings_module)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/conf/__init__.py", line 217, in __init__
    mod = importlib.import_module(self.SETTINGS_MODULE)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/usr/local/lib/python3.11/site-packages/weblate/settings_docker.py", line 54, in <module>
    "NAME": get_env_str("POSTGRES_DATABASE", required=True),
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/weblate/utils/environment.py", line 31, in get_env_str
    raise ValueError(f"{name} has to be configured!")
ValueError: POSTGRES_DATABASE has to be configured!
Failed to load configuration, please see errors above.

And indeed, line 54 is in the old file, it should be 55 in the new file. So it seems the fix isn't included in that docker build, despite the github sha's matching...?

[nijel]

You need to wait for the tomorrow build, bleeding containers are typically updated once a day.

[Skyedra]

Oh ok, thank you!

[Skyedra]

Alright, so I have a new error now 😅

I am now using image: weblate/weblate:bleeding-2023-09-27-e48d8c71a21292fc9b650aef027de407a5341e91 and the WEBLATE_DATABASES=False env var.

I can see it is now pulling in the settings override file, which is as I posted above. (I temporarily uncommented the comment to intentionally trigger an error, so I can be sure it is executing the override file above. I then reverted the comment so it would execute OK)

But, even though it loads the override file, it is still trying to do something with postgres:

Starting...
System check identified no issues (1 silenced).
psql: error: local user with ID 1003 does not exist
Postgres at database is unavailable - retrying 30

1003 uid is user I am running the docker instance as (user: "1003:1003") because it matches the host user ('weblate') which has storage permissions to the host volume (/home/weblate/weblate-data). I am not sure why it is trying to run postgres in the first place, though.

[nijel]

WeblateOrg/docker#1980 might fix this.

[Skyedra]

Cool, I can give it a test once it's dockered up somewhere and report back to you :)

[Skyedra]

Thanks, that got me through initial database setup using weblate/weblate:bleeding-2023-09-27-b75a6fee2daa9f0607f3f1a7e370b331f4711b06!

Next, I got an error:

mkdir: cannot create directory ‘/run’: Permission denied

But I was able to resolve it by following this tip and modify my docker-compose

tmpfs:
      #- /run
      - /run:mode=770,uid=1003,gid=1003

However, then I got into this problem:

410 static files copied to '/app/cache/static'.
Compressing... done
Compressed 10 block(s) from 732 template(s) for 6 context(s).
Auto-scaled to 2 processes, adjust by setting WEBLATE_WORKERS
2023-09-28 03:34:01,962 WARN No file matches via include "/etc/supervisor/conf.d/*.conf"
2023-09-28 03:34:01,969 INFO RPC interface 'supervisor' initialized
2023-09-28 03:34:01,969 INFO supervisord started with pid 1
2023-09-28 03:34:02,972 INFO spawned: 'stdout' with pid 302
2023-09-28 03:34:04,019 INFO success: stdout entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)

And indeed, conf.d does not have relevant permissions for my 1003 uid:

root@test:~# docker compose exec weblate ls /etc/supervisor/ -lth
total 8.0K
drwxrwx--- 1 root root 4.0K Sep 27 15:33 conf.d
-rw-r--r-- 1 root root 1.4K Sep 27 15:33 supervisord.conf

so I ended up configuring only UID and not GID. Though it seems worrying to run in the root group, I saw some references that it's not uncommon in docker, and that resolved both above issues:

tmpfs:
      - /run
      #- /run:mode=770,uid=1003,gid=1003
      - /tmp
    volumes:
      - /home/weblate/weblate-data:/app/data
      - /var/cache/weblate:/app/cache
    env_file:
      - ./weblateEnvironment
    restart: always
    read_only: true
    ports:
      - "5400:8080"
    user: "1003"   # was previously "1003:1003"

That allowed me to get up and running and things seem to be mostly working (able to pull a github repo.) However, when trying to set up an SSH key, I get the error

Could not generate key:

And unfortunately I don't seem to get anything relevant in the logs. Directory permissions seem consistent with everything else, so it should have permission to write in there:

root@test:~# ls -lth /home/weblate/weblate-data/ | grep ssh
drwxr-xr-x 3 weblate root 4.0K Sep 28 05:29 ssh

root@test:~# ls -lth /home/weblate/weblate-data/ssh
total 8.0K
drwxr-xr-x 2 weblate root 4.0K Sep 28 05:29 bin-hexcodewashere
-rw-r--r-- 1 weblate root   57 Sep 28 05:29 config

I even tried blowing the folder up and setting full 777 permission, but doesn't seem to be related.

Any ideas how I can resolve the ssh key gen error so I can push to github?

[Skyedra]

I ended up giving up on running on a non-1000 UID and that allowed me to work around the ssh key issue.

IMO it'd be both cleaner for host filesystem mapping and also enhanced security if UID parameter was supported in the docker version.

[nijel]

The error message should be better with d857a43.

Running under a different user has never been tested, so there might be some glitches in that, I've created WeblateOrg/docker#1988 to track this.

[Skyedra]

Great, thank you :)