Doesn't start on WSL2

[m-mohr]

Problem description

I'm trying to run this on Ubuntu through WSL2 (Windows Subsystem for Linux, v2):

mm@Matthias-Workstation:/mnt/c/Dev/eoAPI$ docker compose up
WARN[0000] The "AWS_ACCESS_KEY_ID" variable is not set. Defaulting to a blank string. 
WARN[0000] The "AWS_SECRET_ACCESS_KEY" variable is not set. Defaulting to a blank string.
[+] Running 5/5
 ✔ Network eoapi-network   Created                                                                                                   0.0s 
 ✔ Container eoapi.db      Created                                                                                                   0.0s 
 ✔ Container eoapi.raster  Created                                                                                                   0.6s 
 ✔ Container eoapi.vector  Created                                                                                                   0.6s 
 ✔ Container eoapi.stac    Created                                                                                                   0.0s 
Attaching to eoapi.db, eoapi.raster, eoapi.stac, eoapi.vector
eoapi.db      | chmod: changing permissions of '/var/lib/postgresql/data': Operation not permitted
eoapi.db      | The files belonging to this database system will be owned by user "postgres".
eoapi.db      | This user must also own the server process.
eoapi.db      | 
eoapi.db      | The database cluster will be initialized with locale "en_US.utf8".
eoapi.db      | The default database encoding has accordingly been set to "UTF8".
eoapi.db      | The default text search configuration will be set to "english".
eoapi.db      | 
eoapi.db      | Data page checksums are disabled.
eoapi.db      | 
eoapi.db      | initdb: error: could not change permissions of directory "/var/lib/postgresql/data": Operation not permitted
eoapi.db      | fixing permissions on existing directory /var/lib/postgresql/data ...
eoapi.db exited with code 1
eoapi.vector  | wait-for-it.sh: waiting 120 seconds for database:5432
eoapi.raster  | wait-for-it.sh: waiting 120 seconds for database:5432
eoapi.stac    | wait-for-it.sh: waiting 120 seconds for database:5432
eoapi.vector  | wait-for-it.sh: timeout occurred after waiting 120 seconds for database:5432
eoapi.raster  | wait-for-it.sh: timeout occurred after waiting 120 seconds for database:5432
eoapi.stac    | wait-for-it.sh: timeout occurred after waiting 120 seconds for database:5432
eoapi.vector exited with code 124
eoapi.raster exited with code 124
eoapi.stac exited with code 124

Expected Output

eoAPI starts. I assume this might be related to https://stackoverflow.com/questions/44878062/initdb-could-not-change-permissions-of-directory-on-postgresql-container

Environment Information

OS: Windows 10 -> WSL2 -> Ubunto 22.04.2

It starts without issues on Windows itself.

[vincentsarago]

@m-mohr is it working if you remove https://github.com/developmentseed/eoAPI/blob/main/docker-compose.yml#L145-L146 ?

I'm not familiar with windows/wsl environment so I can't really try.

It's pretty hard to write a docker compose file (with volumes) which will work in all environments

[m-mohr]

Yeah, indeed, that makes it work.

[tylere]

I encountered a similar issue when trying to run on:
MacBook Pro, Apple M1 Max, macOS Sonoma 14.3.1

Running docker compose up with an unmodified docker-compose.yml results in the following eoapi-test-database-1 log:

2024-02-28 14:10:35 
2024-02-28 14:10:35 PostgreSQL Database directory appears to contain a database; Skipping initialization
2024-02-28 14:10:35 
2024-02-28 14:10:35 2024-02-28 22:10:35.938 UTC [1] LOG:  starting PostgreSQL 15.3 (Debian 15.3-1.pgdg110+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
2024-02-28 14:10:35 2024-02-28 22:10:35.940 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
2024-02-28 14:10:35 2024-02-28 22:10:35.940 UTC [1] LOG:  listening on IPv6 address "::", port 5432
2024-02-28 14:10:35 2024-02-28 22:10:35.944 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2024-02-28 14:10:35 2024-02-28 22:10:35.951 UTC [140] LOG:  database system was shut down at 2024-02-28 22:07:00 UTC
2024-02-28 14:10:35 2024-02-28 22:10:35.964 UTC [1] LOG:  database system is ready to accept connections
2024-02-28 14:10:38 2024-02-28 22:10:38.720 UTC [147] FATAL:  database "postgis" does not exist
2024-02-28 14:10:39 2024-02-28 22:10:39.016 UTC [148] FATAL:  database "postgis" does not exist
...

Commenting out https://github.com/developmentseed/eoAPI/blob/main/docker-compose.yml#L145-L146 allows docker compose up to startup with the following eoapi-test-database-1 log. However, it is not clear why the database shut itself down.

2024-02-28 14:20:47 The files belonging to this database system will be owned by user "postgres".
2024-02-28 14:20:47 This user must also own the server process.
2024-02-28 14:20:47 
2024-02-28 14:20:47 The database cluster will be initialized with locale "en_US.utf8".
2024-02-28 14:20:47 The default database encoding has accordingly been set to "UTF8".
2024-02-28 14:20:47 The default text search configuration will be set to "english".
2024-02-28 14:20:47 
2024-02-28 14:20:47 Data page checksums are disabled.
2024-02-28 14:20:47 
2024-02-28 14:20:47 fixing permissions on existing directory /var/lib/postgresql/data ... ok
2024-02-28 14:20:47 creating subdirectories ... ok
2024-02-28 14:20:47 selecting dynamic shared memory implementation ... posix
2024-02-28 14:20:47 selecting default max_connections ... 100
2024-02-28 14:20:47 selecting default shared_buffers ... 128MB
2024-02-28 14:20:47 selecting default time zone ... Etc/UTC
2024-02-28 14:20:47 creating configuration files ... ok
2024-02-28 14:20:48 running bootstrap script ... ok
2024-02-28 14:20:48 performing post-bootstrap initialization ... ok
2024-02-28 14:20:48 syncing data to disk ... ok
2024-02-28 14:20:48 
2024-02-28 14:20:48 
2024-02-28 14:20:48 Success. You can now start the database server using:
2024-02-28 14:20:48 
2024-02-28 14:20:48     pg_ctl -D /var/lib/postgresql/data -l logfile start
2024-02-28 14:20:48 
2024-02-28 14:20:48 waiting for server to start....2024-02-28 22:20:48.666 UTC [238] LOG:  starting PostgreSQL 15.3 (Debian 15.3-1.pgdg110+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
2024-02-28 14:20:48 2024-02-28 22:20:48.667 UTC [238] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2024-02-28 14:20:48 2024-02-28 22:20:48.673 UTC [251] LOG:  database system was shut down at 2024-02-28 22:20:48 UTC
2024-02-28 14:20:48 2024-02-28 22:20:48.686 UTC [238] LOG:  database system is ready to accept connections
2024-02-28 14:20:48  done
2024-02-28 14:20:48 server started
2024-02-28 14:20:49 CREATE DATABASE
2024-02-28 14:20:49 
2024-02-28 14:20:49 
2024-02-28 14:20:49 /usr/local/bin/docker-entrypoint.sh: running /docker-entrypoint-initdb.d/999_initpgstac.sh
2024-02-28 14:20:49  pgstac_admin_owns 
2024-02-28 14:20:49 -------------------
2024-02-28 14:20:49  
2024-02-28 14:20:49 (1 row)
2024-02-28 14:20:49 
2024-02-28 14:20:50  update_partition_stats_q 
2024-02-28 14:20:50 --------------------------
2024-02-28 14:20:50 (0 rows)
2024-02-28 14:20:50 
2024-02-28 14:20:50  set_version 
2024-02-28 14:20:50 -------------
2024-02-28 14:20:50  0.7.10
2024-02-28 14:20:50 (1 row)
2024-02-28 14:20:50 
2024-02-28 14:20:50 
2024-02-28 14:20:50 2024-02-28 22:20:50.053 UTC [238] LOG:  received fast shutdown request
2024-02-28 14:20:50 2024-02-28 22:20:50.054 UTC [238] LOG:  aborting any active transactions
2024-02-28 14:20:50 waiting for server to shut down....2024-02-28 22:20:50.058 UTC [238] LOG:  background worker "logical replication launcher" (PID 254) exited with exit code 1
2024-02-28 14:20:48 initdb: warning: enabling "trust" authentication for local connections
2024-02-28 14:20:48 initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.
2024-02-28 14:20:50 2024-02-28 22:20:50.221 UTC [1] LOG:  starting PostgreSQL 15.3 (Debian 15.3-1.pgdg110+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
2024-02-28 14:20:50 2024-02-28 22:20:50.224 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
2024-02-28 14:20:50 2024-02-28 22:20:50.224 UTC [1] LOG:  listening on IPv6 address "::", port 5432
2024-02-28 14:20:50 2024-02-28 22:20:50.225 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2024-02-28 14:20:50 2024-02-28 22:20:50.230 UTC [318] LOG:  database system was shut down at 2024-02-28 22:20:50 UTC
2024-02-28 14:20:50 2024-02-28 22:20:50.240 UTC [1] LOG:  database system is ready to accept connections
2024-02-28 14:20:50 2024-02-28 22:20:50.058 UTC [249] LOG:  shutting down
2024-02-28 14:20:50 2024-02-28 22:20:50.059 UTC [249] LOG:  checkpoint starting: shutdown immediate
2024-02-28 14:20:50 2024-02-28 22:20:50.093 UTC [249] LOG:  checkpoint complete: wrote 2085 buffers (12.7%); 0 WAL file(s) added, 0 removed, 1 recycled; write=0.012 s, sync=0.016 s, total=0.034 s; sync files=290, longest=0.006 s, average=0.001 s; distance=15177 kB, estimate=15177 kB
2024-02-28 14:20:50 2024-02-28 22:20:50.100 UTC [238] LOG:  database system is shut down
2024-02-28 14:20:50  done
2024-02-28 14:20:50 server stopped
2024-02-28 14:20:50 

[vincentsarago]

2024-02-28 14:10:35 PostgreSQL Database directory appears to contain a database; Skipping initialization

This is kinda weird because it could mean that it found a .pgdata directory locally with an almost initialized database (without postgis).

However, it is not clear why the database shut itself down.

I think it's fine, the db should be running

[zacharyDez]

@vincentsarago @alukach @emileten ; should we set up a separate version for docker-compose to avoid users hitting an error when onboarding? Is there not a way to have this in a single file?

[vincentsarago]

Well if we remove the local volume, the users will lose the ability to start/stop the same db and will have to do ingestion each time they start the application. I think this could be covered by some documentation.

I only see one issue right now and it's: How to write a volume path for window (and supported by other system)

[emileten]

In the Stack Overflow thread mentioned above, one answer mentions using named volumes to fix the issue. Maybe we could try this on WSL2? Just checked it does not break things on macOS. In the docker-compose.yml configuration it's just the volumes entry that changes, from - ./.pgdata:/var/lib/postgresql/data to pgdata:/var/lib/postgresql/data, and adding a volume spec at the bottom

volumes:
  pgdata: