Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

slurmdbd process does not create pid file #76

Open
jamesbeedy opened this issue Jan 29, 2025 · 0 comments · May be fixed by charmed-hpc/hpc-libs#64
Open

slurmdbd process does not create pid file #76

jamesbeedy opened this issue Jan 29, 2025 · 0 comments · May be fixed by charmed-hpc/hpc-libs#64

Comments

@jamesbeedy
Copy link
Contributor

Bug Description

We observe an error in slurmdbd log due to not being able to create the pid file.

$ sudo service slurmdbd status
● slurmdbd.service - Slurm DBD accounting daemon
     Loaded: loaded (/usr/lib/systemd/system/slurmdbd.service; disabled; preset: enabled)
     Active: active (running) since Wed 2025-01-29 19:14:53 UTC; 2min 32s ago
       Docs: man:slurmdbd(8)
   Main PID: 22572 (slurmdbd)
      Tasks: 6
     Memory: 4.4M (peak: 4.9M)
        CPU: 503ms
     CGroup: /system.slice/slurmdbd.service
             └─22572 /usr/sbin/slurmdbd -D -s

Jan 29 19:14:53 juju-0069f3-0 systemd[1]: Started slurmdbd.service - Slurm DBD accounting daemon.
Jan 29 19:14:53 juju-0069f3-0 (slurmdbd)[22572]: slurmdbd.service: Referenced but unset environment variable evaluates to an empty string: SLURMDBD_OPTIONS
Jan 29 19:14:53 juju-0069f3-0 slurmdbd[22572]: slurmdbd: error: Unable to open pidfile `/var/run/slurmdbd.pid': Permission denied
Jan 29 19:14:53 juju-0069f3-0 slurmdbd[22572]: slurmdbd: Not running as root. Can't drop supplementary groups
Jan 29 19:14:53 juju-0069f3-0 slurmdbd[22572]: slurmdbd: accounting_storage/as_mysql: _check_mysql_concat_is_sane: MySQL server version is: 8.0.39-0ubuntu0.22.0>
Jan 29 19:14:53 juju-0069f3-0 slurmdbd[22572]: slurmdbd: error: Database settings not recommended values: innodb_lock_wait_timeout
Jan 29 19:14:54 juju-0069f3-0 slurmdbd[22572]: slurmdbd: error: mysql_query failed: 1193 Unknown system variable 'wsrep_on'
Jan 29 19:14:54 juju-0069f3-0 slurmdbd[22572]: select @@wsrep_on;
Jan 29 19:14:54 juju-0069f3-0 slurmdbd[22572]: slurmdbd: error: mysql_db_get_var_str: null result from query `select @@wsrep_on;`
Jan 29 19:14:54 juju-0069f3-0 slurmdbd[22572]: slurmdbd: slurmdbd version 23.11.4 started

To Reproduce

deploy charms from edge

Environment

na

Relevant log output

$ sudo cat /var/log/slurm/slurmdbd.log 
[2025-01-29T19:14:53.490] error: Unable to open pidfile `/var/run/slurmdbd.pid': Permission denied
[2025-01-29T19:14:53.490] Not running as root. Can't drop supplementary groups
[2025-01-29T19:14:53.539] accounting_storage/as_mysql: _check_mysql_concat_is_sane: MySQL server version is: 8.0.39-0ubuntu0.22.04.1
[2025-01-29T19:14:53.540] error: Database settings not recommended values: innodb_lock_wait_timeout
[2025-01-29T19:14:54.147] error: mysql_query failed: 1193 Unknown system variable 'wsrep_on'
select @@wsrep_on;
[2025-01-29T19:14:54.147] error: mysql_db_get_var_str: null result from query `select @@wsrep_on;`
[2025-01-29T19:14:54.191] slurmdbd version 23.11.4 started
[2025-01-29T19:18:28.229] Terminate signal (SIGINT or SIGTERM) received
[2025-01-29T19:18:28.292] error: Unable to open pidfile `/var/run/slurmdbd.pid': Permission denied
[2025-01-29T19:18:28.292] Not running as root. Can't drop supplementary groups
[2025-01-29T19:18:28.327] accounting_storage/as_mysql: _check_mysql_concat_is_sane: MySQL server version is: 8.0.39-0ubuntu0.22.04.1
[2025-01-29T19:18:28.327] error: Database settings not recommended values: innodb_lock_wait_timeout
[2025-01-29T19:18:28.472] error: mysql_query failed: 1193 Unknown system variable 'wsrep_on'
select @@wsrep_on;
[2025-01-29T19:18:28.472] error: mysql_db_get_var_str: null result from query `select @@wsrep_on;`
[2025-01-29T19:18:28.507] slurmdbd version 23.11.4 started
[2025-01-29T19:18:32.037] Terminate signal (SIGINT or SIGTERM) received
[2025-01-29T19:18:32.100] error: Unable to open pidfile `/var/run/slurmdbd.pid': Permission denied
[2025-01-29T19:18:32.100] Not running as root. Can't drop supplementary groups
[2025-01-29T19:18:32.135] accounting_storage/as_mysql: _check_mysql_concat_is_sane: MySQL server version is: 8.0.39-0ubuntu0.22.04.1
[2025-01-29T19:18:32.136] error: Database settings not recommended values: innodb_lock_wait_timeout
[2025-01-29T19:18:32.282] error: mysql_query failed: 1193 Unknown system variable 'wsrep_on'
select @@wsrep_on;
[2025-01-29T19:18:32.282] error: mysql_db_get_var_str: null result from query `select @@wsrep_on;`
[2025-01-29T19:18:32.316] slurmdbd version 23.11.4 started

Additional context

No response
jamesbeedy added a commit to jamesbeedy/hpc-libs that referenced this issue Jan 29, 2025
@jamesbeedy
slurmdbd service override
aa978be 
These changes add a slurmdbd service override file that creates
a pid directory and changes the permissions to the slurm user.

Fixes: charmed-hpc/slurm-charms#76
@jamesbeedy jamesbeedy linked a pull request Jan 29, 2025 that will close this issue
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

slurmdbd service override jamesbeedy/hpc-libs
1 participant
@jamesbeedy