run-once parameter executes multiple times, does not stop, does not give out info mails #568

phit42 · 2020-06-04T11:31:08Z

phit42
Jun 4, 2020

Starting watchtower with --run-once does not check all images once, but runs it multiple times (until I stopped the container manually).
Additionally, email notifications about updates in info mode are not generated, but instead the message "Running a one time update." is sent out multiple times via email.

Interestingly enough, a weekly schedule ("--schedule "@Weekly" ") does not work either. Every single time, watchtower finds "multiple instances" after the first week, restarts itself and resets the timer, thus never updating the actual images.
(This is why I ran run-once in the first place.)

Only an interval setting appears to work correctly.

Here is the docker-compose.yml

version: '2'
services:
  watchtower:
    container_name: watchtower
    restart: unless-stopped
    image: containrrr/watchtower:latest
    environment:
      - TZ=Europe/Berlin
    env_file:
      - watchtower_notifications_settings.env
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    command: watchtower nzbhydra nzbhydra2 lidarr radarr sonarrv3 --run-once --stop-timeout 300s --cleanup --debug    ### run once

# watchtower container notification settings environment file
WATCHTOWER_NOTIFICATIONS=email
WATCHTOWER_NOTIFICATION_EMAIL_DELAY=2
WATCHTOWER_NOTIFICATIONS_LEVEL=info

(removed the server and user settings)

Watchtower is running on a Synology diskstation. Server: Engine: Version: 18.09.8, API version: 1.39, Go version: go1.11, Git commit: 3a371f3, Built: Fri Mar 13 06:44:35 2020, OS/Arch: linux/amd64

watchtower.txt

Answered by simskij

Jun 12, 2020

As it's not possible to mark nested comments as answers, I'll go ahead and quote @piksel here:

The shoutrrr verify command only verifies a single URL, but when you enter it in your env file it should look like this:
WATCHTOWER_NOTIFICATIONS=shoutrrr
WATCHTOWER_NOTIFICATION_URL="smtp://user:[email protected]:587/[email protected]&[email protected]&subject=Watchtower+updates+on+ pushbullet://o.xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
I just tried it, and it worked (except for the mail server not accepting user and pw as credentials 😁)

View full answer

piksel · 2020-06-05T20:17:20Z

piksel
Jun 5, 2020
Maintainer

The message "Running a one time update" is the only message you get in --run-one mode unless it finds any new images. But your setup seems a bit odd.

When you use --run-once, then after watchtower has done its check, the process will exit. But your docker-compose.yml is set to restart it whenever it exits, so that is probably why you are seeing it being run multiple times. If you want to do a single update, just run it through docker.

You added a log file at the bottom, which scenario was this logged using?
At 2020-06-04T11:46:16+02:00 it found a new sonarrv3 image and updated it, didn't any notifications get sent then?

1 reply

simskij Jun 5, 2020
Maintainer

To add to @piksel's suggestions, I'd strongly recommend that you switch from the legacy email notification service to shoutrrr's email service. We are much better equipped to help with that implementation going forward.

phit42 · 2020-06-08T14:59:38Z

phit42
Jun 8, 2020
Author

Duh, stupid mistake: "restart: unless-stopped". I was re-using my existing yml.

But no, I did not get any notifications about the sonarrv3 image that was updated. (I do though when I am using a timer.)

Would you also have an idea about the weekly schedule topic?

7 replies

phit42 Jun 9, 2020
Author

It works. Only thing that does not is multiple url's for multiple notifications. Any way I try to format the string, I always get the error "first path segment in url cannot contain colon".
Oh and it crashes the watchtower container without any info with an illformed url (I accidently had "" in my subject initially).

# watchtower container notification settings environment file
WATCHTOWER_NOTIFICATIONS=shoutrrr
WATCHTOWER_NOTIFICATION_URL=smtp://user:[email protected]:587/?[email protected]&[email protected]&subject=Watchtower+updates+on+
WATCHTOWER_NOTIFICATION_TEMPLATE={{range .}}{{.Time.Format "2006-01-02 15:04:05"}} ({{.Level}}): {{.Message}}{{println}}{{end}}
WATCHTOWER_NOTIFICATIONS_LEVEL=info

Note: the notification template needs to look slightly different than in the example.

piksel Jun 9, 2020
Maintainer

That error does not seem to fit with the configuration you posted, but you have made a lot of edits. That error suggests that you either have a invalid character in the schema name, or some other syntax error.
You can use the shoutrrr CLI to verify your URL:

go run github.com/containrrr/shoutrrr/cmd/shoutrrr verify "smtp://user:[email protected]:587/[email protected]&[email protected]&subject=Watchtower+updates+on+"

validates fine

simskij Jun 10, 2020
Maintainer

Any way I try to format the string, I always get the error "first path segment in url cannot contain colon".

If I'm not completely mistaken, multiple notification URLs should be configured in the same env var, separated by a single space, due to how viper works.

watchtower/internal/flags/flags.go

Lines 274 to 278 in 9e7dfe7

    
           flags.StringArrayP( 
        
           	"notification-url", 
        
           	"", 
        
           	viper.GetStringSlice("WATCHTOWER_NOTIFICATION_URL"), 
        
           	"The shoutrrr URL to send notifications to")

Is that what you've tried? As in:

  WATCHTOWER_NOTIFICATION_URL=some://notification.url some://other.notification.url

phit42 Jun 10, 2020
Author

Yes, sorry for the many edits. Initially my template did not work but then I found out why.
Re multiple URLs: Thank you for the soutrr CLI tip. Please do also try it. If I put the pushbullet url part first, I get "invalid character " " in host name" in the verification as a pushbullet url consists only of the api key. If I put pushbullet as second url, it is appended to the subject in verify.
go run github.com/containrrr/shoutrrr/cmd/shoutrrr verify "smtp://user:[email protected]:587/[email protected]&[email protected]&subject=Watchtower+updates+on+ pushbullet://o.xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"

piksel Jun 10, 2020
Maintainer

The shoutrrr verify command only verifies a single URL, but when you enter it in your env file it should look like this:

WATCHTOWER_NOTIFICATIONS=shoutrrr
WATCHTOWER_NOTIFICATION_URL="smtp://user:[email protected]:587/[email protected]&[email protected]&subject=Watchtower+updates+on+ pushbullet://o.xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"

I just tried it, and it worked (except for the mail server not accepting user and pw as credentials 😁)

simskij · 2020-06-12T13:35:50Z

simskij
Jun 12, 2020
Maintainer

As it's not possible to mark nested comments as answers, I'll go ahead and quote @piksel here:

The shoutrrr verify command only verifies a single URL, but when you enter it in your env file it should look like this:
WATCHTOWER_NOTIFICATIONS=shoutrrr
WATCHTOWER_NOTIFICATION_URL="smtp://user:[email protected]:587/[email protected]&[email protected]&subject=Watchtower+updates+on+ pushbullet://o.xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
I just tried it, and it worked (except for the mail server not accepting user and pw as credentials 😁)

4 replies

phit42 Jun 12, 2020
Author

Not with me:
Failed to initialize Shoutrrr notifications: error initializing router services: parse \"\\\"smtp://user:[email protected]:587/[email protected]&[email protected]&subject=Watchtower+updates+on+\": first path segment in URL cannot contain colon\n"

mit folgendem Setting:
WATCHTOWER_NOTIFICATION_URL="smtp://user:[email protected]:587/[email protected]&[email protected]&subject=Watchtower+updates+on+ pushbullet://o.xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"

phit42 Jun 13, 2020
Author

I am thinking the double-quotes themselves around the text are the issue, but why?

simskij Jun 13, 2020
Maintainer

That kind of depends. If the env key in your compose file is used as an array, you cant use quotes. If you use it as an object, you may.
That is:

  env:
    - WATCHTOWER_NOTIFICATION_URL="abc"

won't work, but

  env:
    WATCHTOWER_NOTIFICATION_URL: "abc"

  env:
    - WATCHTOWER_NOTIFICATION_URL=abc

  env:
    WATCHTOWER_NOTIFICATION_URL: abc

should all work

phit42 Jun 14, 2020
Author

Thank you again. While it didn't solve it immediately, it pushed me in the right direction.
I am using env_file parameter and apparently an env_file does not support any other syntax than var=abc. Nor does it handle double quotes, it just passes them along. So against all my scripting logic :), multiple url's works in an env_file only with
var=url1 url2
(with just blanks in between and absolutely no quotes around them).
I tried this once before and failed, because I also needed to figure out that any blank in the subject line of an email notification needs to be replaced with a + (now I know so that it doesn't treat them as different url's of course).

phit42 · 2020-07-01T12:08:54Z

phit42
Jul 1, 2020
Author

Hello again, after all my mistakes und knowledge gaps in the configuration, I just wanted to point out that I have re-tested:
a "run-once" in info mode does not notify about updated containers.

Here a log:

time="2020-07-01T13:36:31+02:00" level=info msg="Running a one time update."
time="2020-07-01T13:36:31+02:00" level=debug msg="Checking containers for updated images"
time="2020-07-01T13:36:31+02:00" level=debug msg="Retrieving running containers"
time="2020-07-01T13:36:31+02:00" level=debug msg="Pulling linuxserver/nzbhydra2:latest for /nzbhydra2"
time="2020-07-01T13:36:31+02:00" level=debug msg="No credentials for linuxserver in /config.json"
time="2020-07-01T13:36:31+02:00" level=debug msg="Got image name: linuxserver/nzbhydra2:latest"
time="2020-07-01T13:36:48+02:00" level=debug msg="No new images found for /nzbhydra2"
time="2020-07-01T13:36:48+02:00" level=debug msg="Pulling linuxserver/lidarr:latest for /lidarr"
time="2020-07-01T13:36:48+02:00" level=debug msg="No credentials for linuxserver in /config.json"
time="2020-07-01T13:36:48+02:00" level=debug msg="Got image name: linuxserver/lidarr:latest"
time="2020-07-01T13:37:01+02:00" level=debug msg="No new images found for /lidarr"
time="2020-07-01T13:37:01+02:00" level=debug msg="Pulling linuxserver/sonarr:preview for /sonarrv3"
time="2020-07-01T13:37:01+02:00" level=debug msg="No credentials for linuxserver in /config.json"
time="2020-07-01T13:37:01+02:00" level=debug msg="Got image name: linuxserver/sonarr:preview"
time="2020-07-01T13:37:15+02:00" level=debug msg="No new images found for /sonarrv3"
time="2020-07-01T13:37:15+02:00" level=debug msg="Pulling linuxserver/radarr:latest for /radarr"
time="2020-07-01T13:37:15+02:00" level=debug msg="No credentials for linuxserver in /config.json"
time="2020-07-01T13:37:15+02:00" level=debug msg="Got image name: linuxserver/radarr:latest"
time="2020-07-01T13:37:33+02:00" level=info msg="Found new linuxserver/radarr:latest image (sha256:dd950d99ea517ffc2e0b0ea791178c167a2905e4be878b1dca23c0c1c6c5209d)"
time="2020-07-01T13:37:33+02:00" level=debug msg="Pulling linuxserver/lazylibrarian:latest for /lazylibrarian"
time="2020-07-01T13:37:33+02:00" level=debug msg="No credentials for linuxserver in /config.json"
time="2020-07-01T13:37:33+02:00" level=debug msg="Got image name: linuxserver/lazylibrarian:latest"
time="2020-07-01T13:38:13+02:00" level=info msg="Found new linuxserver/lazylibrarian:latest image (sha256:83a97ae66af58363d3791d6e76117b7268ca82e7f50b5a0bffa8cab29b8fe50c)"
time="2020-07-01T13:38:13+02:00" level=info msg="Stopping /lazylibrarian (cfe2c6607dcfeca274f1b8c42a4a866441fedf49634ddaf3083de7f4b637b947) with SIGTERM"
time="2020-07-01T13:38:19+02:00" level=debug msg="Removing container cfe2c6607dcfeca274f1b8c42a4a866441fedf49634ddaf3083de7f4b637b947"
time="2020-07-01T13:38:23+02:00" level=info msg="Stopping /radarr (fa941e1e5f3489b938f0518f3e4e626314907dcdbd2cc97f45d1aea7faa3ea9e) with SIGTERM"
time="2020-07-01T13:38:29+02:00" level=debug msg="Removing container fa941e1e5f3489b938f0518f3e4e626314907dcdbd2cc97f45d1aea7faa3ea9e"
time="2020-07-01T13:38:32+02:00" level=info msg="Creating /radarr"
time="2020-07-01T13:38:33+02:00" level=debug msg="Starting container /radarr (46181cf3b2ece8e18d3533bb29987e45cfe70ad30977c78acdddaabd2bb1abcb)"
time="2020-07-01T13:38:36+02:00" level=info msg="Creating /lazylibrarian"
time="2020-07-01T13:38:37+02:00" level=debug msg="Starting container /lazylibrarian (f146c4441d64fee3b10cf56dc9d2a79e5e51c425bae49833cf8a9d56524e9cdb)"
time="2020-07-01T13:38:41+02:00" level=info msg="Removing image sha256:71e5d9eac9f206cfeacba45e0c554eb6238e134007816e1459b5614a5fd4ae64"
time="2020-07-01T13:38:43+02:00" level=info msg="Removing image sha256:ca81442d6b4e48e8eca81a66d712c4bb09ffd0c07642c055d622f82ff2fe305a"

The notification for it only reads: "2020-07-01 13:36:31 (info): Running a one time update."

1 reply

piksel Jul 4, 2020
Maintainer

Hm, maybe your template doesn't loop over the rows? Does it work with the default template?

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

run-once parameter executes multiple times, does not stop, does not give out info mails #568

{{title}}

Replies: 4 comments 13 replies

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

run-once parameter executes multiple times, does not stop, does not give out info mails #568

phit42 Jun 4, 2020

Replies: 4 comments · 13 replies

piksel Jun 5, 2020 Maintainer

simskij Jun 5, 2020 Maintainer

phit42 Jun 8, 2020 Author

phit42 Jun 9, 2020 Author

piksel Jun 9, 2020 Maintainer

simskij Jun 10, 2020 Maintainer

phit42 Jun 10, 2020 Author

piksel Jun 10, 2020 Maintainer

simskij Jun 12, 2020 Maintainer

phit42 Jun 12, 2020 Author

phit42 Jun 13, 2020 Author

simskij Jun 13, 2020 Maintainer

phit42 Jun 14, 2020 Author

phit42 Jul 1, 2020 Author

piksel Jul 4, 2020 Maintainer

phit42
Jun 4, 2020

Replies: 4 comments 13 replies

piksel
Jun 5, 2020
Maintainer

simskij Jun 5, 2020
Maintainer

phit42
Jun 8, 2020
Author

phit42 Jun 9, 2020
Author

piksel Jun 9, 2020
Maintainer

simskij Jun 10, 2020
Maintainer

phit42 Jun 10, 2020
Author

piksel Jun 10, 2020
Maintainer

simskij
Jun 12, 2020
Maintainer

phit42 Jun 12, 2020
Author

phit42 Jun 13, 2020
Author

simskij Jun 13, 2020
Maintainer

phit42 Jun 14, 2020
Author

phit42
Jul 1, 2020
Author

piksel Jul 4, 2020
Maintainer