Argo Workflows disappearing from the UI

[rileyhun]

Pre-requisites

• I have double-checked my configuration
• I have tested with the :latest image tag (i.e. quay.io/argoproj/workflow-controller:latest) and can confirm the issue still exists on :latest. If not, I have explained why, in detail, in my description below.
• I have searched existing issues and could not find a match for this bug
• I'd like to contribute the fix myself (see contributing guide)

What happened? What did you expect to happen?

My workflows are disappearing from the UI after completion, and when I refresh my browser. I am using helm chart version 0.45.4.

Configmap looks like this:

Namespace:    argo-workflows
Labels:       app=workflow-controller
              app.kubernetes.io/component=workflow-controller
              app.kubernetes.io/instance=argo-workflows-controller
              app.kubernetes.io/managed-by=Helm
              app.kubernetes.io/name=argo-workflows-cm
              app.kubernetes.io/part-of=argo-workflows
              argocd.argoproj.io/instance=amps-c-uw2-argo-workflows-controller
              helm.sh/chart=argo-workflows-0.45.4
Annotations:  argocd.argoproj.io/tracking-id:
                argo-workflows-controller:/ConfigMap:argo-workflows/argo-workflows-controller-workflow-controller-configmap

Data
====
config:
----
artifactRepository:
  archiveLogs: true
  s3:
    bucket: argo-artifacts
    endpoint: s3.amazonaws.com
    insecure: false
    keyFormat: "argowf-artifacts/{{workflow.creationTimestamp.Y}}/{{workflow.creationTimestamp.m}}/{{workflow.creationTimestamp.d}}/{{workflow.name}}/{{pod.name}}"
    region: us-west-2
    useSDKCreds: true
persistence:
  archive: true
  archiveTTL: 30d
  postgresql:
    database: argowfdb
    host: <HOST_NAME>
    passwordSecret:
      key: POSTGRESQL_PASSWORD
      name: argo-postgres-config
    port: 5432
    tableName: argo_workflows
    userNameSecret:
      key: POSTGRESQL_USERNAME
      name: argo-postgres-config
nodeEvents:
  enabled: true
workflowEvents:
  enabled: true


BinaryData
====

Version(s)

v3.6.2

Paste a minimal workflow that reproduces the issue. We must be able to run the workflow; don't enter a workflow that uses private images.

N/A

Logs from the workflow controller

time="2025-02-07T21:04:50.871Z" level=info msg="Persistence configuration enabled"
time="2025-02-07T21:04:50.871Z" level=info msg="Node status offloading is disabled"
time="2025-02-07T21:04:50.871Z" level=info msg="Workflow archiving is enabled"
time="2025-02-07T21:04:50.871Z" level=info executorImage="quay.io/argoproj/argoexec:v3.6.2" executorImagePullPolicy= managedNamespace=
time="2025-02-07T21:09:14.205Z" level=info msg="Received Workflow Controller config map argo-workflows/argo-workflows-controller-workflow-controller-configmap update"
time="2025-02-07T21:09:14.211Z" level=info msg="Configuration:\nartifactRepository:\n  archiveLogs: true\n  s3:\n    bucket: argowf-artifacts-amps-c-uw2\n    endpoint: s3.amazonaws.com\n    insecure: false\n    keyFormat: argowf-artifacts/{{workflow.creationTimestamp.Y}}/{{workflow.creationTimestamp.m}}/{{workflow.creationTimestamp.d}}/{{workflow.name}}/{{pod.name}}\n    region: us-west-2\n    useSDKCreds: true\ninitialDelay: 0s\nmetricsConfig: {}\nnodeEvents:\n  enabled: true\npersistence:\n  archive: true\n  archiveTTL: 720h0m0s\n  postgresql:\n    database: argowfdb\n    host: argowfdb.cluster-cq47ebcpmyey.us-west-2.rds.amazonaws.com\n    passwordSecret:\n      key: POSTGRESQL_PASSWORD\n      name: argo-postgres-config\n    port: 5432\n    tableName: argo_workflows\n    userNameSecret:\n      key: POSTGRESQL_USERNAME\n      name: argo-postgres-config\npodSpecLogStrategy: {}\nsso:\n  clientId:\n    key: \"\"\n  clientSecret:\n    key: \"\"\n  issuer: \"\"\n  redirectUrl: \"\"\n  sessionExpiry: 0s\ntelemetryConfig: {}\nworkflowEvents:\n  enabled: true\n"
time="2025-02-07T21:09:14.211Z" level=info msg="Persistence configuration enabled"
time="2025-02-07T21:09:14.211Z" level=info msg="Node status offloading is disabled"
time="2025-02-07T21:09:14.212Z" level=info msg="Workflow archiving is enabled"
time="2025-02-07T21:09:14.212Z" level=info executorImage="quay.io/argoproj/argoexec:v3.6.2" executorImagePullPolicy= managedNamespace=
time="2025-02-07T21:09:32.255Z" level=info msg="Alloc=18676 TotalAlloc=3026751 Sys=40021 NumGC=3044 Goroutines=181"
time="2025-02-07T21:14:32.255Z" level=info msg="Alloc=13442 TotalAlloc=3027908 Sys=40021 NumGC=3047 Goroutines=181"
time="2025-02-07T21:19:32.255Z" level=info msg="Alloc=12779 TotalAlloc=3030833 Sys=40021 NumGC=3049 Goroutines=181"
time="2025-02-07T21:24:32.255Z" level=info msg="Alloc=12771 TotalAlloc=3031993 Sys=40021 NumGC=3052 Goroutines=181"
time="2025-02-07T21:29:32.255Z" level=info msg="Alloc=13541 TotalAlloc=3033156 Sys=40021 NumGC=3054 Goroutines=181"
time="2025-02-07T21:34:32.255Z" level=info msg="Alloc=12731 TotalAlloc=3034192 Sys=40021 NumGC=3057 Goroutines=181"
time="2025-02-07T21:39:32.254Z" level=info msg="Alloc=12755 TotalAlloc=3037196 Sys=40021 NumGC=3059 Goroutines=181"
time="2025-02-07T21:44:32.255Z" level=info msg="Alloc=12706 TotalAlloc=3038321 Sys=40021 NumGC=3062 Goroutines=181"

Logs from in your workflow's wait container

N/A

[rileyhun]

Can confirm that 3.5.11 is working fine though. Can see the archived flows in the UI.

[Joibel]

Are there any logs of note from the argo-server?