Skip to content
This repository has been archived by the owner on Oct 9, 2024. It is now read-only.

Constraint "${attr.consul.version} semver >= 1.7.0": 1 nodes excluded by filter #4

Closed
makp0 opened this issue Apr 2, 2023 · 2 comments
Closed

Constraint "${attr.consul.version} semver >= 1.7.0": 1 nodes excluded by filter #4

makp0 opened this issue Apr 2, 2023 · 2 comments
Assignees
@loicsaintroch
Labels
bug Something isn't working duplicate This issue or pull request already exists

Comments

@makp0
Copy link

makp0 commented Apr 2, 2023 •
edited
Loading

What OS are you running?

Linux

What architecture are you running?

amd64

What Vagrant provider are you running?

parallels

Unexpected behavior

When trying to run the Consul service mesh dashboard example from the official Hashi docs,
I'm getting:
image

Steps to reproduce

Hashibox on commit: 0d3fc70

  1. run make init
  2. run the dashboard example job from Nomad UI

Expected behavior

No warning should appear

Logs

Apr 02 08:19:20 node-server-1 systemd[1]: Starting "HashiCorp Consul"...
░░ Subject: A start job for unit consul.service has begun execution
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░
░░ A start job for unit consul.service has begun execution.
░░
░░ The job identifier is 3287.
Apr 02 08:19:20 node-server-1 consul[3692]: ==> Starting Consul agent...
Apr 02 08:19:20 node-server-1 consul[3692]: Version: '1.15.2'
Apr 02 08:19:20 node-server-1 consul[3692]: Build Date: '2023-03-30 17:51:19 +0000 UTC'
Apr 02 08:19:20 node-server-1 consul[3692]: Node ID: 'a2dbd0ce-3500-2a9b-4624-2ddf766082c9'
Apr 02 08:19:20 node-server-1 consul[3692]: Node name: 'node-server-1'
Apr 02 08:19:20 node-server-1 consul[3692]: Datacenter: 'us' (Segment: '')
Apr 02 08:19:20 node-server-1 consul[3692]: Server: true (Bootstrap: false)
Apr 02 08:19:20 node-server-1 consul[3692]: Client Addr: [192.168.60.10] (HTTP: 8500, HTTPS: -1, gRPC: 8502, gRPC-TLS: 8503, DNS: 8600)
Apr 02 08:19:20 node-server-1 consul[3692]: Cluster Addr: 192.168.60.10 (LAN: 8301, WAN: 8302)
Apr 02 08:19:20 node-server-1 consul[3692]: Gossip Encryption: false
Apr 02 08:19:20 node-server-1 consul[3692]: Auto-Encrypt-TLS: false
Apr 02 08:19:20 node-server-1 consul[3692]: HTTPS TLS: Verify Incoming: false, Verify Outgoing: false, Min Version: TLSv1_2
Apr 02 08:19:20 node-server-1 consul[3692]: gRPC TLS: Verify Incoming: false, Min Version: TLSv1_2
Apr 02 08:19:20 node-server-1 consul[3692]: Internal RPC TLS: Verify Incoming: false, Verify Outgoing: false (Verify Hostname: false), Min Version: TLSv1_2
Apr 02 08:19:20 node-server-1 consul[3692]: ==> Log data will now stream in as it occurs:
Apr 02 08:19:20 node-server-1 consul[3692]: 2023-04-02T08:19:20.941-0500 [WARN] agent: bootstrap_expect > 0: expecting 3 servers
Apr 02 08:19:20 node-server-1 consul[3692]: 2023-04-02T08:19:20.949-0500 [WARN] agent.auto_config: bootstrap_expect > 0: expecting 3 servers
Apr 02 08:19:20 node-server-1 consul[3692]: 2023-04-02T08:19:20.955-0500 [INFO] agent.server.raft: initial configuration: index=0 servers=[]
Apr 02 08:19:20 node-server-1 consul[3692]: 2023-04-02T08:19:20.955-0500 [INFO] agent.server.serf.wan: serf: EventMemberJoin: node-server-1.us 192.168.60.10
Apr 02 08:19:20 node-server-1 consul[3692]: 2023-04-02T08:19:20.955-0500 [INFO] agent.server.serf.lan: serf: EventMemberJoin: node-server-1 192.168.60.10
Apr 02 08:19:20 node-server-1 consul[3692]: 2023-04-02T08:19:20.955-0500 [INFO] agent.router: Initializing LAN area manager
Apr 02 08:19:20 node-server-1 consul[3692]: 2023-04-02T08:19:20.956-0500 [INFO] agent.server.autopilot: reconciliation now disabled
Apr 02 08:19:20 node-server-1 consul[3692]: 2023-04-02T08:19:20.957-0500 [INFO] agent.server.raft: entering follower state: follower="Node at 192.168.60.10:8300 [Follower]" leader-address= leader-id=
Apr 02 08:19:20 node-server-1 consul[3692]: 2023-04-02T08:19:20.959-0500 [INFO] agent.server: Adding LAN server: server="node-server-1 (Addr: tcp/192.168.60.10:8300) (DC: us)"
Apr 02 08:19:20 node-server-1 consul[3692]: 2023-04-02T08:19:20.960-0500 [INFO] agent.server: Handled event for server in area: event=member-join server=node-server-1.us area=wan
Apr 02 08:19:20 node-server-1 consul[3692]: 2023-04-02T08:19:20.960-0500 [WARN] agent: [core][Channel #1 SubChannel #3] grpc: addrConn.createTransport failed to connect to {
Apr 02 08:19:20 node-server-1 consul[3692]: "Addr": "us-192.168.60.10:8300",
Apr 02 08:19:20 node-server-1 consul[3692]: "ServerName": "node-server-1",
Apr 02 08:19:20 node-server-1 consul[3692]: "Attributes": null,
Apr 02 08:19:20 node-server-1 consul[3692]: "BalancerAttributes": null,
Apr 02 08:19:20 node-server-1 consul[3692]: "Type": 0,
Apr 02 08:19:20 node-server-1 consul[3692]: "Metadata": null
Apr 02 08:19:20 node-server-1 consul[3692]: }. Err: connection error: desc = "transport: Error while dialing dial tcp 192.168.60.10:0->192.168.60.10:8300: operation was canceled"
Apr 02 08:19:20 node-server-1 consul[3692]: 2023-04-02T08:19:20.961-0500 [INFO] agent.server.cert-manager: initialized server certificate management
Apr 02 08:19:20 node-server-1 consul[3692]: 2023-04-02T08:19:20.967-0500 [INFO] agent: Started DNS server: address=192.168.60.10:8600 network=udp
Apr 02 08:19:20 node-server-1 consul[3692]: 2023-04-02T08:19:20.967-0500 [INFO] agent: Started DNS server: address=192.168.60.10:8600 network=tcp
Apr 02 08:19:20 node-server-1 consul[3692]: 2023-04-02T08:19:20.967-0500 [INFO] agent: Starting server: address=192.168.60.10:8500 network=tcp protocol=http
Apr 02 08:19:20 node-server-1 consul[3692]: 2023-04-02T08:19:20.968-0500 [INFO] agent: started state syncer
Apr 02 08:19:20 node-server-1 consul[3692]: 2023-04-02T08:19:20.968-0500 [INFO] agent: Consul agent running!
Apr 02 08:19:20 node-server-1 consul[3692]: 2023-04-02T08:19:20.968-0500 [INFO] agent: Started gRPC listeners: port_name=grpc address=192.168.60.10:8502 network=tcp
Apr 02 08:19:20 node-server-1 consul[3692]: 2023-04-02T08:19:20.968-0500 [INFO] agent: Started gRPC listeners: port_name=grpc_tls address=192.168.60.10:8503 network=tcp
Apr 02 08:19:20 node-server-1 consul[3692]: 2023-04-02T08:19:20.968-0500 [INFO] agent: Retry join is supported for the following discovery methods: cluster=LAN discovery_methods="aliyun aws azure digitalocean gce hcp k8s linode mdns os packet scaleway softlayer tencentcloud triton vsphere"
Apr 02 08:19:20 node-server-1 consul[3692]: 2023-04-02T08:19:20.968-0500 [INFO] agent: Joining cluster...: cluster=LAN
Apr 02 08:19:20 node-server-1 consul[3692]: 2023-04-02T08:19:20.968-0500 [INFO] agent: (LAN) joining: lan_addresses=["192.168.60.10", "192.168.60.20", "192.168.60.30"]
Apr 02 08:19:20 node-server-1 consul[3692]: 2023-04-02T08:19:20.992-0500 [INFO] agent.server.serf.lan: serf: EventMemberJoin: node-server-3 192.168.60.30
Apr 02 08:19:20 node-server-1 consul[3692]: 2023-04-02T08:19:20.992-0500 [INFO] agent.server.serf.lan: serf: EventMemberJoin: node-server-2 192.168.60.20
Apr 02 08:19:20 node-server-1 consul[3692]: 2023-04-02T08:19:20.992-0500 [INFO] agent.server: Adding LAN server: server="node-server-3 (Addr: tcp/192.168.60.30:8300) (DC: us)"
Apr 02 08:19:20 node-server-1 consul[3692]: 2023-04-02T08:19:20.994-0500 [INFO] agent: (LAN) joined: number_of_nodes=3
Apr 02 08:19:20 node-server-1 systemd[1]: Started "HashiCorp Consul".
░░ Subject: A start job for unit consul.service has finished successfully
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░
░░ A start job for unit consul.service has finished successfully.
░░
░░ The job identifier is 3287.
Apr 02 08:19:21 node-server-1 consul[3692]: 2023-04-02T08:19:21.017-0500 [INFO] agent: Join cluster completed. Synced with initial agents: cluster=LAN num_agents=3
Apr 02 08:19:21 node-server-1 consul[3692]: 2023-04-02T08:19:21.028-0500 [INFO] agent.server: Found expected number of peers, attempting bootstrap: peers="192.168.60.30:8300,192.168.60.20:8300,192.168.60.10:8300"
Apr 02 08:19:21 node-server-1 consul[3692]: 2023-04-02T08:19:21.048-0500 [INFO] agent.server.serf.wan: serf: EventMemberJoin: node-server-3.us 192.168.60.30
Apr 02 08:19:21 node-server-1 consul[3692]: 2023-04-02T08:19:21.048-0500 [INFO] agent.server.serf.wan: serf: EventMemberJoin: node-server-2.us 192.168.60.20
Apr 02 08:19:21 node-server-1 consul[3692]: 2023-04-02T08:19:21.048-0500 [INFO] agent.server: Handled event for server in area: event=member-join server=node-server-3.us area=wan
Apr 02 08:19:21 node-server-1 consul[3692]: 2023-04-02T08:19:21.048-0500 [INFO] agent.server: Handled event for server in area: event=member-join server=node-server-2.us area=wan
Apr 02 08:19:21 node-server-1 consul[3692]: 2023-04-02T08:19:21.061-0500 [INFO] agent.server: Adding LAN server: server="node-server-2 (Addr: tcp/192.168.60.20:8300) (DC: us)"
Apr 02 08:19:28 node-server-1 consul[3692]: 2023-04-02T08:19:28.623-0500 [INFO] agent.server: New leader elected: payload=node-server-3
Apr 02 08:19:32 node-server-1 consul[3692]: 2023-04-02T08:19:32.665-0500 [INFO] agent.server.serf.lan: serf: EventMemberJoin: node-client-1 192.168.61.10
Apr 02 08:19:32 node-server-1 consul[3692]: 2023-04-02T08:19:32.790-0500 [INFO] agent.server.serf.lan: serf: EventMemberJoin: node-client-3 192.168.61.30
Apr 02 08:19:32 node-server-1 consul[3692]: 2023-04-02T08:19:32.808-0500 [INFO] agent.server.serf.lan: serf: EventMemberJoin: node-client-2 192.168.61.20
Apr 02 08:19:33 node-server-1 consul[3692]: 2023-04-02T08:19:33.957-0500 [WARN] agent.server.memberlist.lan: memberlist: Was able to connect to node-client-1 over TCP but UDP probes failed, network may be misconfigured
Apr 02 08:19:34 node-server-1 consul[3692]: 2023-04-02T08:19:34.960-0500 [WARN] agent.server.memberlist.lan: memberlist: Was able to connect to node-client-2 over TCP but UDP probes failed, network may be misconfigured
Apr 02 08:19:35 node-server-1 consul[3692]: 2023-04-02T08:19:35.940-0500 [ERROR] agent.anti_entropy: failed to sync remote state: error="rpc error making call: ACL system must be bootstrapped before making any requests that require authorization: ACL not found"
Apr 02 08:19:36 node-server-1 consul[3692]: 2023-04-02T08:19:36.522-0500 [ERROR] agent: Coordinate update error: error="rpc error making call: ACL system must be bootstrapped before making any requests that require authorization: ACL not found"
Apr 02 08:19:36 node-server-1 consul[3692]: 2023-04-02T08:19:36.959-0500 [WARN] agent.server.memberlist.lan: memberlist: Was able to connect to node-client-3 over TCP but UDP probes failed, network may be misconfigured
Apr 02 08:19:37 node-server-1 consul[3692]: 2023-04-02T08:19:37.451-0500 [ERROR] agent.anti_entropy: failed to sync remote state: error="ACL system must be bootstrapped before making any requests that require authorization: ACL not found"
Apr 02 08:19:39 node-server-1 consul[3692]: 2023-04-02T08:19:39.959-0500 [WARN] agent.server.memberlist.lan: memberlist: Was able to connect to node-client-3 over TCP but UDP probes failed, network may be misconfigured
Apr 02 08:19:40 node-server-1 consul[3692]: 2023-04-02T08:19:40.963-0500 [WARN] agent.server.memberlist.lan: memberlist: Was able to connect to node-client-2 over TCP but UDP probes failed, network may be misconfigured
Apr 02 08:19:41 node-server-1 consul[3692]: 2023-04-02T08:19:41.963-0500 [WARN] agent.server.memberlist.lan: memberlist: Was able to connect to node-client-1 over TCP but UDP probes failed, network may be misconfigured
Apr 02 08:19:42 node-server-1 consul[3692]: 2023-04-02T08:19:42.965-0500 [WARN] agent.server.memberlist.lan: memberlist: Was able to connect to node-client-1 over TCP but UDP probes failed, network may be misconfigured
Apr 02 08:19:44 node-server-1 consul[3692]: 2023-04-02T08:19:44.958-0500 [WARN] agent.server.memberlist.lan: memberlist: Was able to connect to node-client-3 over TCP but UDP probes failed, network may be misconfigured
Apr 02 08:19:46 node-server-1 consul[3692]: 2023-04-02T08:19:46.054-0500 [INFO] agent: Caught: signal=interrupt
Apr 02 08:19:46 node-server-1 consul[3692]: 2023-04-02T08:19:46.054-0500 [INFO] agent: Graceful shutdown disabled. Exiting
Apr 02 08:19:46 node-server-1 consul[3692]: 2023-04-02T08:19:46.054-0500 [INFO] agent: Requesting shutdown
Apr 02 08:19:46 node-server-1 consul[3692]: 2023-04-02T08:19:46.054-0500 [INFO] agent.server: shutting down server
Apr 02 08:19:46 node-server-1 consul[3692]: 2023-04-02T08:19:46.054-0500 [WARN] agent.server.serf.lan: serf: Shutdown without a Leave
Apr 02 08:19:46 node-server-1 systemd[1]: Stopping "HashiCorp Consul"...
░░ Subject: A stop job for unit consul.service has begun execution
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░
░░ A stop job for unit consul.service has begun execution.
░░
░░ The job identifier is 4336.
Apr 02 08:19:46 node-server-1 consul[3692]: 2023-04-02T08:19:46.056-0500 [WARN] agent.server.serf.wan: serf: Shutdown without a Leave
Apr 02 08:19:46 node-server-1 consul[3692]: 2023-04-02T08:19:46.056-0500 [INFO] agent.router.manager: shutting down
Apr 02 08:19:46 node-server-1 consul[3692]: 2023-04-02T08:19:46.056-0500 [INFO] agent.router.manager: shutting down
Apr 02 08:19:46 node-server-1 consul[3692]: 2023-04-02T08:19:46.057-0500 [INFO] agent: consul server down
Apr 02 08:19:46 node-server-1 consul[3692]: 2023-04-02T08:19:46.057-0500 [INFO] agent: shutdown complete
Apr 02 08:19:46 node-server-1 consul[3692]: 2023-04-02T08:19:46.057-0500 [INFO] agent: Stopping server: protocol=DNS address=192.168.60.10:8600 network=tcp
Apr 02 08:19:46 node-server-1 consul[3692]: 2023-04-02T08:19:46.057-0500 [INFO] agent: Stopping server: protocol=DNS address=192.168.60.10:8600 network=udp
Apr 02 08:19:46 node-server-1 consul[3692]: 2023-04-02T08:19:46.057-0500 [INFO] agent: Stopping server: address=192.168.60.10:8500 network=tcp protocol=http
Apr 02 08:19:46 node-server-1 consul[3692]: 2023-04-02T08:19:46.057-0500 [INFO] agent: Waiting for endpoints to shut down
Apr 02 08:19:46 node-server-1 consul[3692]: 2023-04-02T08:19:46.057-0500 [INFO] agent: Endpoints down
Apr 02 08:19:46 node-server-1 consul[3692]: 2023-04-02T08:19:46.057-0500 [INFO] agent: Exit code: code=1
Apr 02 08:19:46 node-server-1 systemd[1]: consul.service: Main process exited, code=exited, status=1/FAILURE
░░ Subject: Unit process exited
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░
░░ An ExecStart= process belonging to unit consul.service has exited.
░░
░░ The process' exit code is 'exited' and its exit status is 1.
Apr 02 08:19:46 node-server-1 systemd[1]: consul.service: Failed with result 'exit-code'.
░░ Subject: Unit failed
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░
░░ The unit consul.service has entered the 'failed' state with result 'exit-code'.
Apr 02 08:19:46 node-server-1 systemd[1]: Stopped "HashiCorp Consul".
░░ Subject: A stop job for unit consul.service has finished
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░
░░ A stop job for unit consul.service has finished.
░░
░░ The job identifier is 4336 and the job result is done.
Apr 02 08:19:50 node-server-1 systemd[1]: Starting "HashiCorp Consul"...
░░ Subject: A start job for unit consul.service has begun execution
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░
░░ A start job for unit consul.service has begun execution.
░░
░░ The job identifier is 4336.
Apr 02 08:19:50 node-server-1 consul[3822]: ==> Starting Consul agent...
Apr 02 08:19:50 node-server-1 consul[3822]: Version: '1.15.2'
Apr 02 08:19:50 node-server-1 consul[3822]: Build Date: '2023-03-30 17:51:19 +0000 UTC'
Apr 02 08:19:50 node-server-1 consul[3822]: Node ID: 'a2dbd0ce-3500-2a9b-4624-2ddf766082c9'
Apr 02 08:19:50 node-server-1 consul[3822]: Node name: 'node-server-1'
Apr 02 08:19:50 node-server-1 consul[3822]: Datacenter: 'us' (Segment: '')
Apr 02 08:19:50 node-server-1 consul[3822]: Server: true (Bootstrap: false)
Apr 02 08:19:50 node-server-1 consul[3822]: Client Addr: [192.168.60.10] (HTTP: 8500, HTTPS: -1, gRPC: 8502, gRPC-TLS: 8503, DNS: 8600)
Apr 02 08:19:50 node-server-1 consul[3822]: Cluster Addr: 192.168.60.10 (LAN: 8301, WAN: 8302)
Apr 02 08:19:50 node-server-1 consul[3822]: Gossip Encryption: false
Apr 02 08:19:50 node-server-1 consul[3822]: Auto-Encrypt-TLS: false
Apr 02 08:19:50 node-server-1 consul[3822]: HTTPS TLS: Verify Incoming: false, Verify Outgoing: false, Min Version: TLSv1_2
Apr 02 08:19:50 node-server-1 consul[3822]: gRPC TLS: Verify Incoming: false, Min Version: TLSv1_2
Apr 02 08:19:50 node-server-1 consul[3822]: Internal RPC TLS: Verify Incoming: false, Verify Outgoing: false (Verify Hostname: false), Min Version: TLSv1_2
Apr 02 08:19:50 node-server-1 consul[3822]: ==> Log data will now stream in as it occurs:
Apr 02 08:19:50 node-server-1 consul[3822]: 2023-04-02T08:19:50.548-0500 [WARN] agent: bootstrap_expect > 0: expecting 3 servers
Apr 02 08:19:50 node-server-1 consul[3822]: 2023-04-02T08:19:50.551-0500 [WARN] agent.auto_config: bootstrap_expect > 0: expecting 3 servers
Apr 02 08:19:50 node-server-1 consul[3822]: 2023-04-02T08:19:50.554-0500 [INFO] agent.server.raft: initial configuration: index=1 servers="[{Suffrage:Voter ID:247b43d4-680e-91a7-2559-ed5f7edf04f0 Address:192.168.60.30:8300} {Suffrage:Voter ID:0568fec4-01e3-982f-d971-9954e2fe12c6 Address:192.168.60.20:8300} {Suffrage:Voter ID:a2dbd0ce-3500-2a9b-4624-2ddf766082c9 Address:192.168.60.10:8300}]"
Apr 02 08:19:50 node-server-1 consul[3822]: 2023-04-02T08:19:50.554-0500 [INFO] agent.server.serf.wan: serf: EventMemberJoin: node-server-1.us 192.168.60.10
Apr 02 08:19:50 node-server-1 consul[3822]: 2023-04-02T08:19:50.554-0500 [INFO] agent.server.serf.lan: serf: EventMemberJoin: node-server-1 192.168.60.10
Apr 02 08:19:50 node-server-1 consul[3822]: 2023-04-02T08:19:50.554-0500 [INFO] agent.router: Initializing LAN area manager
Apr 02 08:19:50 node-server-1 consul[3822]: 2023-04-02T08:19:50.555-0500 [INFO] agent.server.autopilot: reconciliation now disabled
Apr 02 08:19:50 node-server-1 consul[3822]: 2023-04-02T08:19:50.555-0500 [INFO] agent.server.raft: entering follower state: follower="Node at 192.168.60.10:8300 [Follower]" leader-address= leader-id=
Apr 02 08:19:50 node-server-1 consul[3822]: 2023-04-02T08:19:50.555-0500 [INFO] agent.server.serf.wan: serf: Attempting re-join to previously known node: node-server-3.us: 192.168.60.30:8302
Apr 02 08:19:50 node-server-1 consul[3822]: 2023-04-02T08:19:50.555-0500 [INFO] agent.server.serf.lan: serf: Attempting re-join to previously known node: node-server-2: 192.168.60.20:8301
Apr 02 08:19:50 node-server-1 consul[3822]: 2023-04-02T08:19:50.555-0500 [INFO] agent.server: Adding LAN server: server="node-server-1 (Addr: tcp/192.168.60.10:8300) (DC: us)"
Apr 02 08:19:50 node-server-1 consul[3822]: 2023-04-02T08:19:50.556-0500 [INFO] agent.server: Raft data found, disabling bootstrap mode
Apr 02 08:19:50 node-server-1 consul[3822]: 2023-04-02T08:19:50.556-0500 [INFO] agent.server: Handled event for server in area: event=member-join server=node-server-1.us area=wan
Apr 02 08:19:50 node-server-1 consul[3822]: 2023-04-02T08:19:50.557-0500 [INFO] agent.server.cert-manager: initialized server certificate management
Apr 02 08:19:50 node-server-1 consul[3822]: 2023-04-02T08:19:50.557-0500 [INFO] agent: Started DNS server: address=192.168.60.10:8600 network=udp
Apr 02 08:19:50 node-server-1 consul[3822]: 2023-04-02T08:19:50.557-0500 [INFO] agent.server.serf.lan: serf: EventMemberJoin: node-client-2 192.168.61.20
Apr 02 08:19:50 node-server-1 consul[3822]: 2023-04-02T08:19:50.557-0500 [INFO] agent.server.serf.lan: serf: EventMemberJoin: node-server-3 192.168.60.30
Apr 02 08:19:50 node-server-1 consul[3822]: 2023-04-02T08:19:50.557-0500 [INFO] agent.server.serf.lan: serf: EventMemberJoin: node-server-2 192.168.60.20
Apr 02 08:19:50 node-server-1 consul[3822]: 2023-04-02T08:19:50.557-0500 [INFO] agent.server.serf.lan: serf: Re-joined to previously known node: node-server-2: 192.168.60.20:8301
Apr 02 08:19:50 node-server-1 consul[3822]: 2023-04-02T08:19:50.557-0500 [INFO] agent.server: Adding LAN server: server="node-server-3 (Addr: tcp/192.168.60.30:8300) (DC: us)"
Apr 02 08:19:50 node-server-1 consul[3822]: 2023-04-02T08:19:50.557-0500 [INFO] agent.server: Adding LAN server: server="node-server-2 (Addr: tcp/192.168.60.20:8300) (DC: us)"
Apr 02 08:19:50 node-server-1 consul[3822]: 2023-04-02T08:19:50.557-0500 [INFO] agent: Started DNS server: address=192.168.60.10:8600 network=tcp
Apr 02 08:19:50 node-server-1 consul[3822]: 2023-04-02T08:19:50.557-0500 [INFO] agent: Starting server: address=192.168.60.10:8500 network=tcp protocol=http
Apr 02 08:19:50 node-server-1 consul[3822]: 2023-04-02T08:19:50.558-0500 [INFO] agent: started state syncer
Apr 02 08:19:50 node-server-1 consul[3822]: 2023-04-02T08:19:50.558-0500 [INFO] agent: Consul agent running!
Apr 02 08:19:50 node-server-1 consul[3822]: 2023-04-02T08:19:50.558-0500 [INFO] agent: Started gRPC listeners: port_name=grpc address=192.168.60.10:8502 network=tcp
Apr 02 08:19:50 node-server-1 consul[3822]: 2023-04-02T08:19:50.558-0500 [INFO] agent: Started gRPC listeners: port_name=grpc_tls address=192.168.60.10:8503 network=tcp
Apr 02 08:19:50 node-server-1 consul[3822]: 2023-04-02T08:19:50.558-0500 [INFO] agent: Retry join is supported for the following discovery methods: cluster=LAN discovery_methods="aliyun aws azure digitalocean gce hcp k8s linode mdns os packet scaleway softlayer tencentcloud triton vsphere"
Apr 02 08:19:50 node-server-1 consul[3822]: 2023-04-02T08:19:50.558-0500 [INFO] agent: Joining cluster...: cluster=LAN
Apr 02 08:19:50 node-server-1 consul[3822]: 2023-04-02T08:19:50.558-0500 [INFO] agent: (LAN) joining: lan_addresses=["192.168.60.10", "192.168.60.20", "192.168.60.30"]
Apr 02 08:19:50 node-server-1 consul[3822]: 2023-04-02T08:19:50.559-0500 [INFO] agent.server.serf.wan: serf: EventMemberJoin: node-server-2.us 192.168.60.20
Apr 02 08:19:50 node-server-1 consul[3822]: 2023-04-02T08:19:50.559-0500 [INFO] agent.server: Handled event for server in area: event=member-join server=node-server-2.us area=wan
Apr 02 08:19:50 node-server-1 consul[3822]: 2023-04-02T08:19:50.564-0500 [INFO] agent.server.serf.wan: serf: EventMemberJoin: node-server-3.us 192.168.60.30
Apr 02 08:19:50 node-server-1 consul[3822]: 2023-04-02T08:19:50.564-0500 [INFO] agent.server: Handled event for server in area: event=member-join server=node-server-3.us area=wan
Apr 02 08:19:50 node-server-1 consul[3822]: 2023-04-02T08:19:50.566-0500 [INFO] agent: (LAN) joined: number_of_nodes=3
Apr 02 08:19:50 node-server-1 systemd[1]: Started "HashiCorp Consul".
░░ Subject: A start job for unit consul.service has finished successfully
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░
░░ A start job for unit consul.service has finished successfully.
░░
░░ The job identifier is 4336.
Apr 02 08:19:50 node-server-1 consul[3822]: 2023-04-02T08:19:50.587-0500 [INFO] agent.server.serf.wan: serf: Re-joined to previously known node: node-server-3.us: 192.168.60.30:8302
Apr 02 08:19:50 node-server-1 consul[3822]: 2023-04-02T08:19:50.587-0500 [INFO] agent: Join cluster completed. Synced with initial agents: cluster=LAN num_agents=3
Apr 02 08:19:50 node-server-1 consul[3822]: 2023-04-02T08:19:50.634-0500 [INFO] agent.server.serf.lan: serf: EventMemberJoin: node-client-3 192.168.61.30
Apr 02 08:19:50 node-server-1 consul[3822]: 2023-04-02T08:19:50.651-0500 [INFO] agent.server.serf.lan: serf: EventMemberJoin: node-client-1 192.168.61.10
Apr 02 08:19:50 node-server-1 consul[3822]: 2023-04-02T08:19:50.663-0500 [ERROR] agent.http: Request error: method=GET url=/v1/agent/self from=192.168.60.10:48840 error="ACL system must be bootstrapped before making any requests that require authorization: ACL not found"
Apr 02 08:19:51 node-server-1 consul[3822]: 2023-04-02T08:19:51.670-0500 [ERROR] agent.http: Request error: method=GET url=/v1/agent/self from=192.168.60.10:48840 error="ACL system must be bootstrapped before making any requests that require authorization: ACL not found"
Apr 02 08:19:53 node-server-1 consul[3822]: 2023-04-02T08:19:53.569-0500 [WARN] agent.server.memberlist.lan: memberlist: Was able to connect to node-client-2 over TCP but UDP probes failed, network may be misconfigured
Apr 02 08:19:54 node-server-1 consul[3822]: 2023-04-02T08:19:54.575-0500 [WARN] agent.server.memberlist.lan: memberlist: Was able to connect to node-client-1 over TCP but UDP probes failed, network may be misconfigured
Apr 02 08:19:55 node-server-1 consul[3822]: 2023-04-02T08:19:55.679-0500 [ERROR] agent.http: Request error: method=GET url=/v1/agent/self from=192.168.60.10:48840 error="ACL system must be bootstrapped before making any requests that require authorization: ACL not found"
Apr 02 08:19:56 node-server-1 consul[3822]: 2023-04-02T08:19:56.559-0500 [WARN] agent.server.memberlist.lan: memberlist: Was able to connect to node-client-3 over TCP but UDP probes failed, network may be misconfigured
Apr 02 08:19:56 node-server-1 consul[3822]: 2023-04-02T08:19:56.583-0500 [WARN] agent.server.raft: heartbeat timeout reached, starting election: last-leader-addr= last-leader-id=
Apr 02 08:19:56 node-server-1 consul[3822]: 2023-04-02T08:19:56.585-0500 [INFO] agent.server.raft: entering candidate state: node="Node at 192.168.60.10:8300 [Candidate]" term=3
Apr 02 08:19:56 node-server-1 consul[3822]: 2023-04-02T08:19:56.611-0500 [INFO] agent.server.raft: election won: term=3 tally=2
Apr 02 08:19:56 node-server-1 consul[3822]: 2023-04-02T08:19:56.611-0500 [INFO] agent.server.raft: entering leader state: leader="Node at 192.168.60.10:8300 [Leader]"
Apr 02 08:19:56 node-server-1 consul[3822]: 2023-04-02T08:19:56.611-0500 [INFO] agent.server.raft: added peer, starting replication: peer=247b43d4-680e-91a7-2559-ed5f7edf04f0
Apr 02 08:19:56 node-server-1 consul[3822]: 2023-04-02T08:19:56.612-0500 [INFO] agent.server.raft: added peer, starting replication: peer=0568fec4-01e3-982f-d971-9954e2fe12c6
Apr 02 08:19:56 node-server-1 consul[3822]: 2023-04-02T08:19:56.612-0500 [INFO] agent.server: cluster leadership acquired
Apr 02 08:19:56 node-server-1 consul[3822]: 2023-04-02T08:19:56.612-0500 [INFO] agent.server: New leader elected: payload=node-server-1
Apr 02 08:19:56 node-server-1 consul[3822]: 2023-04-02T08:19:56.613-0500 [INFO] agent.server.raft: pipelining replication: peer="{Voter 0568fec4-01e3-982f-d971-9954e2fe12c6 192.168.60.20:8300}"
Apr 02 08:19:56 node-server-1 consul[3822]: 2023-04-02T08:19:56.614-0500 [INFO] agent.server.raft: pipelining replication: peer="{Voter 247b43d4-680e-91a7-2559-ed5f7edf04f0 192.168.60.30:8300}"
Apr 02 08:19:56 node-server-1 consul[3822]: 2023-04-02T08:19:56.619-0500 [INFO] agent.server: initializing acls
Apr 02 08:19:56 node-server-1 consul[3822]: 2023-04-02T08:19:56.623-0500 [INFO] agent.leader: started routine: routine="acl token reaping"
Apr 02 08:19:56 node-server-1 consul[3822]: 2023-04-02T08:19:56.623-0500 [INFO] agent.server.autopilot: reconciliation now enabled
Apr 02 08:19:56 node-server-1 consul[3822]: 2023-04-02T08:19:56.623-0500 [INFO] agent.leader: started routine: routine="federation state anti-entropy"
Apr 02 08:19:56 node-server-1 consul[3822]: 2023-04-02T08:19:56.624-0500 [INFO] agent.leader: started routine: routine="federation state pruning"
Apr 02 08:19:56 node-server-1 consul[3822]: 2023-04-02T08:19:56.624-0500 [INFO] agent.leader: started routine: routine="streaming peering resources"
Apr 02 08:19:56 node-server-1 consul[3822]: 2023-04-02T08:19:56.624-0500 [INFO] agent.leader: started routine: routine="metrics for streaming peering resources"
Apr 02 08:19:56 node-server-1 consul[3822]: 2023-04-02T08:19:56.624-0500 [INFO] agent.leader: started routine: routine="peering deferred deletion"
Apr 02 08:19:56 node-server-1 consul[3822]: 2023-04-02T08:19:56.624-0500 [INFO] connect.ca: initialized primary datacenter CA from existing CARoot with provider: provider=consul
Apr 02 08:19:56 node-server-1 consul[3822]: 2023-04-02T08:19:56.624-0500 [INFO] agent.leader: started routine: routine="intermediate cert renew watch"
Apr 02 08:19:56 node-server-1 consul[3822]: 2023-04-02T08:19:56.624-0500 [INFO] agent.leader: started routine: routine="CA root pruning"
Apr 02 08:19:56 node-server-1 consul[3822]: 2023-04-02T08:19:56.625-0500 [INFO] agent.leader: started routine: routine="CA root expiration metric"
Apr 02 08:19:56 node-server-1 consul[3822]: 2023-04-02T08:19:56.625-0500 [INFO] agent.leader: started routine: routine="CA signing expiration metric"
Apr 02 08:19:56 node-server-1 consul[3822]: 2023-04-02T08:19:56.625-0500 [INFO] agent.leader: started routine: routine="virtual IP version check"
Apr 02 08:19:56 node-server-1 consul[3822]: 2023-04-02T08:19:56.625-0500 [INFO] agent.leader: started routine: routine="config entry controllers"
Apr 02 08:19:56 node-server-1 consul[3822]: 2023-04-02T08:19:56.626-0500 [INFO] agent.leader: stopping routine: routine="virtual IP version check"
Apr 02 08:19:56 node-server-1 consul[3822]: 2023-04-02T08:19:56.626-0500 [INFO] agent.leader: stopped routine: routine="virtual IP version check"
Apr 02 08:19:57 node-server-1 consul[3822]: 2023-04-02T08:19:57.300-0500 [WARN] agent.server.acl: failed to remove bootstrap file: error="remove /opt/consul/acl-bootstrap-reset: no such file or directory"
Apr 02 08:19:57 node-server-1 consul[3822]: 2023-04-02T08:19:57.329-0500 [INFO] agent.server.acl: ACL bootstrap completed
Apr 02 08:19:57 node-server-1 consul[3822]: 2023-04-02T08:19:57.365-0500 [INFO] agent.http: Request cancelled: method=GET url="/v1/catalog/service/nomad?dc=us&near=_agent&stale=&tag=serf&wait=2000ms" from=192.168.60.10:48842 error="ACL not found"
Apr 02 08:19:57 node-server-1 consul[3822]: 2023-04-02T08:19:57.560-0500 [WARN] agent.server.memberlist.lan: memberlist: Was able to connect to node-client-2 over TCP but UDP probes failed, network may be misconfigured
Apr 02 08:19:58 node-server-1 consul[3822]: 2023-04-02T08:19:58.560-0500 [WARN] agent.server.memberlist.lan: memberlist: Was able to connect to node-client-1 over TCP but UDP probes failed, network may be misconfigured
Apr 02 08:19:59 node-server-1 consul[3822]: 2023-04-02T08:19:59.563-0500 [WARN] agent.server.memberlist.lan: memberlist: Was able to connect to node-client-3 over TCP but UDP probes failed, network may be misconfigured
Apr 02 08:20:02 node-server-1 consul[3822]: 2023-04-02T08:20:02.425-0500 [ERROR] agent.anti_entropy: failed to sync remote state: error="ACL not found"
Apr 02 08:20:02 node-server-1 consul[3822]: 2023-04-02T08:20:02.711-0500 [ERROR] agent.anti_entropy: failed to sync remote state: error="ACL not found"
@loicsaintroch loicsaintroch self-assigned this Apr 3, 2023
@loicsaintroch loicsaintroch added the bug Something isn't working label Apr 3, 2023
@loicsaintroch
Copy link
Member

Given the logs of the Consul Client, it looks like this is related to #3, right?

@makp0
Copy link
Author

makp0 commented Apr 3, 2023

Oh my gosh, I somehow managed to miss the single issue in this repo. Hilarious!
Yes, it's definitely a duplicate. Closing as duplicate.

@makp0 makp0 closed this as completed Apr 3, 2023
@loicsaintroch loicsaintroch added the duplicate This issue or pull request already exists label Apr 3, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@loicsaintroch loicsaintroch

Labels
bug Something isn't working duplicate This issue or pull request already exists
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@loicsaintroch @makp0