Grafana-agent subordinate to kafka fails on channel=3/stable

[MichaelThamm]

Bug Description

From Enrico Deusebio:

With grafana-agent on VMs where the new revision on latest/stable (also on latest/edge) for grafana-agent seem to have broken some backward compatibility. Upon relation with Kafka (3/stable), the grafana agent charm goes into error state.

The relation between kafka and grafana-agent used to work fine with a previous revision, e.g. 223. Interestingly, the error does not show up with our (Data team) newer version of the Kafka charm, so the code that provides this exception is on this branch.

The issue happens on line 660 because snap_yaml["slots"] only has keys: ['kafka-logs', 'cc-logs'],

but endpoint.name in log_dirs = snap_yaml["slots"][endpoint.name]["source"]["read"] resolves to "logs" which comes from the COSAgentRequirer.snap_log_endpoints_with_topology resulting in SnapEndpoint(owner='charmed-kafka', name='logs')

To Reproduce

1. Deploy kafka and zookeeper, both from 3/stable channel
2. Relate kafka and zookeeper and wait for the charms to go to active/idle
3. Deploy grafana-agent from latest/stable (rev332)
4. Relate grafana-agent with kafka

Environment

juju lxd controller

Relevant log output

unit-grafana-agent-2: 18:29:01 ERROR unit.grafana-agent/2.juju-log cos-agent:14: Uncaught exception while in charm code:
Traceback (most recent call last):
  File "/var/lib/juju/agents/unit-grafana-agent-2/charm/./src/charm.py", line 701, in <module>
    main(GrafanaAgentMachineCharm)
  ...  
  File "/var/lib/juju/agents/unit-grafana-agent-2/charm/./src/charm.py", line 637, in _snap_plugs_logging_configs
    log_dirs = snap_yaml["slots"][endpoint.name]["source"]["read"]
KeyError: 'logs'

Additional context

No response

[jeffreychang911]

SolQA filed this earlier, #245

[deusebio]

Oh yes, indeed. I'm extremely pleased that SolQA is definitely on top of these regressions! Great job @jeffreychang911 !!!

As suggested by @sed-i on Matrix, for revisions 261+, it is required to set a config option classic_snap=false to get the deployment to run smoothly. I have tested this, and I confirm that with this setting the deployment works fine.

I'm missing the permission to close the issue. @MichaelThamm could you do that?