Troubleshooting Guide: KymaCR stuck during deletion

[c-pius]

Description

Write the following troubleshooting guide for a KymaCR that is supposed to be deleted but stuck

• Symptom:
  • KymaCR is stuck in non-ready state (likely Error, could also be Deleting or Processing)
  • KymaCR has a deletion timestamp
• Typicall log output (to be double checked):
  • KymaCR has condition: failed to ensure remote namespace exists
  • Logs show frequent entries of: failed fetching CRDs from remote cluster
• Likely Cause:
  • KymaCR is supposed to be deleted (hence the deletion timestamp) but the related kubeconfig Secret has not been deleted
• Solution:
  • Actions:
    • Delete kubeconfig Secret; this is in the responsibililty of KIM (framefrog) and not KLM (jellyfish); once deleted, KLM will proceed with deleting the KymaCR
    • SRE should decide whether it is deleted on the fly or left for now for framefrogs to investigate
  • Verification:
    • kubectl -n kcp-system get secret <secret-name> returns not found
    • kubectl -n kcp-system get kyma <kyma-name> returns not found

General Remarks

• guide should be written in a way so that L1 / L2 support and module teams can also understand them
• guides may describe problems looking like KLM-related on the surface, but are in fact related to other components
  • in that case, the resolution part should state clearly that this is not KLM-related and provide a cross-reference to the team that should be responsible

Acceptance Criteria

• trouble shooting guide with above contents available on on-call-guides
• trouble shooting guide follows the template template-troubleshooting.md