Reliability issues for Altinn Events integration

[martinothamar]

Description of the bug

Service owners create reciever systems to download completed/archived instances to their systems. There are 2 flavors:

• Poll-based - poll archived instances through the Storage API
• Events-based - recieve events from Altinn Events (we publish app.instance.process.completed from the app, if RegisterEventsWithEventsComponent is enabled)

Previously there were reliability issues in both of these approaches. Committing the process state and instance events (which reciever systems rely on to be in sync) were 2 separate requests from the proces engine in the app and therefore two separate DB transactions in Storage (2pc). This was fixed in the v8.5 release of the app: Altinn/altinn-storage#544

The events-based approach however is using a separate Altinn API/service (Altinn Events), which makes it harder to fix. Options:

• Fix the process engine. There is a larger project planned for 2025 where the aim is to address all reliability issues in the process engine. We can rarely rely on Storage to give us transactionality and reliability as the process definition for an instance may use a variety of external APIs to drive the instance process forward. So it is a hard requirement for the process engine to be able to drive itself forward reliably.
• Extend Storage with outbox pattern or similar. The Storage DB already serves as the "source of truth" in terms of the state of the instances, so we could add outbox events as part of the larger transaction implemented in the previous Storage PR: Update process state and history (InstanceEvents) within a single transaction altinn-storage#544. It is easier to do in isolation, but could be viewed as temporary

For now we should document that the poll-based approach is recommended as of v8.5, since we have transactionality around process state and instance events (which some service owners rely on to be in sync).

Steps To Reproduce

1. Complete instance

Additional Information

No response