Support for Durable Entities

[cgillum]

Tasks

Preview Give feedback

1. Durable Entities: SDK level abstractions #163
   durable-entities +1
   
2. Durable Entities: Entity operation dispatching #164
   durable-entities +1
   
3. Durable Entities: Client abstractions #166
   durable-entities +1
   
4. Durable entities: Orchestration context abstractions #167
   durable-entities +1
   
5. Documentation for .NET isolated Entity #182
   durable-entities +1
6. Durable Entities: add support to functions extension #188
   durable-entities +1
   
7. Durable Entities Public Preview Release #221
   18 of 18
   durable-entities +1
   

[AlexEngblom]

Have you planned schedule for this yet? Interested, because without this we cannot upgrade at all. It's a bottleneck for going forward with our NET 7 upgrade altogether, because it doesn't have in-proc support.

[wahyuen]

Appreciate the work the team is investing into this. <3 Just wanted to echo @AlexEngblom as we too are bottlenecked with our overall solution being unable to migrate to NET 7 due to lack of support for Durable Entities.

You may not/can not share the information which I can understand, but if you do have the information at hand, we would love to know if this is something we could potentially expect in the near future or something likely we would wait for NET 8 LTS for? This helps tremendously with internal planning and other streams of work internally for us.

[cgillum]

Adding @AnatoliB, @lilyjma, and @sebastianburckhardt

We've gotten a lot of feedback about this as a blocker for upgrading to the .NET Isolated worker, which we take very seriously. The team is currently working on a plan for when we think we can deliver this. Apologies for the lack of clarity here!

[kirk-marple]

@cgillum I know y'all are working hard on Durable Entities, but is there any way to get some guidance from Microsoft on when this may available? (Even roughly like, don't expect it at .NET 8 release.)

Also, I was reading over the ongoing PRs and with the new abstractions, will this require a rewrite of our Durable Entities, or is the expectation that minimal porting will be required?

[davidmrdavid]

@kirk-marple: I'll defer to @lilyjma on the final word, but my understanding is that we're shooting to have this released more or less in the 2nd week of October, if everything goes according to plan.

Also, I was reading over the ongoing PRs and with the new abstractions, will this require a rewrite of our Durable Entities, or is the expectation that minimal porting will be required?

I'll have to defer to @jviau for that.

[jviau]

@kirk-marple the functionality should be very close, but some code changes will be required. It will be about equivalent to the porting needed for orchestrations.

[VenkateshSrini]

@jviau ,
From Road map of Azure function, we see that isolated workers are going to be the way forward. That means we will have to migrate inproc to isolated workers anyway with .NET 8. So, if we could get this structure in place and firm with .NET 8 then in one migration go, we can make this change also. One important aspect I would like to have is all the Azure function Isolated workers to be in AOT. This can be left configurable at the function level, so that if there are some components which is used in the function can not undergo AOT then those function alone can be marked as Just in time compile.

[jviau]

One important aspect I would like to have is all the Azure function Isolated workers to be in AOT

AOT is not in the scope of durable functions, especially not in this work item. AOT is still being investigated / worked on for the dotnet isolated worker library, any issues/updates on that would be in that repo: https://github.com/Azure/azure-functions-dotnet-worker

[VenkateshSrini]

@jviau ,
Thank you. Can you please give details on any specific issue on AOT that is being workupon

[ksgopal4]

@kirk-marple: I'll defer to @lilyjma on the final word, but my understanding is that we're shooting to have this released more or less in the 2nd week of October, if everything goes according to plan.

Also, I was reading over the ongoing PRs and with the new abstractions, will this require a rewrite of our Durable Entities, or is the expectation that minimal porting will be required?

I'll have to defer to @jviau for that.

Hey All, Any further updates on when Durable entities will be available in isolated model?

[jviau]

Public preview is now available for durable entities: https://www.nuget.org/packages/Microsoft.Azure.Functions.Worker.Extensions.DurableTask/1.1.0-preview.2

[olitomlinson]

@jviau tested and works :) Thanks

[lilyjma]

Hi all, we announced GA for Durable entities in .NET isolated recently! package: https://www.nuget.org/packages/Microsoft.Azure.Functions.Worker.Extensions.DurableTask/1.1.0

docs: https://learn.microsoft.com/azure/azure-functions/durable/durable-functions-dotnet-isolated-overview#durable-entities

closing this issue.

[rlucassen]

I'm still missing the functionality to access durable entities through interfaces as described here: https://learn.microsoft.com/en-us/azure/azure-functions/durable/durable-functions-dotnet-entities?pivots=in-proc#accessing-entities-through-interfaces

CreateEntityProxy is not implemented it seems and SignalEntity doesn't seem to have a generic any longer.

Is this functionality coming to the new packages or do we have to implement this ourselves?

[prashantagrawalcrowe]

Do we have any replacement for IDurableActivityContext?

[wahyuen]

I'm still missing the functionality to access durable entities through interfaces as described here: https://learn.microsoft.com/en-us/azure/azure-functions/durable/durable-functions-dotnet-entities?pivots=in-proc#accessing-entities-through-interfaces

CreateEntityProxy is not implemented it seems and SignalEntity doesn't seem to have a generic any longer.

Is this functionality coming to the new packages or do we have to implement this ourselves?

Echoing this! Disappointing to hear this is now GA but does not have feature parity still. Is there any indication from the team if these are being looked at in the near future or not something we can expect to see for weeks/months/years? There is now a large amount of technical debt we now have across multiple projects/customers that would require rework with this being implemented!

[lilyjma]

Do we have any replacement for IDurableActivityContext?

@prashantagrawalcrowe - There's no equivalent for that in the isolated model, please see this table for API changes.

[lilyjma]

@rlucassen @wahyuen - Noted. Reopening this issue to keep track of this. We're committed to bring full parity to the isolated model - will keep you posted on the timeline.

[jviau]

@rlucassen, @wahyuen entity proxies are not planned in the isolated model. We have already dropped support for interface-based activity proxies in the isolated SDK. We will evaluate introducing an alternative way for strong-typing of entity calls, but it most likely will not be the proxy approach.

[jviau]

@prashantagrawalcrowe we do have TaskActivityContext, however it needs some enhancements. See #240

[jviau]

For entity proxies, you can also author your own. This sample uses primary ctors from C# 12:

public interface ICounter
{
    Task<int> Add(int amount);
}

public class CounterProxy(TaskEntityOrchestrationFeature feature, string key) : ICounter
{
    private readonly EntityInstanceId _id = new("counter", key);

    public Task<int> Add(int amount) => feature.CallEntityAsync<int>(_id, "add", amount);
}

// code in orchestration:
ICounter proxy = new CounterProxy(context.Entities, "__id__");
int count = await proxy.Add(amount);

[jviau]

Opened #252 to discuss a strong typed entity invocation feature. Going to reclose this issue.

[wahyuen]

@jviau whilst I understand this is completely the teams call in terms of what is and is not supported, it would have been nice if this was communicated that the support for interfaces was going away for durable entities.

We as consumers of this product only have the available documentation and guidance that the team gives, noting that this is what is currently documented for the in process model.

Accessing entities through interfaces
Interfaces can be used for accessing entities via generated proxy objects. This approach ensures that the name and argument type of an operation matches what is implemented. We recommend using interfaces for accessing entities whenever possible.

Note the recommended practice in the above statement! We now have a large code base across multiple projects with technical debt hanging over to migrate away from what we have written for Durable Entities. The work isn't so much of the issue, we just need to dedicate time/resources to it....the hope of the feedback here is that we were not TOLD this was going to be an issue. Given some of us have waited multiple iterations of .NET releases and being told support for durable entities would be coming .NET8 there has been a lot of expectation setting and preparation that many of us know we have to do...this unfortunately, is very last-minute spanner to throw into the works.

[lilyjma]

Thanks for your feedback @wahyuen. We acknowledge that the design plan for Entities in .NET isolated was not communicated as clearly as it should be. Proxies (or a strongly typed interface more generally) are an important part of existing .NET in-process solutions that use entities, so we definitely want some kind of equivalent in the isolated model, or at the very least updated guidance. This issue is about Entities in general, so we've started another
thread specifically for this discussion. Please let us know your thoughts there.

[jrichardsz]

I think DurableEntities are not full supported yet for isolated mode.

IDurableOrchestrationClient is not available in Microsoft.Azure.Functions.Worker.Extensions.DurableTask 1.1.4 for isolated mode. This version is the latest (2024 Jun)

The visual studio suggestion is to add Microsoft.Azure.WebJobs.Extensions.DurableTask but that is not accepted for isolated mode, right?

I'm using:

• net8
• dotnet-isolated mode
• Microsoft.Azure.Functions.Worker.Extensions.DurableTask 1.1.4
• visual 2022
• windows 10

[davidmrdavid]

@jrichardsz - for .NET isolated, that type should be DurableTaskClient, as per: https://learn.microsoft.com/en-us/azure/azure-functions/durable/durable-functions-dotnet-isolated-overview#public-api-changes

You can find .NET isolated entity templates here: https://github.com/microsoft/durabletask-dotnet/blob/main/samples/AzureFunctionsApp/Entities/Counter.cs

[jrichardsz]

Thank you David. I will review the DurableTaskClient.