Cosmos: Navigation to parent item has different behavior in EFC 7,8 and 9

[jujinfu]

We are experiencing something strange in EFC8.

Our DBContext:
LicenseDbContext.cs

using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Diagnostics;

namespace testcosmosdbapi
{
    public class LicenseDbContext : DbContext
    {
        protected override void OnConfiguring(DbContextOptionsBuilder options)
        {
            options.UseLazyLoadingProxies()
                .UseCosmos("", "", "")
                .ConfigureWarnings(b => b.Ignore(CosmosEventId.SyncNotSupported))
                ;
        }

        public DbSet<LicenseOptionData> LicenseOptionDatas { get; set; }

        public DbSet<ProductLicenseOption> ProductLicenseOptions { get; set; }

        protected override void OnModelCreating(ModelBuilder modelBuilder)
        {
            modelBuilder.Entity<LicenseOptionData>()
                .HasOne(x => x.ProductLicenseOption)
                .WithMany(x => x.LicenseOptionData)
                .HasForeignKey(x => x.ProductLicenseOptionId);

            modelBuilder.Entity<ProductLicenseOption>().HasDiscriminator<string>("Discriminator");
            modelBuilder.Entity<ProductLicenseOption>().HasDiscriminatorInJsonId();

            modelBuilder.Entity<LicenseOptionData>().HasDiscriminator<string>("Discriminator");
            modelBuilder.Entity<LicenseOptionData>().HasDiscriminatorInJsonId();
        }
    }
}

Our Models:
LicenseOptionData.cs

namespace testcosmosdbapi
{
    public class LicenseOptionData
    {
        public Guid Id { get; set; }

        public string Item { get; set; }

        public string Value { get; set; }

        public Guid ProductLicenseOptionId { get; set; }

        public virtual ProductLicenseOption ProductLicenseOption { get; set; }
    }
}

ProductLicenseOption.cs

namespace testcosmosdbapi
{
    public class ProductLicenseOption
    {
        public Guid Id { get; set; }

        public string LicenseSection { get; set; }

        public string LicenseSectionItem { get; set; }

        public string Description { get; set; }

        public string ValueName { get; set; }

        public string AvailableValues { get; set; }
        
        public bool Active { get; set; }

        public virtual List<LicenseOptionData> LicenseOptionData { get; set; }
    }
}

Our Controller:

namespace testcosmosdbapi.Controllers
{
    public class Controller : ControllerBase
    {
        [HttpGet("licenseoptiondata/notracking/{id}")]
        public async Task<IActionResult> GetResult(string id)
        {
            var dbContext = new LicenseDbContext();
            var result = await dbContext.LicenseOptionDatas.Where(x => x.Id == Guid.Parse(id)).AsNoTracking().FirstOrDefaultAsync();            
            return Ok(result);
        }

        [HttpGet("licenseoptiondata/tracking/{id}")]
        public async Task<IActionResult> GetResulttracking(string id)
        {
            var dbContext = new LicenseDbContext();
            var result = await dbContext.LicenseOptionDatas.Where(x => x.Id == Guid.Parse(id)).FirstOrDefaultAsync();
            return Ok(result);
        }
    }
}

When we try this code in EFC7, EFC9
We get the following result:
No Tracking and Tracking are the same

     {
          "productLicenseOption": {
            "licenseSection": "edition",
            "licenseSectionItem": "edition",
            "description": "Edition",
            "valueName": "Edition",
            "availableValues": "Pro",
            "productVersionId": "04edfba1-4f25-43b0-848a-a695089b1b0f",
            "active": true,
            "id": "d4ed44c6-21ff-4385-828f-bf369307688f",
            "_last_modified_date": "2022-10-25T20:28:28.1924574Z",
            "_last_modified_by": null
          },
          "item": "edition",
          "value": "pro",
          "licenseId": "a8998840-d624-4202-b98b-15e11ce29803",
          "productLicenseOptionId": "d4ed44c6-21ff-4385-828f-bf369307688f",
          "id": "0d73847b-dde1-42c7-80f3-08db61e1df94",
          "_last_modified_date": "2023-10-13T14:40:00.8540057Z",
          "_last_modified_by": null
        }

When we try this in EFC8:
we get the following results:
No Tracking:

{
          "productLicenseOption": {
            "licenseSection": "edition",
            "licenseSectionItem": "edition",
            "description": "Edition",
            "valueName": "Edition",
            "availableValues": "Pro",
            "productVersionId": "04edfba1-4f25-43b0-848a-a695089b1b0f",
            "active": true,
            "id": "d4ed44c6-21ff-4385-828f-bf369307688f",
            "_last_modified_date": "2022-10-25T20:28:28.1924574Z",
            "_last_modified_by": null
          },
          "item": "edition",
          "value": "pro",
          "licenseId": "a8998840-d624-4202-b98b-15e11ce29803",
          "productLicenseOptionId": "d4ed44c6-21ff-4385-828f-bf369307688f",
          "id": "0d73847b-dde1-42c7-80f3-08db61e1df94",
          "_last_modified_date": "2023-10-13T14:40:00.8540057Z",
          "_last_modified_by": null
        }

Tracking:

{
          "productLicenseOption": null 
          "item": "edition",
          "value": "pro",
          "licenseId": "a8998840-d624-4202-b98b-15e11ce29803",
          "productLicenseOptionId": "d4ed44c6-21ff-4385-828f-bf369307688f",
          "id": "0d73847b-dde1-42c7-80f3-08db61e1df94",
          "_last_modified_date": "2023-10-13T14:40:00.8540057Z",
          "_last_modified_by": null
        }

Final result, in EFC8, when the child references the parent item with Change Tracking, it will not return the parent by Navigation.
However, in EFC7,9, this works as expected.

We have tried many different things but we couldn't resolve this issue.

Some help would be greatly appreciated

[AndriySvyryd]

There've been quite a few changes to the Cosmos provider in EF 9. And it seems that it works as expected. Is there anything that prevents you from upgrading to EF 9?

[jujinfu]

Yes a couple of thing locking us right now.

The discriminator changes
The sync IO throws
The discriminator in JSON Id change

Those are the main ones. We haven’t really deep dive into it yet

[roji]

@jujinfu for all three of these changes, EF can be configured to revert to the previous behavior - please consult the Cosmos breaking changes notes. We intentionally built these opt-ins so that people wouldn't be blocked from upgrading.

[jujinfu]

@roji thank you for the info
we are aware of those. however, we still more time to prepare the upgrade to EFC9

Our plan is now get everything stable with efc 7, and then we can spend some time upgrading to efc 9

thought would be good to report the issue with efc 8

[roji]

@jujinfu thanks, makes sense. Realistically, there's going to be little chance that we do a patch to version 8 - the bar for doing that is quite high, and the many changes we did to the provider in 9 make this more difficult (note that 6 and 7 are out of support).

I suggest the following: please let us know if you're encountering this on 9; if so, we'll definitely take a look. At that point, if the fix is really simple, we can consider also backporting to 8. How does that sound?

If you won't have time to test this on 9 in the near future, we can close this for now, and reopen whenever you do get to it.

[jujinfu]

thank you @roji that makes sense

we expect to move to net8 and efc7 fully by the end of the month

and we will start the migration to efc9, we dont anticipate this problem to show up again in efc9

I agree we can close this ticket for the moment.

but should we list this as a known issue for efc8 or not?

[roji]

FWIW I'm not sure I'd spend time on first moving to EF 7 (which is out of support) and only then transition to 9 - it may be easier to just make the jump in one go (but YMMV).

We don't really have a list of known issues for EF releases - the github issues (this one) are generally good enough for that kind of thing.

Let us know if you hit trouble when upgrading to 9!