YesSql.ConcurrencyException: The document could not be updated as it has been changed by another process. When rebuilding elasticsearch index with version 1.8.3

[yqzhen1990]

Describe the bug

Failed to rebuild elasticsearch index

To Reproduce

Steps to reproduce the behavior:

1. Go to 'Search -> Elasticsearch Indices '
2. Click on 'Rebuild' of one of the indices
3. The spinner starts spinning...
4. After a while (about 1~2min) , the page shows: err_empty_response
5. And you can see the log in the screenshot.

Expected behavior

A clear and concise description of what you expected to happen.

Screenshots

logs from docker :

query indices from elastic:

[Piedone]

Were other users accessing the app at the same time, and editing content?

The same kind of issue: #15743.

[yqzhen1990]

Were other users accessing the app at the same time, and editing content?

The same kind of issue: #15743.

No other users editing content at the same time.

[Piedone]

Does this reproduce you on every index rebuild?

[sebastienros]

Could be two requests or threads saving the site settings.

@yqzhen1990 can you try with the latest main branch? We fixed a concurrency issue related to site settings that could explain it.

If not that, then maybe a background task that is updating the index is happening while the Rebuild is invoked from the UI, and that would be something to handle. To debug that we should add some logging to display when either is invoked/done and see if this is what is happening.

[yqzhen1990]

Does this reproduce you on every index rebuild?

yes

[yqzhen1990]

@sebastienros ok, I'll try with the latest main branch this weekend.

[hyzx86]

OC version : v2.0.latest

The same problem occurs when executing an installed recipe, I applied this PR in my own repository:
A similar problem occurs when executing a tenant-installed recipe,

In my installation recipe, there are several modules similar to EasyOC.Amis that contain migration recipes, which automatically create some ContentType

After applying this PR sebastienros/yessql#568, you can see from the logs that the error message comes from ContentDefinitionRecord.

 YesSql.ConcurrencyException: The document with ID '4' and type 
'OrchardCore.ContentManagement.Metadata.Records.ContentDefinitionRecord, 
OrchardCore.ContentManagement.Abstractions' could not be updated as it has been changed by another process.

2024-06-03 00:49:20.1419|006003|OrchardCore.Data.Migration.DataMigrationManager|ERROR|::1|admin http://localhost:2919/saas/Admin
Error while running migration version 0 for 'EasyOC.Amis'. OrchardCore.Recipes.Models.RecipeExecutionException: The document with ID '4' and type 'OrchardCore.ContentManagement.Metadata.Records.ContentDefinitionRecord, OrchardCore.ContentManagement.Abstractions' could not be updated as it has been changed by another process.
 ---> YesSql.ConcurrencyException: The document with ID '4' and type 'OrchardCore.ContentManagement.Metadata.Records.ContentDefinitionRecord, OrchardCore.ContentManagement.Abstractions' could not be updated as it has been changed by another process.
   at YesSql.Commands.UpdateDocumentCommand.ExecuteAsync(DbConnection connection, DbTransaction transaction, ISqlDialect dialect, ILogger logger)
   at YesSql.Session.FlushInternalAsync(Boolean saving)
   at YesSql.Session.FlushInternalAsync(Boolean saving)
   at YesSql.Services.DefaultQuery.Query`1.FirstOrDefaultImpl()
   at OrchardCore.Data.Documents.DocumentStore.GetOrCreateMutableAsync[T](Func`1 factoryAsync)
   at OrchardCore.Documents.DocumentManager`1.GetOrCreateMutableAsync(Func`1 factoryAsync)
   at OrchardCore.Templates.Services.TemplatesManager.UpdateTemplateAsync(String name, Template template)
   at OrchardCore.Templates.Recipes.TemplateStep.ExecuteAsync(RecipeExecutionContext context)
   at OrchardCore.Recipes.Services.RecipeExecutor.<>c__DisplayClass8_0.<<ExecuteStepAsync>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at OrchardCore.Environment.Shell.Scope.ShellScope.UsingAsync(Func`2 execute, Boolean activateShell)
   at OrchardCore.Recipes.Services.RecipeExecutor.ExecuteStepAsync(RecipeExecutionContext recipeStep)
   at OrchardCore.Recipes.Services.RecipeExecutor.ExecuteAsync(String executionId, RecipeDescriptor recipeDescriptor, IDictionary`2 environment, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at OrchardCore.Recipes.Services.RecipeExecutor.ExecuteAsync(String executionId, RecipeDescriptor recipeDescriptor, IDictionary`2 environment, CancellationToken cancellationToken)
   at OrchardCore.Recipes.Services.RecipeExecutor.ExecuteAsync(String executionId, RecipeDescriptor recipeDescriptor, IDictionary`2 environment, CancellationToken cancellationToken)
   at OrchardCore.Recipes.Services.RecipeExecutor.ExecuteAsync(String executionId, RecipeDescriptor recipeDescriptor, IDictionary`2 environment, CancellationToken cancellationToken)
   at EasyOC.Core.Recipes.EocRecipeExecutor.ExecuteAsync(String executionId, RecipeDescriptor recipeDescriptor, IDictionary`2 environment, CancellationToken cancellationToken) in D:\SourceCodes\JZSoft\EasyOC.OrchardCore\EasyOC\src\Core\EasyOC.Core\Recipes\EocRecipeExecutor.cs:line 32
   at OrchardCore.Recipes.Services.RecipeMigrator.ExecuteAsync(String recipeFileName, IDataMigration migration)
   at EasyOC.Amis.Migrations.Migrations.CreateAsync() in D:\SourceCodes\JZSoft\EasyOC.OrchardCore\EasyOC\src\Modules\EasyOC.Amis\Migrations\Migrations.cs:line 18
   at OrchardCore.Data.Migration.DataMigrationManager.InvokeMethodAsync(MethodInfo method, IDataMigration migration)
   at OrchardCore.Data.Migration.DataMigrationManager.UpdateAsync(String featureId)    at OrchardCore.Recipes.Services.RecipeExecutor.ExecuteAsync(String executionId, RecipeDescriptor recipeDescriptor, IDictionary`2 environment, CancellationToken cancellationToken)
   at OrchardCore.Recipes.Services.RecipeExecutor.ExecuteAsync(String executionId, RecipeDescriptor recipeDescriptor, IDictionary`2 environment, CancellationToken cancellationToken)
   at OrchardCore.Recipes.Services.RecipeExecutor.ExecuteAsync(String executionId, RecipeDescriptor recipeDescriptor, IDictionary`2 environment, CancellationToken cancellationToken)
   at EasyOC.Core.Recipes.EocRecipeExecutor.ExecuteAsync(String executionId, RecipeDescriptor recipeDescriptor, IDictionary`2 environment, CancellationToken cancellationToken) in D:\SourceCodes\JZSoft\EasyOC.OrchardCore\EasyOC\src\Core\EasyOC.Core\Recipes\EocRecipeExecutor.cs:line 32
   at OrchardCore.Recipes.Services.RecipeMigrator.ExecuteAsync(String recipeFileName, IDataMigration migration)
   at EasyOC.Amis.Migrations.Migrations.CreateAsync() in D:\SourceCodes\JZSoft\EasyOC.OrchardCore\EasyOC\src\Modules\EasyOC.Amis\Migrations\Migrations.cs:line 18
   at OrchardCore.Data.Migration.DataMigrationManager.InvokeMethodAsync(MethodInfo method, IDataMigration migration)
   at OrchardCore.Data.Migration.DataMigrationManager.UpdateAsync(String featureId)

[hyzx86]

Strangely enough this issue is not reproducible in unit tests, it only occurs when performing tenant initialisation from the page

[Piedone]

I executed the Elasticsearch Search recipe after a Blog setup with the latest main a few days ago. Are you saying something like this fails for you?

[hyzx86]

I executed the Elasticsearch Search recipe after a Blog setup with the latest main a few days ago. Are you saying something like this fails for you?

The version he mentions is 1.8.3, and the problem I'm having is with the latest version, so I've moved to #16214

[Piedone]

Which DB are you using? I just tried with the latest main (8e3c818), SQLite, local Elasticsearch in Docker, and rebuilds just work.

[SzymonSel]

Could be two requests or threads saving the site settings.

@yqzhen1990 can you try with the latest main branch? We fixed a concurrency issue related to site settings that could explain it.

If not that, then maybe a background task that is updating the index is happening while the Rebuild is invoked from the UI, and that would be something to handle. To debug that we should add some logging to display when either is invoked/done and see if this is what is happening.

How was this fixed? I have a similar issues concering part settings, being accessed and updated by many users (PartHandler, workflow - a simple counter being incremented and used for transaction numbering)