Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[automated] Merge branch 'main' => 'dev' #5992

Open
wants to merge 327 commits into
base: dev
Choose a base branch
from
Open

Conversation

github-actions[bot]
Copy link
Contributor

@github-actions github-actions bot commented Feb 28, 2025

I detected changes in the main branch which have not been merged yet to dev. I'm a robot and am configured to help you automatically keep dev up to date, so I've opened this PR.

This PR merges commits made on main by the following committers:

  • RussKie
  • MackinnonBuck
  • evgenyfedorov2
  • artl93
  • stephentoub
  • mgravell
  • eiriktsarpalis
  • jeffhandley
  • jmatthiesen
  • halter73
  • peterwald
  • shyamnamboodiripad
  • SteveSandersonMS
  • dariusclay
  • IbrahimMNada
  • dotnet-maestro[bot]
  • wtgodbe
  • gewarren
  • joperezr
  • amadeuszl
  • Demo30
  • rainsxng
  • jodydonetti
  • tarekgh
  • martincostello
  • iliar-turdushev
  • NatMarchand
  • eerhardt
  • haipz
  • makazeu
  • carolineRe13
  • kzu

Instructions for merging from UI

This PR will not be auto-merged. When pull request checks pass, complete this PR by creating a merge commit, not a squash or rebase commit.

merge button instructions

If this repo does not allow creating merge commits from the GitHub UI, use command line instructions.

Instructions for merging via command line

Run these commands to merge this pull request from the command line.

git fetch
git checkout main
git pull --ff-only
git checkout dev
git pull --ff-only
git merge --no-ff main

# If there are merge conflicts, resolve them and then run git merge --continue to complete the merge
# Pushing the changes to the PR branch will re-trigger PR validation.
git push https://github.com/dotnet/extensions HEAD:merge/main-to-dev
or if you are using SSH
git push [email protected]:dotnet/extensions HEAD:merge/main-to-dev

After PR checks are complete push the branch

git push

Instructions for resolving conflicts

⚠️ If there are merge conflicts, you will need to resolve them manually before merging. You can do this using GitHub or using the command line.

Instructions for updating this pull request

Contributors to this repo have permission update this pull request by pushing to the branch 'merge/main-to-dev'. This can be done to resolve conflicts or make other changes to this pull request before it is merged.
The provided examples assume that the remote is named 'origin'. If you have a different remote name, please replace 'origin' with the name of your remote.

git fetch
git checkout -b merge/main-to-dev origin/dev
git pull https://github.com/dotnet/extensions merge/main-to-dev
(make changes)
git commit -m "Updated PR with my changes"
git push https://github.com/dotnet/extensions HEAD:merge/main-to-dev
or if you are using SSH
git fetch
git checkout -b merge/main-to-dev origin/dev
git pull [email protected]:dotnet/extensions merge/main-to-dev
(make changes)
git commit -m "Updated PR with my changes"
git push [email protected]:dotnet/extensions HEAD:merge/main-to-dev

Contact .NET Core Engineering (dotnet/dnceng) if you have questions or issues.
Also, if this PR was generated incorrectly, help us fix it. See https://github.com/dotnet/arcade/blob/main/.github/workflows/scripts/inter-branch-merge.ps1.

Microsoft Reviewers: Open in CodeFlow

joperezr and others added 30 commits October 25, 2024 16:12
#### AI description  (iteration 1)
#### PR Classification
New feature

#### PR Summary
This pull request introduces new scripts and classes to enhance the functionality of the repository.
- Added `Get-RepoDigest.ps1` script to generate a repository digest report.
- Added `AzureAIInferenceEmbeddingGenerator.cs` to support embedding generation using Azure AI Inference.
- Added `repo-digest-template.html` to serve as a template for the repository digest report.
* Add NativeAOT testapp project for M.E.AI

* Address PR feedback
…5582)

* Lower AIJsonUtilities to STJv8 and move to Abstractions library.

* Add README.md
* Plug JsonSchemaExporter test data to the AIJsonUtilities tests

* Update src/LegacySupport/DiagnosticAttributes/README.md

* Update src/Libraries/Microsoft.Extensions.AI.Abstractions/Utilities/AIJsonUtilities.Schema.cs

* Address feedback.
* Improve JsonSchemaExporter trimmer safety.

* Remove var

* Address feedback.

* Remove DynamicallyAccessedMemberTypes.All

* Extract reflection helpers into separate file and remove a number of warning suppressions.

* Re-enable failing tests that were patched in .NET 9
- Add a strongly-typed Enumerator
- Add a TryAdd method
- Add a DebuggerDisplay for Count
- Add a DebuggerTypeProxy for the collection of properties
* Add UseEmbeddingGenerationOptions

Counterpart to UseChatOptions

* Document/test null options returned from callback
* - handle serialization failures
- enforce payload quota
- enforce key validity
- add proper logging (infrastructure failure: needs attn)

# Conflicts:
#	src/Libraries/Microsoft.Extensions.Caching.Hybrid/Microsoft.Extensions.Caching.Hybrid.csproj

* - add "callback" to .dic
- log deserialization failures
- expose serialization failures
- tests for serialization logging scenarios

* support and tests for stability despite unreliable L2

* nit

* Compile for NS2.0

* include enabled check in our log output

* add event-source tracing and counters

* explicitly specify event-source guid

* satisfy the stylebot overloads

* nix SDT

* fix failing CI test

* limit to net462

* PR feedback (all except event tests)

* naming

* add event source tests

* fix redundant comment

* add clarification

* more clarifications

* dance for our robot overlords

* drop Microsoft.Extensions.Telemetry.Abstractions package-ref

* fix glitchy L2 test

* better tracking for invalid event-source state

* reserve non-printable characters from keys, to prevent L2 abuse

* improve test output for ETW

* tyop

* ETW tests: allow longer if needed

* whitespace

* more ETW fixins

---------

Co-authored-by: Jose Perez Rodriguez <[email protected]>
* Add NativeAOT testapp project for M.E.AI

* Address PR feedback
…5582)

* Lower AIJsonUtilities to STJv8 and move to Abstractions library.

* Add README.md
* Plug JsonSchemaExporter test data to the AIJsonUtilities tests

* Update src/LegacySupport/DiagnosticAttributes/README.md

* Update src/Libraries/Microsoft.Extensions.AI.Abstractions/Utilities/AIJsonUtilities.Schema.cs

* Address feedback.
* Improve JsonSchemaExporter trimmer safety.

* Remove var

* Address feedback.

* Remove DynamicallyAccessedMemberTypes.All

* Extract reflection helpers into separate file and remove a number of warning suppressions.

* Re-enable failing tests that were patched in .NET 9
- Add a strongly-typed Enumerator
- Add a TryAdd method
- Add a DebuggerDisplay for Count
- Add a DebuggerTypeProxy for the collection of properties
* Add UseEmbeddingGenerationOptions

Counterpart to UseChatOptions

* Document/test null options returned from callback
* - handle serialization failures
- enforce payload quota
- enforce key validity
- add proper logging (infrastructure failure: needs attn)

# Conflicts:
#	src/Libraries/Microsoft.Extensions.Caching.Hybrid/Microsoft.Extensions.Caching.Hybrid.csproj

* - add "callback" to .dic
- log deserialization failures
- expose serialization failures
- tests for serialization logging scenarios

* support and tests for stability despite unreliable L2

* nit

* Compile for NS2.0

* include enabled check in our log output

* add event-source tracing and counters

* explicitly specify event-source guid

* satisfy the stylebot overloads

* nix SDT

* fix failing CI test

* limit to net462

* PR feedback (all except event tests)

* naming

* add event source tests

* fix redundant comment

* add clarification

* more clarifications

* dance for our robot overlords

* drop Microsoft.Extensions.Telemetry.Abstractions package-ref

* fix glitchy L2 test

* better tracking for invalid event-source state

* reserve non-printable characters from keys, to prevent L2 abuse

* improve test output for ETW

* tyop

* ETW tests: allow longer if needed

* whitespace

* more ETW fixins

---------

Co-authored-by: Jose Perez Rodriguez <[email protected]>
stephentoub and others added 25 commits February 24, 2025 16:20
* Address API feedback on M.E.AI middleware

- Remove the public AnonymousDelegatingChatClient. If we decide to add it back later, we should pick a better name. For now the majority use case is handled by the still-public ChatClientBuilder.Use method that uses this type internally.
- Use Func rather than a bespoke delegate type for ChatClientBuilder.Use.
- Remove AIFunctionContext and just special-case CancellationToken in AIFunctionFactory.
- Special-case CancellationToken in AIJsonUtilities. We don't want the CancellationToken showing up in generated schemas; it's not really part of the functionality of methods, and how it's described to an LLM is meaningless and problematic.
- Fix AddResponseMessages chat messages parameter name.
- Made FunctionInvokingChatClient.CurrentContext setter protected. It should only be settable by this instance, including in derived types.
- Renamed FunctionInvokingChatClient.DetailedErrors to IncludeDetailedErrors, as such bool properties should start with verbs.
- Renamed FunctionInvokingChatClient.ConcurrentInvocation to AllowConcurrentInvocation, as such bool properties should start with verbs.
- Renamed FunctionInvokingChatClient.KeepFunctionCallingMessages to KeepFunctionCallingContent, as it's really about the content and not necessarily how they're bundled into messages.
- Rename FunctionStatus to FunctionInvocationStatus.
- Rename FunctionInvocationStatus.Failed to Exception.
- Move FunctionInvocationContext out of FunctionInvokingChatClient. Now that it's not just for types derived from FICC, it shouldn't be nested.
- AddKeyed*'s serviceKey should be nullable, to match the core AddKeyed* APIs.

* Address PR feedback
…ity (#5897)

* Fixing up namespaces and broken template code in Chat.razor

* Fixed Azure Search support, removed the Attach Media button.

* Fixing Ollama usage and OpenAI + Azure AI Search.

* Updated default model to Llama3.2, which has worked well in testing.

* Apply suggestions from code review

Co-authored-by: Steve Sanderson <[email protected]>

* Addressed feedback from PR review

* Added a README to display on project create, with setup instructions.

Added support for Managed Identity in Azure AI Search

* Added checks to see if the environment is configured, and direct users if it isn't.

* Update src/ProjectTemplates/Microsoft.Extensions.AI.Templates/src/ChatWithCustomData/ChatWithCustomData.Web/README.md

Co-authored-by: Steve Sanderson <[email protected]>

* Apply suggestions to README, from code review

Co-authored-by: Steve Sanderson <[email protected]>

* Addressing feedback from PR - removing redundant configuration checks. Adding clarifying details in README for AzOAI endpoint.

* Added a new icon for Preview 1. Addressed PR feedback and simplified checks for Ollama setup.

* Backed out try/catch statements per PR feedback.

---------

Co-authored-by: Steve Sanderson <[email protected]>
* Cancel current response when navigating away

* Add comment about ingesting only trusted content

* Move citation content to wwwroot

* Use RegexOptions.NonBacktracking

* HTML-encode LLM output before passing through marked/purify

* Enable chat client logging by default
* Updated name and new sample data

* Updated with sample data for the Preview 1.

* Move PDFs to new location after rebase

* Program.cs namespace fixes

---------

Co-authored-by: Steve Sanderson <[email protected]>
JSON schema supplied as part of structured output was conflicting with the instructions in the prompt. This was causing the evaluator to include the reasoning property even when includeReasoning was set to false.

For now, the fix is to refrain from using structured output. In the future we can look to improve the prompt and / or split out the JSON types used when includeReasoning is true v/s false.
- prefer explicit types to var (PR feedback)
- prefer span-copy to buffer block-copy (PR feedback)
…5978)

* Add vsix to @(ItemsToPushToBlobFeed)

* Explicitly specify VSIX file to sign
* Updated name and new sample data

* Updated with sample data for the Preview 1.

* Fixing PDFs to work with Azure AI Search, adding GitHub Models support

* Updated the "welcome" message to provide better guidance

Updated the description to help developer's understand this is just example data that they should replace with their own.

Fixed broken citation links (due to the PDF file rename in this PR).
* Add third party notices

* Add .NET third party dependencies
* HybridCache JSON serializer:

- support value-tuples (requires field mode)
- support global custom JSON options via keyed DI

fix #5716
fix #5827

* - fix PR nits (var, comment whitespace)
- add support for per-type and global JSON options registration
- fix up unrelated devenv noise in BufferReleaseTests
* Address missing reference to Microsoft.Extensions.AI.OpenAI.
@RussKie
Copy link
Member

RussKie commented Feb 28, 2025

@stephentoub could you lend me a hand? I'm struggling to resolve the JSON dependencies

/mnt/vss/_work/1/s/src/Shared/JsonSchemaExporter/JsonSchemaExporter.cs(50,95): warning CS0436: The type 'JsonSchemaExporterOptions' in '/mnt/vss/_work/1/s/eng/MSBuild/../../src/Shared/JsonSchemaExporter/JsonSchemaExporterOptions.cs' conflicts with the imported type 'JsonSchemaExporterOptions' in 'System.Text.Json, Version=10.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51'. Using the type defined in '/mnt/vss/_work/1/s/eng/MSBuild/../../src/Shared/JsonSchemaExporter/JsonSchemaExporterOptions.cs'. [/mnt/vss/_work/1/s/src/Libraries/Microsoft.Extensions.AI.Abstractions/Microsoft.Extensions.AI.Abstractions.csproj::TargetFramework=net462]
##[warning]src/Shared/JsonSchemaExporter/JsonSchemaExporter.cs(50,95): warning CS0436: (NETCORE_ENGINEERING_TELEMETRY=Build) The type 'JsonSchemaExporterOptions' in '/mnt/vss/_work/1/s/eng/MSBuild/../../src/Shared/JsonSchemaExporter/JsonSchemaExporterOptions.cs' conflicts with the imported type 'JsonSchemaExporterOptions' in 'System.Text.Json, Version=10.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51'. Using the type defined in '/mnt/vss/_work/1/s/eng/MSBuild/../../src/Shared/JsonSchemaExporter/JsonSchemaExporterOptions.cs'.
/mnt/vss/_work/1/s/src/Shared/JsonSchemaExporter/JsonSchemaExporter.cs(70,76): warning CS0436: The type 'JsonSchemaExporterOptions' in '/mnt/vss/_work/1/s/eng/MSBuild/../../src/Shared/JsonSchemaExporter/JsonSchemaExporterOptions.cs' conflicts with the imported type 'JsonSchemaExporterOptions' in 'System.Text.Json, Version=10.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51'. Using the type defined in '/mnt/vss/_work/1/s/eng/MSBuild/../../src/Shared/JsonSchemaExporter/JsonSchemaExporterOptions.cs'. [/mnt/vss/_work/1/s/src/Libraries/Microsoft.Extensions.AI.Abstractions/Microsoft.Extensions.AI.Abstractions.csproj::TargetFramework=net462]
##[warning]src/Shared/JsonSchemaExporter/JsonSchemaExporter.cs(70,76): warning CS0436: (NETCORE_ENGINEERING_TELEMETRY=Build) The type 'JsonSchemaExporterOptions' in '/mnt/vss/_work/1/s/eng/MSBuild/../../src/Shared/JsonSchemaExporter/JsonSchemaExporterOptions.cs' conflicts with the imported type 'JsonSchemaExporterOptions' in 'System.Text.Json, Version=10.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51'. Using the type defined in '/mnt/vss/_work/1/s/eng/MSBuild/../../src/Shared/JsonSchemaExporter/JsonSchemaExporterOptions.cs'.

@stephentoub
Copy link
Member

What is the dev branch and why are we trying to merge main into it?

@RussKie
Copy link
Member

RussKie commented Mar 1, 2025

Dev branch targets .NET 10. Gradual forward-merge helps us reducing the future toll of retargeting the codebase to .NET 10.

Additionally, it helps fast-tracking the resolution of errors, as the original authors are likely to have the context fresh in their minds. This expedites the process compared to locating the original authors later and requesting them to address the issue some months later.

@stephentoub
Copy link
Member

Dev branch targets .NET 10.

Ok, thanks.

could you lend me a hand? I'm struggling to resolve the JSON dependencies

Microsoft.Extensions.AI.Abstractions needs some functionality that's new to System.Text.Json 9.0, but .Abstractions targets the latest LTS of STJ, which in main is 8.0, which lacks that functionality, and so it gets built in via source copy:

<InjectJsonSchemaExporterOnLegacy>true</InjectJsonSchemaExporterOnLegacy>

With the dev branch being for .NET 10, presumably it's bumping the STJ from 8.0 to 10.0? In which case that STJ now exposes the relevant types. I expect we can delete the above line and anything in the branch that mentions InjectJsonSchemaExporterOnLegacy, along with the whole source folder it brings in.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.