Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[NativeAOT-LLVM] Merge oct 24 #2745

Draft
wants to merge 239 commits into
base: feature/NativeAOT-LLVM
Choose a base branch
from

Conversation

yowl
Copy link
Contributor

@yowl yowl commented Oct 31, 2024

This PR merges forward to October 16.

Update .net from 9 to 10

Disable IDynamicInterfaceCastable tests as they now use a thunk pool (and RhGetCommonStubAddress which we haven't implemented) dotnet/runtime#108235

Cherry pick the PR dotnet/runtime#109224 to avoid having to do another merge.

Remove if around call to SetExpandedEarly in importCall as we assume virtual vtable calls are expanded into gtControlExpr

The second merge is merging from NativeAOT-LLVM and essentially just removes a comment (after fixing a duplicate line removed in the last commit)

Last commit is reverting some formatting that VSCode decided it wanted.

ViktorHofer and others added 30 commits September 25, 2024 11:28
* Update dotnet10 feeds

* Update hardcoded old TFM values

* More hardcode updates and TFM checks

* Updates and delete duplicate file in Bcl.Numerics

* Linker stays on net9.0

* Fix .NET SDK max supported version

* Update apicompat sfx apicompat baseline file

* Update Microsoft.NET.WebAssembly.Threading APICompat settings

* Fix DiagnosticSource build

* Update SDK to RC1 for needed GetAlternateLookup API

to be able to build System.Text.Json on net9.0.

* Fix APICompat runtime regex pattern

* Suppress Bcl.Memory compatibility error for dropped framework

* Fix BinaryFormatter tests regex pattern for version subsitution

* Update BinaryFormatterTests.cs

* Pin SdkVersionForWorkloadTesting to the version in global.json

* Treat net9 and net10 as current for now

* Bump MicrosoftDotNetILCompilerVersion

* Update BinaryFormatterTests.cs

* Update System.Resources.Extensions

* Update AutoImports as well

* Fix ImporterTests.cs

* Block failing SignalRClientTests

* Update XmlSerialzation Schema Import tests.

* Update Versions.props

* Update CompatibilitySuppressions.xml for System.Numerics.Tensors

* Don't set EnableDefaultCompileItems, it doesn't work since we set EnableDefaultItems=false higher up

* Update regex in legacy BinaryFormatterTests.cs too

* Hardcode BinaryFormatter test assembly version to 9.0

This allows us to not change the majority of the tests

* Fix typo

* Fix issue in Microsoft.NET.Workload.Mono.Toolchain.Current.Manifest.pkgproj

* Replace ForceNet8Current in a few more places

* Fix wasi workload

* Fix TreatAsCurrent for 10.0

* Add net9 workload manifests

* Update PackageId in icrosoft.NET.Runtime.WebAssembly.Templates.csproj

* Update ApiCompatBaseline.NetCoreAppLatestStable.xml

Changes from dotnet/runtime#107758

* Remove duplicate dotnet10 feeds from nuget.config

* Update Versions.props

* Fake SDK band version for now

* Update Versions.props

* Update the SDK band versions for real

* Update the dotnet/sdk dependencies

* Update emsdk to net10 workloads

* Install a 9.0 runtime for workload tests too

* Update the compat tasks

* Merge in changes from net10-manifests

* Tmp: Trying out @am11's branch.

* Tmp2: Trying out @am11's branch.

* Tmp3: Replace the last usage of main in clone action.

* Update eng/pipelines/coreclr/templates/build-perf-bdn-app.yml

Co-authored-by: Adeel Mujahid <[email protected]>

* Update eng/pipelines/coreclr/templates/perf-job.yml

Co-authored-by: Adeel Mujahid <[email protected]>

* Update eng/pipelines/runtime-wasm-perf.yml

Co-authored-by: Adeel Mujahid <[email protected]>

---------

Co-authored-by: Larry Ewing <[email protected]>
Co-authored-by: Jackson Schuster <[email protected]>
Co-authored-by: Alexander Köplinger <[email protected]>
Co-authored-by: Ilona Tomkowicz <[email protected]>
Co-authored-by: Steve Molloy <[email protected]>
Co-authored-by: Adeel Mujahid <[email protected]>
* Update dependencies from https://github.com/dotnet/arcade build 20240923.2

Microsoft.SourceBuild.Intermediate.arcade , Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Build.Tasks.Archives , Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Build.Tasks.Installers , Microsoft.DotNet.Build.Tasks.Packaging , Microsoft.DotNet.Build.Tasks.TargetFramework , Microsoft.DotNet.Build.Tasks.Templating , Microsoft.DotNet.Build.Tasks.Workloads , Microsoft.DotNet.CodeAnalysis , Microsoft.DotNet.GenAPI , Microsoft.DotNet.GenFacades , Microsoft.DotNet.Helix.Sdk , Microsoft.DotNet.PackageTesting , Microsoft.DotNet.RemoteExecutor , Microsoft.DotNet.SharedFramework.Sdk , Microsoft.DotNet.VersionTools.Tasks , Microsoft.DotNet.XliffTasks , Microsoft.DotNet.XUnitAssert , Microsoft.DotNet.XUnitConsoleRunner , Microsoft.DotNet.XUnitExtensions
 From Version 10.0.0-beta.24467.1 -> To Version 10.0.0-beta.24473.2

* Update dependencies from https://github.com/dotnet/arcade build 20240924.1

Microsoft.SourceBuild.Intermediate.arcade , Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Build.Tasks.Archives , Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Build.Tasks.Installers , Microsoft.DotNet.Build.Tasks.Packaging , Microsoft.DotNet.Build.Tasks.TargetFramework , Microsoft.DotNet.Build.Tasks.Templating , Microsoft.DotNet.Build.Tasks.Workloads , Microsoft.DotNet.CodeAnalysis , Microsoft.DotNet.GenAPI , Microsoft.DotNet.GenFacades , Microsoft.DotNet.Helix.Sdk , Microsoft.DotNet.PackageTesting , Microsoft.DotNet.RemoteExecutor , Microsoft.DotNet.SharedFramework.Sdk , Microsoft.DotNet.VersionTools.Tasks , Microsoft.DotNet.XliffTasks , Microsoft.DotNet.XUnitAssert , Microsoft.DotNet.XUnitConsoleRunner , Microsoft.DotNet.XUnitExtensions
 From Version 10.0.0-beta.24473.2 -> To Version 10.0.0-beta.24474.1

---------

Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>
… dotnet/icu, dotnet/source-build-externals (#108183)

* Update dependencies from https://github.com/dotnet/icu build 20240923.1

Microsoft.NETCore.Runtime.ICU.Transport
 From Version 10.0.0-alpha.1.24466.3 -> To Version 10.0.0-alpha.1.24473.1

* Update dependencies from https://github.com/dotnet/hotreload-utils build 20240923.1

Microsoft.DotNet.HotReload.Utils.Generator.BuildTool
 From Version 10.0.0-alpha.0.24466.1 -> To Version 10.0.0-alpha.0.24473.1

* Update dependencies from https://github.com/dotnet/cecil build 20240923.1

Microsoft.SourceBuild.Intermediate.cecil , Microsoft.DotNet.Cecil
 From Version 0.11.5-alpha.24467.1 -> To Version 0.11.5-alpha.24473.1

* Update dependencies from https://github.com/dotnet/source-build-externals build 20240923.2

Microsoft.SourceBuild.Intermediate.source-build-externals
 From Version 10.0.0-alpha.1.24466.1 -> To Version 10.0.0-alpha.1.24473.2

---------

Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>
…923.3 (#108191)

Microsoft.SourceBuild.Intermediate.roslyn , Microsoft.CodeAnalysis , Microsoft.CodeAnalysis.CSharp , Microsoft.Net.Compilers.Toolset
 From Version 4.12.0-3.24470.11 -> To Version 4.12.0-3.24473.3

Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>
* ARM64: Fix lsra for AdvSimd_LoadAndInsertScalar

* Fix comment

* Expand LoadAndInsertScalar testing

* Only delayfree when register types match

* fix formatting

* Fix comment

* Check that multiregister nodes use fp registers

* Revert rmw assert
* Bring back Microsoft.BCL.Memory netstandard2.1 facade

* Reorder TargetFrameworks

Co-authored-by: Larry Ewing <[email protected]>

---------

Co-authored-by: Larry Ewing <[email protected]>
* Fix handling of appending keywords to boolean schemas.

* Add a few comments documenting boolean schemas.
* working

* comments from PR

* can always reshape to self

* fixed tests

* comments from PR

* fixing tests
Match what we do for other android RIDs.

The _Unwind_Backtrace support is only used when llvmonly mode is
enabled, which is not something that we do for Android
* Added tests for server response DirectoryControls

* Addressing platform inconsistencies

* There are looser rules in OpenLDAP when an octet string exceeds the length of its containing sequence.
* Windows 8.1 and below treats trailing 0x80 bytes differently to every other platform.
* Added a test which covers an octet string containing invalid Unicode bytes.

* Handle differing .NET Framework behaviour

Also handle changes to the "a" format string: Winldap in Windows 10 is stricter when a SEQUENCE's contents overflow its length

* Update SortResponseControlTests.cs
…925.5 (#108277)

Microsoft.SourceBuild.Intermediate.arcade , Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Build.Tasks.Archives , Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Build.Tasks.Installers , Microsoft.DotNet.Build.Tasks.Packaging , Microsoft.DotNet.Build.Tasks.TargetFramework , Microsoft.DotNet.Build.Tasks.Templating , Microsoft.DotNet.Build.Tasks.Workloads , Microsoft.DotNet.CodeAnalysis , Microsoft.DotNet.GenAPI , Microsoft.DotNet.GenFacades , Microsoft.DotNet.Helix.Sdk , Microsoft.DotNet.PackageTesting , Microsoft.DotNet.RemoteExecutor , Microsoft.DotNet.SharedFramework.Sdk , Microsoft.DotNet.VersionTools.Tasks , Microsoft.DotNet.XliffTasks , Microsoft.DotNet.XUnitAssert , Microsoft.DotNet.XUnitConsoleRunner , Microsoft.DotNet.XUnitExtensions
 From Version 10.0.0-beta.24474.1 -> To Version 10.0.0-beta.24475.5

Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>
…926.1 (#108280)

Microsoft.SourceBuild.Intermediate.roslyn , Microsoft.CodeAnalysis , Microsoft.CodeAnalysis.CSharp , Microsoft.Net.Compilers.Toolset
 From Version 4.12.0-3.24473.3 -> To Version 4.12.0-3.24476.1

Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>
…25.1 (#108279)

Microsoft.SourceBuild.Intermediate.cecil , Microsoft.DotNet.Cecil
 From Version 0.11.5-alpha.24473.1 -> To Version 0.11.5-alpha.24475.1

Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>
…24.1 (#108278)

Microsoft.SourceBuild.Intermediate.emsdk , Microsoft.NET.Workload.Emscripten.Current.Manifest-10.0.100.Transport
 From Version 10.0.0-alpha.1.24468.9 -> To Version 10.0.0-alpha.1.24474.1

Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>
We used to stackalloc a lot of data because the Target callback had to be an UnmanagedCallersOnly function pointer. But now it's just a delegate. So we can heap allocate the mock memory space and get rid of a bunch of incidental fixed statements
We used to spell out the JSON contract descriptor in every test case. We dont' need to do that - most tests are not testing parsing. Hide all that stuff in MockMemorySpace.Builder

* make MockMemorySpace.ReadContext a class

   Now that Target's reader callback is just a normal managed delegate, we don't have a lot of reason to make the testing ReadContext a ref struct.  So simplify our resource usage and just store the descriptors and json as heap allocated arrays

* remove uses of fixed in tests

   The ReadContext for the  tests keeps a copy of the json payload, we don't need to use unmanaged pointers

* make the MockMemorySpace.Builder a normal class

   we don't hold pointers to span data anymore

* make MockMemorySpace.ReadContext private

* begin hiding ContractDescriptorFill

   most of our tests that use a valid contract descriptor don't need to repeat the boilerplate to fill it in

* start removing json and pointer data boilerplate

* Don't fill the pointer data when there isn't any

   We previously did the wrong thing with TargetTestHelpers.MakeGlobalJson: it always filled in direct values.  So technically filling in the pointer data in all the other tests is unnecessary.

   Add a second SetGlobals overload for the builder that takes a 4-tuple (either ulong? Value or uint IndirectIndex) and a second collection of actual values for the indices.

* remove unused MockMemorySpace.Builder functions

   everything is using high level construction now

* remove unneeded stackalloc

* describe test addresses
reviously we used EmbeddedResource to collect the contract descriptor baselines into cdac-build-tool and we also embedded the C source template.

But we only ever run cdac-build-tool during the runtime build - the files are in the same source tree. Just pass their paths when invoking the tool, don't embed them.

Fixes #107958 

* don't embed baselines into cdac-build-tool

* don't embed the template source file in the tool
* Remove HelperMethodFrame from Delegate construction
Root the System.Runtime EventSource

The System.Runtime EventSource (RuntimeEventSource), was unintentionally being garbage collected because it wasn't rooted. This caused runtime EventCounters to no longer be available after GC occurred.

This was a regression from a recent change (dotnet/runtime#106014). That change accidentally converted the static field that was intended to the root the object into a property getter that returned a new instance each time it was called. This fix converts the property back to being initialized only once.

This will fix #107919 once it is backported.

Co-authored-by: Tarek Mahmoud Sayed <[email protected]>
…ser` templates (#108271)

* Clean up icu tests (automatic mode).

* Block the failing test, mark problems with issue links.
…8187)

Enable and disable tests for hybrid globalization on Apple
vcsjones and others added 24 commits October 15, 2024 12:48
Newer versions of Clang warn when you are doing enum comparisons of different enum types.
The TLS helper expansion would produce `ADD(long, int)` trees like
```
▌  STORE_LCL_VAR long   V01 rat0
└──▌  ADD       long
   ├──▌  ADD       long
   │  ├──▌  LCL_VAR   long   V02 rat1
   │  └──▌  CNS_INT   int    72 $41
   └──▌  CNS_INT   long   192
```

which is not legal IR. Fix that by inserting a cast. Also do some more
aggressive folding to get rid of these casts in the normal cases, and to
fold some constant arithmeticf in other cases.
…(#108873)

It should be _extremely_ rare to need to disable a test during SuperPMI
collection. However, one case was found, so introduce the ability to do so.

Use `<SuperPmiCollectIncompatible>true</SuperPmiCollectIncompatible>` in
a test project file.

(which also requires `<RequiresProcessIsolation>true</RequiresProcessIsolation>`)

Fixes #108215
This change is some preparatory refactoring for the randomized allocation sampling feature. We need to add more state onto allocation context but we don't want to do a breaking change of the GC interface. The new state only needs to be visible to the EE but we want it physically near the existing alloc context state for good cache locality. To accomplish this we created a new ee_alloc_context struct which contains an instance of gc_alloc_context within it.

The new ee_alloc_context.combined_limit field should be used by fast allocation helpers to determine when to go down the slow path. Most of the time combined_limit has the same value as alloc_limit, but periodically we need to emit an allocation sampling event on an object that is somewhere in the middle of an AC. Using combined_limit rather than alloc_limit as the slow path trigger allows us to keep all the sampling event logic in the slow path.

PR feedback Co-authored-by: Jan Kotas <[email protected]>
* Move IsDebuggerPresent to minipal, convert to QCall

---------

Co-authored-by: Aaron Robinson <[email protected]>
…14.2 (#108884)

Microsoft.SourceBuild.Intermediate.emsdk , Microsoft.NET.Workload.Emscripten.Current.Manifest-10.0.100.Transport
 From Version 10.0.0-alpha.1.24508.1 -> To Version 10.0.0-alpha.1.24514.2

Dependency coherency updates

runtime.linux-arm64.Microsoft.NETCore.Runtime.JIT.Tools,runtime.linux-x64.Microsoft.NETCore.Runtime.JIT.Tools,runtime.linux-musl-arm64.Microsoft.NETCore.Runtime.JIT.Tools,runtime.linux-musl-x64.Microsoft.NETCore.Runtime.JIT.Tools,runtime.win-arm64.Microsoft.NETCore.Runtime.JIT.Tools,runtime.win-x64.Microsoft.NETCore.Runtime.JIT.Tools,runtime.osx-arm64.Microsoft.NETCore.Runtime.JIT.Tools,runtime.osx-x64.Microsoft.NETCore.Runtime.JIT.Tools,runtime.linux-arm64.Microsoft.NETCore.Runtime.Mono.LLVM.Sdk,runtime.linux-arm64.Microsoft.NETCore.Runtime.Mono.LLVM.Tools,runtime.linux-musl-arm64.Microsoft.NETCore.Runtime.Mono.LLVM.Sdk,runtime.linux-musl-arm64.Microsoft.NETCore.Runtime.Mono.LLVM.Tools,runtime.linux-x64.Microsoft.NETCore.Runtime.Mono.LLVM.Sdk,runtime.linux-x64.Microsoft.NETCore.Runtime.Mono.LLVM.Tools,runtime.linux-musl-x64.Microsoft.NETCore.Runtime.Mono.LLVM.Sdk,runtime.linux-musl-x64.Microsoft.NETCore.Runtime.Mono.LLVM.Tools,runtime.win-x64.Microsoft.NETCore.Runtime.Mono.LLVM.Sdk,runtime.win-x64.Microsoft.NETCore.Runtime.Mono.LLVM.Tools,runtime.osx-arm64.Microsoft.NETCore.Runtime.Mono.LLVM.Sdk,runtime.osx-arm64.Microsoft.NETCore.Runtime.Mono.LLVM.Tools,runtime.osx-x64.Microsoft.NETCore.Runtime.Mono.LLVM.Sdk,runtime.osx-x64.Microsoft.NETCore.Runtime.Mono.LLVM.Tools
 From Version 19.1.0-alpha.1.24501.1 -> To Version 19.1.0-alpha.1.24510.2 (parent: Microsoft.NET.Workload.Emscripten.Current.Manifest-10.0.100.Transport

Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>
This change is some preparatory refactoring for the randomized allocation sampling feature. We need to add more state onto allocation context but we don't want to do a breaking change of the GC interface. The new state only needs to be visible to the EE but we want it physically near the existing alloc context state for good cache locality. To accomplish this we created a new ee_alloc_context struct which contains an instance of gc_alloc_context within it.

The new ee_alloc_context::combined_limit should be used by fast allocation helpers to determine when to go down the slow path. Most of the time combined_limit has the same value as alloc_limit, but periodically we need to emit an allocation sampling event on an object that is somewhere in the middle of an AC. Using combined_limit rather than alloc_limit as the slow path trigger allows us to keep all the sampling event logic in the slow path.

* PR feedback

Co-authored-by: Jan Kotas <[email protected]>
* Do not run tests with v8 scenario on CI. Leave code that supports running v8 scenario.

* Remove console app, consolidate template base class, order inheritance, clean the code.

* Fix build error: Keep all types of bundle options in `Common` dir.

* Rename the param after merge with main.

* Fix cast errors in providers.

* Blazor and Browser bundle names got unified some time ago.

* Fix running of browser published apps.

* Fix the test transformed from console to browser.

* Browser sdk does not allow appending RID.

* Fix "item was already added" in build.
* Consolidate LocateNativeCompiler target

* Set install_name on all Apple platforms

---------

Co-authored-by: Jan Kotas <[email protected]>
* Generate vectorized left shift for Vector128<Byte>

* Use lower ISA instead

* Expand for larger size vector widths

* jit format

* Move transformation to gtNewSimdBinOp

* Remove conditional, use assert similar to GT_RSZ case

* Fix missing semicolon

* Use fgMakeMultiUse instead of gtCloneExpr, re-use previous shift count calculation

* Consolidate GT_LSH path into same path as GT_RSZ

* Use TYP_INT instead of TYP_SHORT for GT_LSH

* Also only use CORINFO_TYPE_INT

---------

Co-authored-by: Alex Covington (Advanced Micro Devices <[email protected]>
Co-authored-by: Tanner Gooding <[email protected]>
- Replace them with using the normal monitor helpers
- Change CORINFO_HELP_GETSYNCFROMCLASSHANDLE to return an `object` instead of the underlying lock object, implementation of the JIT helper can be shared with other `TypeHandle` to `RuntimeType` scenarios
- This will likely improve the performance of synchronized methods, but that is beside the point, since this feature is so rarely used
* Add -ANAME=xxx option to ilasm
* Update IL SDK to add SynthesizeIlasmAssemblyName property
* Update one test subfolder to use new property
# Conflicts:
#	.github/policies/resourceManagement.yml
#	eng/liveBuilds.targets
#	eng/pipelines/runtimelab.yml
#	src/coreclr/jit/compiler.h
#	src/coreclr/tools/aot/ILCompiler.Compiler/IL/ILImporter.Scanner.cs
#	src/tests/Common/Directory.Build.targets
#	src/tests/Directory.Build.targets
update docs 9->10

resolve conflicts
Co-authored-by: egorbot <[email protected]>
# Conflicts:
#	src/coreclr/jit/assertionprop.cpp
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.