Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[NativeAOT-LLVM] Merge oct 01 #2750

Open
wants to merge 72 commits into
base: feature/NativeAOT-LLVM
Choose a base branch
from

Conversation

yowl
Copy link
Contributor

@yowl yowl commented Nov 5, 2024

Dont set thread name for wasm (upstream 107943)
Exclude tests that require RhGetCommonStubAddress
Update from 9 - 10 in tests and using document
Add the AcdKeyDesignator parameter.

ViktorHofer and others added 30 commits September 25, 2024 11:28
* Update dotnet10 feeds

* Update hardcoded old TFM values

* More hardcode updates and TFM checks

* Updates and delete duplicate file in Bcl.Numerics

* Linker stays on net9.0

* Fix .NET SDK max supported version

* Update apicompat sfx apicompat baseline file

* Update Microsoft.NET.WebAssembly.Threading APICompat settings

* Fix DiagnosticSource build

* Update SDK to RC1 for needed GetAlternateLookup API

to be able to build System.Text.Json on net9.0.

* Fix APICompat runtime regex pattern

* Suppress Bcl.Memory compatibility error for dropped framework

* Fix BinaryFormatter tests regex pattern for version subsitution

* Update BinaryFormatterTests.cs

* Pin SdkVersionForWorkloadTesting to the version in global.json

* Treat net9 and net10 as current for now

* Bump MicrosoftDotNetILCompilerVersion

* Update BinaryFormatterTests.cs

* Update System.Resources.Extensions

* Update AutoImports as well

* Fix ImporterTests.cs

* Block failing SignalRClientTests

* Update XmlSerialzation Schema Import tests.

* Update Versions.props

* Update CompatibilitySuppressions.xml for System.Numerics.Tensors

* Don't set EnableDefaultCompileItems, it doesn't work since we set EnableDefaultItems=false higher up

* Update regex in legacy BinaryFormatterTests.cs too

* Hardcode BinaryFormatter test assembly version to 9.0

This allows us to not change the majority of the tests

* Fix typo

* Fix issue in Microsoft.NET.Workload.Mono.Toolchain.Current.Manifest.pkgproj

* Replace ForceNet8Current in a few more places

* Fix wasi workload

* Fix TreatAsCurrent for 10.0

* Add net9 workload manifests

* Update PackageId in icrosoft.NET.Runtime.WebAssembly.Templates.csproj

* Update ApiCompatBaseline.NetCoreAppLatestStable.xml

Changes from dotnet/runtime#107758

* Remove duplicate dotnet10 feeds from nuget.config

* Update Versions.props

* Fake SDK band version for now

* Update Versions.props

* Update the SDK band versions for real

* Update the dotnet/sdk dependencies

* Update emsdk to net10 workloads

* Install a 9.0 runtime for workload tests too

* Update the compat tasks

* Merge in changes from net10-manifests

* Tmp: Trying out @am11's branch.

* Tmp2: Trying out @am11's branch.

* Tmp3: Replace the last usage of main in clone action.

* Update eng/pipelines/coreclr/templates/build-perf-bdn-app.yml

Co-authored-by: Adeel Mujahid <[email protected]>

* Update eng/pipelines/coreclr/templates/perf-job.yml

Co-authored-by: Adeel Mujahid <[email protected]>

* Update eng/pipelines/runtime-wasm-perf.yml

Co-authored-by: Adeel Mujahid <[email protected]>

---------

Co-authored-by: Larry Ewing <[email protected]>
Co-authored-by: Jackson Schuster <[email protected]>
Co-authored-by: Alexander Köplinger <[email protected]>
Co-authored-by: Ilona Tomkowicz <[email protected]>
Co-authored-by: Steve Molloy <[email protected]>
Co-authored-by: Adeel Mujahid <[email protected]>
* Update dependencies from https://github.com/dotnet/arcade build 20240923.2

Microsoft.SourceBuild.Intermediate.arcade , Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Build.Tasks.Archives , Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Build.Tasks.Installers , Microsoft.DotNet.Build.Tasks.Packaging , Microsoft.DotNet.Build.Tasks.TargetFramework , Microsoft.DotNet.Build.Tasks.Templating , Microsoft.DotNet.Build.Tasks.Workloads , Microsoft.DotNet.CodeAnalysis , Microsoft.DotNet.GenAPI , Microsoft.DotNet.GenFacades , Microsoft.DotNet.Helix.Sdk , Microsoft.DotNet.PackageTesting , Microsoft.DotNet.RemoteExecutor , Microsoft.DotNet.SharedFramework.Sdk , Microsoft.DotNet.VersionTools.Tasks , Microsoft.DotNet.XliffTasks , Microsoft.DotNet.XUnitAssert , Microsoft.DotNet.XUnitConsoleRunner , Microsoft.DotNet.XUnitExtensions
 From Version 10.0.0-beta.24467.1 -> To Version 10.0.0-beta.24473.2

* Update dependencies from https://github.com/dotnet/arcade build 20240924.1

Microsoft.SourceBuild.Intermediate.arcade , Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Build.Tasks.Archives , Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Build.Tasks.Installers , Microsoft.DotNet.Build.Tasks.Packaging , Microsoft.DotNet.Build.Tasks.TargetFramework , Microsoft.DotNet.Build.Tasks.Templating , Microsoft.DotNet.Build.Tasks.Workloads , Microsoft.DotNet.CodeAnalysis , Microsoft.DotNet.GenAPI , Microsoft.DotNet.GenFacades , Microsoft.DotNet.Helix.Sdk , Microsoft.DotNet.PackageTesting , Microsoft.DotNet.RemoteExecutor , Microsoft.DotNet.SharedFramework.Sdk , Microsoft.DotNet.VersionTools.Tasks , Microsoft.DotNet.XliffTasks , Microsoft.DotNet.XUnitAssert , Microsoft.DotNet.XUnitConsoleRunner , Microsoft.DotNet.XUnitExtensions
 From Version 10.0.0-beta.24473.2 -> To Version 10.0.0-beta.24474.1

---------

Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>
… dotnet/icu, dotnet/source-build-externals (#108183)

* Update dependencies from https://github.com/dotnet/icu build 20240923.1

Microsoft.NETCore.Runtime.ICU.Transport
 From Version 10.0.0-alpha.1.24466.3 -> To Version 10.0.0-alpha.1.24473.1

* Update dependencies from https://github.com/dotnet/hotreload-utils build 20240923.1

Microsoft.DotNet.HotReload.Utils.Generator.BuildTool
 From Version 10.0.0-alpha.0.24466.1 -> To Version 10.0.0-alpha.0.24473.1

* Update dependencies from https://github.com/dotnet/cecil build 20240923.1

Microsoft.SourceBuild.Intermediate.cecil , Microsoft.DotNet.Cecil
 From Version 0.11.5-alpha.24467.1 -> To Version 0.11.5-alpha.24473.1

* Update dependencies from https://github.com/dotnet/source-build-externals build 20240923.2

Microsoft.SourceBuild.Intermediate.source-build-externals
 From Version 10.0.0-alpha.1.24466.1 -> To Version 10.0.0-alpha.1.24473.2

---------

Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>
…923.3 (#108191)

Microsoft.SourceBuild.Intermediate.roslyn , Microsoft.CodeAnalysis , Microsoft.CodeAnalysis.CSharp , Microsoft.Net.Compilers.Toolset
 From Version 4.12.0-3.24470.11 -> To Version 4.12.0-3.24473.3

Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>
* ARM64: Fix lsra for AdvSimd_LoadAndInsertScalar

* Fix comment

* Expand LoadAndInsertScalar testing

* Only delayfree when register types match

* fix formatting

* Fix comment

* Check that multiregister nodes use fp registers

* Revert rmw assert
* Bring back Microsoft.BCL.Memory netstandard2.1 facade

* Reorder TargetFrameworks

Co-authored-by: Larry Ewing <[email protected]>

---------

Co-authored-by: Larry Ewing <[email protected]>
* Fix handling of appending keywords to boolean schemas.

* Add a few comments documenting boolean schemas.
* working

* comments from PR

* can always reshape to self

* fixed tests

* comments from PR

* fixing tests
Match what we do for other android RIDs.

The _Unwind_Backtrace support is only used when llvmonly mode is
enabled, which is not something that we do for Android
* Added tests for server response DirectoryControls

* Addressing platform inconsistencies

* There are looser rules in OpenLDAP when an octet string exceeds the length of its containing sequence.
* Windows 8.1 and below treats trailing 0x80 bytes differently to every other platform.
* Added a test which covers an octet string containing invalid Unicode bytes.

* Handle differing .NET Framework behaviour

Also handle changes to the "a" format string: Winldap in Windows 10 is stricter when a SEQUENCE's contents overflow its length

* Update SortResponseControlTests.cs
…925.5 (#108277)

Microsoft.SourceBuild.Intermediate.arcade , Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Build.Tasks.Archives , Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Build.Tasks.Installers , Microsoft.DotNet.Build.Tasks.Packaging , Microsoft.DotNet.Build.Tasks.TargetFramework , Microsoft.DotNet.Build.Tasks.Templating , Microsoft.DotNet.Build.Tasks.Workloads , Microsoft.DotNet.CodeAnalysis , Microsoft.DotNet.GenAPI , Microsoft.DotNet.GenFacades , Microsoft.DotNet.Helix.Sdk , Microsoft.DotNet.PackageTesting , Microsoft.DotNet.RemoteExecutor , Microsoft.DotNet.SharedFramework.Sdk , Microsoft.DotNet.VersionTools.Tasks , Microsoft.DotNet.XliffTasks , Microsoft.DotNet.XUnitAssert , Microsoft.DotNet.XUnitConsoleRunner , Microsoft.DotNet.XUnitExtensions
 From Version 10.0.0-beta.24474.1 -> To Version 10.0.0-beta.24475.5

Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>
…926.1 (#108280)

Microsoft.SourceBuild.Intermediate.roslyn , Microsoft.CodeAnalysis , Microsoft.CodeAnalysis.CSharp , Microsoft.Net.Compilers.Toolset
 From Version 4.12.0-3.24473.3 -> To Version 4.12.0-3.24476.1

Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>
…25.1 (#108279)

Microsoft.SourceBuild.Intermediate.cecil , Microsoft.DotNet.Cecil
 From Version 0.11.5-alpha.24473.1 -> To Version 0.11.5-alpha.24475.1

Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>
…24.1 (#108278)

Microsoft.SourceBuild.Intermediate.emsdk , Microsoft.NET.Workload.Emscripten.Current.Manifest-10.0.100.Transport
 From Version 10.0.0-alpha.1.24468.9 -> To Version 10.0.0-alpha.1.24474.1

Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>
We used to stackalloc a lot of data because the Target callback had to be an UnmanagedCallersOnly function pointer. But now it's just a delegate. So we can heap allocate the mock memory space and get rid of a bunch of incidental fixed statements
We used to spell out the JSON contract descriptor in every test case. We dont' need to do that - most tests are not testing parsing. Hide all that stuff in MockMemorySpace.Builder

* make MockMemorySpace.ReadContext a class

   Now that Target's reader callback is just a normal managed delegate, we don't have a lot of reason to make the testing ReadContext a ref struct.  So simplify our resource usage and just store the descriptors and json as heap allocated arrays

* remove uses of fixed in tests

   The ReadContext for the  tests keeps a copy of the json payload, we don't need to use unmanaged pointers

* make the MockMemorySpace.Builder a normal class

   we don't hold pointers to span data anymore

* make MockMemorySpace.ReadContext private

* begin hiding ContractDescriptorFill

   most of our tests that use a valid contract descriptor don't need to repeat the boilerplate to fill it in

* start removing json and pointer data boilerplate

* Don't fill the pointer data when there isn't any

   We previously did the wrong thing with TargetTestHelpers.MakeGlobalJson: it always filled in direct values.  So technically filling in the pointer data in all the other tests is unnecessary.

   Add a second SetGlobals overload for the builder that takes a 4-tuple (either ulong? Value or uint IndirectIndex) and a second collection of actual values for the indices.

* remove unused MockMemorySpace.Builder functions

   everything is using high level construction now

* remove unneeded stackalloc

* describe test addresses
reviously we used EmbeddedResource to collect the contract descriptor baselines into cdac-build-tool and we also embedded the C source template.

But we only ever run cdac-build-tool during the runtime build - the files are in the same source tree. Just pass their paths when invoking the tool, don't embed them.

Fixes #107958 

* don't embed baselines into cdac-build-tool

* don't embed the template source file in the tool
* Remove HelperMethodFrame from Delegate construction
Root the System.Runtime EventSource

The System.Runtime EventSource (RuntimeEventSource), was unintentionally being garbage collected because it wasn't rooted. This caused runtime EventCounters to no longer be available after GC occurred.

This was a regression from a recent change (dotnet/runtime#106014). That change accidentally converted the static field that was intended to the root the object into a property getter that returned a new instance each time it was called. This fix converts the property back to being initialized only once.

This will fix #107919 once it is backported.

Co-authored-by: Tarek Mahmoud Sayed <[email protected]>
…ser` templates (#108271)

* Clean up icu tests (automatic mode).

* Block the failing test, mark problems with issue links.
…8187)

Enable and disable tests for hybrid globalization on Apple
dotnet-maestro bot and others added 24 commits September 30, 2024 13:03
…ence-packages build 20240923.1 (#108389)

Microsoft.SourceBuild.Intermediate.source-build-reference-packages
 From Version 10.0.0-alpha.1.24467.1 -> To Version 10.0.0-alpha.1.24473.1

Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>
Break up the monolithic cdacreader assembly into four parts:

1. `Microsoft.Diagnostics.DataContractReader.Abstractions` just the API surface for contract implementations and clients. **Note**: everything is internal for now (with IVT for the other assemblies) - we're not committing to a public API surface yet
2. `Microsoft.Diagnostics.DataContractReader.Contracts`: the concrete implementations of the contracts and data
3. `Microsoft.Diagnostics.DataContractReader`: a concrete Target that ties everything together
4. `cdacreader`: just the unmanaged entrypoints and the legacy DAC API surface `SOSDacImpl`

To untangle things I had to add a new `IContractFactory<TContract>` interface - this is what the target's ContractRegistry uses to instantiate specific versions of contracts.

Goals:

* Make it possible to mock a Target and its ContractRegistry so that concrete contracts can be tested in isolation for example by making dummy dependent contracts that return canned answers.
* Eventually make it possible to inject additional contract implementations into a ContractRegistry implementation
Make it possible to consume just the Target and Contracts without the unmanaged entrypoints or the legacy interfaces


Changes:
* [cdac] break up cdacreader into 4 separate assemblies
* rename the contract factories using libraries naming convention
* removed unused usings
* document all abstract Target members
* rename Target -> ContractDescriptorTarget
* Add ReadTargetPointerFromSpan to abstract Target
   Allows the TypeNameBuidler and SigFormat to depend on the abstract target
* change SOSDacImpl to depend on the abstract Target
* fixup filenames and namespaces
These had to be excluded in #108235 but #108328 fixed them in the meantime. One of these PRs might be serviced for .NET 9 and the other may not, so I'm just reenabling this in a separate PR to make backports easier.
When we clone a finally or remove an empty try, the cloned handler
entry or former try region entry no longer need special protection.
Include basic descriptions of host error codes in `error_codes.h`. Slight clean-up (and some renames) of some error codes - these are all failures with no expected programmatic recovery, so the value change should be fine:
- Merged `LibHostCurExeFindFailure` with `CurrentHostFindFailure`
- Merged `LibHostSdkFindFailure` with `SdkResolveFailure`
- Merged `LibHostUnknownCommand` with `LibHostInvalidArgs`
* Set unwind table kind

To stop hitting assert in the verbose output, when aot compiler is built
with the debug configuration of llvm

* Set synchronous uw table kind on arm64

* Feedback
* [swift interop] Add inline array tests

* Rename test suite
Some more cases where we only have the one app domain and we don't need to pass it all over.
* remove dead code from BCL.cs

* Update src/tools/illink/src/linker/Linker/BCL.cs

* Update BCL.cs
This code was trying to answer question: "Was this method overriden by something else in a more derived class"? It was walking the base hierarchy in canonical form, but that was leading to methods not resolving at all. The fix is to walk the non-canonical hierarchy and canonicalize after we resolved everything.

I ran into this in #108379 that unlocked more whole program devirtualization and `StringSearchValuesBase` is in this shape.
…n (#108430)

* Restore System.Formats.Nrbf package description alongside package README

* Add missing struct members to ref source

* Update ref to match dotnet msbuild /t:GenerateReferenceAssemblySource

* Remove 'EnableDefaultPackageReadmeFile' since 'true' is the default
…t-xx

# Conflicts:
#	src/coreclr/jit/compiler.h
#if out pthread_setname_np
remove tests that require RhGetCommonStubAddress
@yowl
Copy link
Contributor Author

yowl commented Nov 8, 2024

retry, works locally

@yowl yowl closed this Nov 8, 2024
@yowl yowl reopened this Nov 8, 2024
@yowl yowl marked this pull request as ready for review November 8, 2024 17:02
@yowl
Copy link
Contributor Author

yowl commented Nov 8, 2024

cc @dotnet/nativeaot-llvm

(Dont know why the previous run failed on missing jit_interface dll, worked 2nd time)

@yowl
Copy link
Contributor Author

yowl commented Nov 8, 2024

Merging small amounts as there seems to be a GC problem occurring at some point, and less commits to look through will make it easier to find, I hope.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.