Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[NativeAOT-LLVM] Merge feb25 #2993

Open
wants to merge 1,274 commits into
base: feature/NativeAOT-LLVM
Choose a base branch
from
Open

Conversation

yowl
Copy link
Contributor

@yowl yowl commented Feb 12, 2025

Merge from upstream b1ab309 .

I think it is esasier to read the commits in reverse order because of a couple of mistakes I made along the way.

The commit history is one way to see the issues, but here is a summary

  • Fix linux build by removing extern from main in test.
  • Add back some Rational IR specific bits in ssabuilder
  • Updates for removal of "init BB". I didn't leave in the possible optimisation for excluding an empty first block from the unhandled exception handler as I hit a few issues. For this EH, I set bbRefs to 1 for the filter and handler.
  • Update Wasi SDK to 25
  • Remove unsupported --no-lock from brew: presume this is also upstream in a later commit, but didn't check.
  • Reinstate GS Stress as RhpGcStressOnce and CORINFO_HELP_LLVM_STRESS_GC
  • Exclude TestRuntime109496Regression from the Interfaces test, we don't support RhGetCommonStubAddress.
  • A little churn around ReflectionExecution.ConvertStackTraceIpToFunctionPointer
  • Exclude Preinitialization test , looks like an uncovered bug in codegen for floats, but not sure.
  • Don't set TargetsMobile for Wasm, instead add checks for TargetsWasm
  • We now install aotminipal

tomeksowi and others added 30 commits January 24, 2025 15:30
This test was marked SkipOnPlatform for Android because the test would fail. However,
it turns out that Android does support empty subjects in certificates. If the subject
is empty, then the SubjectAltName extension must be marked critical. This is in accordance
with RFC 5280:

> If the subject field contains an empty sequence, then the issuing CA MUST include a subjectAltName extension that is marked as critical.

With a critical SAN extension, this test now passes on all platforms.
…d (#111821)

Before this change the RuntimeFlavor defaulted to Mono and as a result the right clr subsets were not substituted.
Part of #107749. For PGO scenarios, we plan to rely entirely on profile synthesis to propagate profile-derived weights throughout the flowgraph. This is the first step in transitioning away from the existing weight propagation heuristics. We want to keep these around for non-PGO scenarios for now, though ideally, we'll eventually be able to replace these entirely with profile synthesis (plus branch prediction heuristics when we don't have profile data), and consolidate how the profile is maintained for PGO and non-PGO scenarios.
Thanks to #95234, RBO can draw inferences when the same value is compared to
different constants, if the initial comparison dominates and was false. Generalize this
to also handle cases where the initial comparison dominates and is true.

Fixes #111725.
* Remove redundant check

* Add R2R assertion

* Fix isReadyToRun check

* Oops
This ensures that when a constant vector is replaced with a broadcast node, the replacement node is of the correct size.  Since only the scalar is read from the broadcast node, SIMD size doesn't really matter for codegen, but this fix satisfies the assert in #111613 by ensuring we replace like for like.
* Clean up Stopwatch a bit

- Remove unnecessary Reset call from ctor
- Remove unnecessary branch for _elapsed < 0
- Remove some defunct comments
- Clean up style of Stop to match that of Start

* More stylistic cleanup

- Use expression-bodied properties
- Remove unnecessary Debug.Asserts
- Remove private method whose impl could just have been that of an existing public property
- Change a private GetXx method to an Xx property.
- Remove some type names that could be inferred
…1664)

* Fix linux build image in eng/pipelines/libraries/stress/http.yml

* Align docker-compose.yml with the one from SslStress

* Fix copy/paste mistake

* Set correct WORKDIR

* fix

* Disable asan

---------

Co-authored-by: Radek Zikmund <[email protected]>
This definition was needed for portable builds against CentOS baseline. We do not need it anymore now that portable builds are done against Ubuntu 16 baseline that includes it.
Reenable long Process.Name test on OSX

Fixes #111460
Fixes #29330
…olved on Windows (#111711)

* Enable the test after updating the windows image.
* Check for concurrent IO is now thread safe.

* Add test

* Use compare exchange, fix cancellation token registration

* Typo

* Fixed unit tests.

* Test exception narrow down

* Narrow down the exception

* Don't dispose handle outside of lock
* Fix alternate stack overflow with superpmi

The superpmi-shim-collector shared library registers sigsegv handler
so that it can flush out the logged data in case the JIT crashes
due to some null reference. The sigsegv handler of the superpmi then
calls the previous handler, which is the one of coreclr. Both of these
handlers run on the alternate stack. Each sigsegv handler has stack
frame of about 0xd70 bytes mostly due to the `SignalHandlerWorkerReturnPoint`
structure that contains `CONTEXT` structure.
The alternate stack size that we allocate is then on the tipping point
of not being sufficient and some variation on how much of the alternate
stack is used by the unix kernel causes it to tip over in some cases.
This started to happen on x64 Linux after we've added the APX registers
to the context.

The problem actually happens due to the `SwitchStackAndExecuteHandler`
being inlined into the `sigsegv_handler`. That moves the large
`SignalHandlerWorkerReturnPoint` structure to the frame of the
`sigsegv_handler`.

This change fixes the problem by marking the
`SwitchStackAndExecuteHandler` as noinline. That makes the
`sigsegv_handler` frame significantly smaller.

Close #111707

* Add comment based on PR feedback
…erator (#111769)

The emitted IsMatched check should come after processing the child, not before.
The problem was always `// Note: We don't set the IsUnboxingStub flag on template methods (all template lookups performed at runtime are performed with this flag not set`, I tried working around it in the original fix, but looks like I actually need the function pointer in #111178 (the tests are not failing but I have a local test that does).

So trying an alternative approach that just deletes the weird code. It's possible this only had to be weird due to universal shared code.
* win32: add fallback to environment vars for system folder

Implement fallback to environment variables for system folders in NanoServer, ensuring correct paths when SHGetKnownFolderPath fails.

* feedback

* Update src/libraries/System.Private.CoreLib/src/System/Environment.Win32.cs

---------

Co-authored-by: Tanner Gooding <[email protected]>
* Improve docs on building ILVerify

* Update src/coreclr/tools/ILVerify/README.md

Co-authored-by: Jan Kotas <[email protected]>
…p (#108732)

* Avoid generic virtual dispatch for frozen collections alternate lookup

* Various stylistic changes addressing PR feedback

* Tidy up some more xmldocs

---------

Co-authored-by: Stephen Toub <[email protected]>
… Build ID 2629821 (#111884)

* Localized file check-in by OneLocBuild Task: Build definition ID 679: Build ID 2629686

* Localized file check-in by OneLocBuild Task: Build definition ID 679: Build ID 2629686
dotnet-maestro bot and others added 28 commits February 10, 2025 19:16
…ence-packages build 20250204.2 (#112339)

Microsoft.SourceBuild.Intermediate.source-build-reference-packages
 From Version 10.0.608004 -> To Version 10.0.610402

Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>
…819)

This PR adds an optimization in lowering to utilize the new parameter
register to local mappings added in #110795. The optimization detects IR
that is going to result in stack spills/loads and instead replaces them
with scalar locals that will be able to stay in registers.

Physical promotion benefits especially from this as it creates the kind
of IR that the optimization ends up kicking in for. The heuristics of
physical promotion are updated to account for the fact that the backend
is now able to do this optimization, making physical promotion more
likely to promote struct parameters.
* add hooks to debug OpenSSL memory

* opensslshim

* 1.x

* 1.0.1

* android

* Collections

* unsafe

* build

* feedback

* update

* feedback

* feedback

* feedback

* fix build

* gcc

* gcc

* Update src/native/libs/System.Security.Cryptography.Native/openssl.c

Co-authored-by: Adeel Mujahid <[email protected]>

* Move init to Interop.Crypto

* Fix data race on GetIncrementalAllocations

* Use modern tuple type

* Fix typo

* Move functionality to separate file

* Revert unnecessary changes

* WIP tracking in native code

* Improvements

* Reintroduce rw lock

* Add readme

* Fix build

* Code review feedback

* code review changes - cont.

* More code review feedback

* Expose the "API" via system.security.cryptography

* Fix infinite link list traversal

* Fix memory accounting for realloc

* Refactoring

* Improve comments

* Improve Readme

* Simplify implementation

* Don't use CRYPTO_atomic_add

* Readme fixes

* Fix build

* Fix compilation, attempt 2

* Fix build - attemt no 3

* Apply suggestions from code review

Co-authored-by: Jeremy Barton <[email protected]>

* Feedback

---------

Co-authored-by: wfurt <[email protected]>
Co-authored-by: Adeel Mujahid <[email protected]>
Co-authored-by: Jeremy Barton <[email protected]>
* Guard members of MonoType union behind type check helpers
* Add _unchecked to call sites that are obviously guarded correctly
* Fix type confusion in bulk_type_log_single_type
* Fix genericinst fallthrough treating a MonoGenericClass ptr as a MonoClass ptr
* Fix type confusion in amd64 mini
* Fix type confusion in aot-runtime-wasm
* Prune a dead goto to make unchecked safe
* Fix two cases where we were partially initializing a stack-allocated MonoType instead of fully initializing it
* See dotnet/runtime#112395 for detailed list of bugs fixed
* Build LoongArch64 and RiscV64 AltJit

* Add a new target for alljitscommunity
This change refactors the frame identification mechanism in the runtime by replacing the use of the VTable with an enum FrameIdentifier. The VTable-based approach more suited to C++ code, but presented challenges for assembly code and out-of-process inspections. By using FrameIdentifier, polymorphic dispatch is now handled via if-statements. This change also eliminates the need for GS cookies on Frame objects, as it is no longer necessary to protect the VTable.

Key changes include:

Renaming FrameType to FrameIdentifier
Introducing a GetFrameIdentifier method
This update streamlines frame management while improving performance and flexibility for various platforms.
There are race conditions in HTTP 1.1 response stream reading methods that may prevent the exceptional termination of those methods despite cancellation callbacks firing and disposing the connection, see the discussion starting with dotnet/runtime#31254 (comment).

As a result, these methods may call `CompleteResponse` returning the disposed connection to the pool.

This is a minimal fix for the mentioned issue by guarding against `_disposed` in `CompleteResponse`.

Fixes #31254
Fixes #110577
…387)

Otherwise we can end up with a use-after-free
# Conflicts:
#	eng/Subsets.props
#	eng/pipelines/official/stages/publish.yml
#	eng/pipelines/runtimelab-official.yml
#	eng/pipelines/runtimelab.yml
#	src/coreclr/CMakeLists.txt
#	src/coreclr/build-runtime.cmd
#	src/coreclr/components.cmake
#	src/coreclr/gc/unix/cgroup.cpp
#	src/coreclr/inc/jithelpers.h
#	src/coreclr/jit/CMakeLists.txt
#	src/coreclr/jit/compiler.cpp
#	src/coreclr/jit/compiler.h
#	src/coreclr/jit/gentree.cpp
#	src/coreclr/jit/morph.cpp
#	src/coreclr/jit/ssabuilder.cpp
#	src/coreclr/jit/ssabuilder.h
#	src/coreclr/jit/target.h
#	src/coreclr/nativeaot/BuildIntegration/BuildIntegration.proj
#	src/coreclr/nativeaot/BuildIntegration/Microsoft.DotNet.ILCompiler.SingleEntry.targets
#	src/coreclr/nativeaot/BuildIntegration/Microsoft.NETCore.Native.Publish.targets
#	src/coreclr/nativeaot/BuildIntegration/Microsoft.NETCore.Native.targets
#	src/coreclr/nativeaot/Runtime/EHHelpers.cpp
#	src/coreclr/nativeaot/Runtime/Portable/CMakeLists.txt
#	src/coreclr/nativeaot/Runtime/unix/NativeContext.h
#	src/coreclr/nativeaot/Runtime/unix/PalRedhawkUnix.cpp
#	src/coreclr/nativeaot/Runtime/unix/cgroupcpu.cpp
#	src/coreclr/nativeaot/System.Private.CoreLib/src/Internal/Reflection/Augments/ReflectionAugments.cs
#	src/coreclr/nativeaot/System.Private.CoreLib/src/System/Diagnostics/StackFrame.NativeAot.cs
#	src/coreclr/nativeaot/System.Private.CoreLib/src/System/Reflection/Runtime/General/ReflectionCoreCallbacksImplementation.cs
#	src/coreclr/nativeaot/System.Private.DisabledReflection/src/Internal/Reflection/ReflectionCoreCallbacksImplementation.cs
#	src/coreclr/nativeaot/System.Private.StackTraceMetadata/src/Internal/StackTraceMetadata/StackTraceMetadata.cs
#	src/coreclr/tools/Common/Compiler/InstructionSetSupport.cs
#	src/coreclr/tools/aot/ILCompiler.Compiler/Compiler/ExportsFileWriter.cs
#	src/coreclr/tools/aot/ILCompiler.Compiler/IL/ILImporter.Scanner.cs
#	src/coreclr/tools/aot/ILCompiler/ILCompiler.csproj
#	src/installer/pkg/projects/nativeaot-packages.proj
#	src/installer/prepare-artifacts.proj
#	src/libraries/System.Private.CoreLib/src/System.Private.CoreLib.Shared.projitems
#	src/libraries/System.Private.CoreLib/src/System/Globalization/CultureData.cs
#	src/libraries/System.Runtime.InteropServices.JavaScript/gen/JSImportGenerator/JSExportGenerator.cs
#	src/mono/browser/browser.proj
#	src/native/libs/CMakeLists.txt
#	src/tests/Common/Directory.Build.targets
#	src/tests/Directory.Build.targets
add back CORINFO_HELP_LLVM_STRESS_GC was CORINFO_HELP_STRESS_GC
RhpGcStressOnce now RhpStressGc
turn off FEATURE_STANDALONE_GC for CLR_CMAKE_TARGET_ARCH_WASM
stub membarrier
add CMAKE_CXX_STANDARD to jit compilation
PHASE_DFS_BLOCKS to PHASE_DFS_BLOCKS3
dont assert some register checks
remove NodeRef from gentree.h
Wide config strings
add CORINFO_HELP_GETDYNAMIC_NONGCTHREADSTATIC_BASE_NOCTOR_OPTIMIZED2_NOJITOPT to llvm.cpp
BlockSetOps to BitVecOps
fgFirstBBScratch to init block
fgAddEHTableEntry to fgTryAddEHTableEntries
allow retBufVarDsc->TypeGet() == TYP_I_IMPL  in initializeLlvmArgInfo
some merge mistakes
Some REG -> JITREG additions
add back GetPhiNodeForRationalIRForm
export _minipal_get_non_cryptographically_secure_random_bytes
build and link in libaotminipal as user compile time
Exclude som exception helpers for USE_PORTABLE_HELPERS
remove pExContext from PalCreateCrashDumpIfEnabled overload
remove get_Hybrid from link substitutions
…ge-feb25

# Conflicts:
#	src/coreclr/gc/unix/gcenv.unix.cpp
#	src/mono/browser/browser.proj
Reinstate ConvertStackTraceIpToFunctionPointer in ReflectionAugments.cs
Exclude Preinitialization tests for Wasm
Remove test that requires RhGetCommonStubAddress
Comment out printing of variables in LSSA

Wasi release tests now build and pass
… `extern C ` from main

For LIR use GetPhiNodeForRationalIRForm to detect the existence of a Phi
@yowl yowl marked this pull request as ready for review March 23, 2025 16:54
@yowl
Copy link
Contributor Author

yowl commented Mar 23, 2025

Thanks all on discord for the help!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.