Skip to content

[Java.Interop] JNI handles are now in a "control block" #1339

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Jun 12, 2025

Conversation

jonathanpeppers
Copy link
Member

Originally from: #1334
Context: dotnet/runtime#114184
Context: dotnet/android#10125
Context: dotnet/android#10125 (comment)

Part of adding a GC bridge to CoreCLR are the new APIs:

namespace System.Runtime.InteropServices.Java;
public struct ComponentCrossReference {
    public nint SourceGroupIndex, DestinationGroupIndex;
}
public unsafe struct StronglyConnectedComponent {
    public nint Count;
    public IntPtr* Context;
}
public static partial class JavaMarshal {
    public static unsafe void Initialize(
        delegate* unmanaged<
            System.IntPtr,                  // sccsLen
            StronglyConnectedComponent*,    // sccs
            System.IntPtr,                  // ccrsLen
            ComponentCrossReference*,       // ccrs
            void> markCrossReferences);
    public static GCHandle CreateReferenceTrackingHandle(object obj, IntPtr context);
    public static IntPtr GetContext(GCHandle obj);
}

Of note is the "data flow" of context:

  • JavaMarshal.CreateReferenceTrackingHandle() has a "context" parameter.

  • The context parameter to JavaMarshal.CreateReferenceTrackingHandle() is the return value of JavaMarshal.GetContext()

  • The context parameter to JavaMarshal.CreateReferenceTrackingHandle() is stored within StronglyConnectedComponent.Context.

  • The markCrossReferences parameter of JavaMarshal.Initialize() is called by the GC bridge and given a native array of StronglyConnectedComponent instances, which contains Context.

The short of it is that the proposed GC bridge doesn't contain direct access to the IJavaPeerable instances in play. Instead, it has access to "context" which must contain the JNI Object Reference information that the markCrossReferences callback needs access to.

Furthermore, the context pointer value cannot change, i.e. it needs to be a native pointer value a'la malloc(3), not a value which can be moved by the GC. (The contents can change; the pointer value cannot.))

While we're still prototyping this, what we currently believe we need is the JNI object reference, JNI object reference type, and (maybe?) the JNI Weak Global Reference value and "refs added" values.

Update IJavaPeerable to add a JniObjectReferenceControlBlock property which can be used as the context parameter:

partial interface IJavaPeerable {
    IntPtr JniObjectReferenceControlBlock => 0;
}

This supports usage of:

IJavaPeerable   value   = …
GCHandle        handle  = JavaMarshal.CreateReferenceTrackingHandle(
    value,
    value.JniObjectReferenceControlBlock
);

It works!

Build:

dotnet build -c Release -p:DotNetTargetFramework=net8.0 -t:Prepare
dotnet build -c Release -p:DotNetTargetFramework=net8.0
dotnet publish --self-contained -p:UseMonoRuntime=true -p:DotNetTargetFramework=net8.0 \
  -p:UseAppHost=true -p:ErrorOnDuplicatePublishOutputFiles=false \
  samples/Hello-Java.Base/Hello-Java.Base.csproj -r osx-x64

Run:

JAVA_INTEROP_GREF_LOG=g.txt  ./samples/Hello-Java.Base/bin/Release/osx-x64/publish/Hello-Java.Base

g.txt contains:

…
*take_weak obj=0x10560dc70; handle=0x7f7f42f15148
+w+ grefc 8 gwrefc 1 obj-handle 0x7f7f42f15148/G -> new-handle 0x7f7f43008401/W from thread '(null)'(1)
take_weak_global_ref_jni
-g- grefc 7 gwrefc 1 handle 0x7f7f42f15148/G from thread '(null)'(1)
take_weak_global_ref_jni
*try_take_global obj=0x10560dc70 -> wref=0x7f7f43008401 handle=0x0
-w- grefc 7 gwrefc 0 handle 0x7f7f43008401/W from thread '(null)'(1)
take_global_ref_jni
Finalizing PeerReference=0x0/G IdentityHashCode=0x70dea4e Instance=0x59b98e2b Instance.Type=Hello.MyJLO

Implement MonoVM BC Bridge :: Java Marshaling API "thunk"

Commit 9e9daf4 suggested that we could probably "thunk" the MonoVM API to implement the proposed Java Bridge API..

Implemented the thunk!

jonpryor and others added 2 commits June 9, 2025 08:37
Originally from: #1334
Context: dotnet/runtime#114184
Context: dotnet/android#10125
Context: dotnet/android#10125 (comment)

Part of adding a GC bridge to CoreCLR are the new APIs:

	namespace System.Runtime.InteropServices.Java;
	public struct ComponentCrossReference {
	    public nint SourceGroupIndex, DestinationGroupIndex;
	}
	public unsafe struct StronglyConnectedComponent {
	    public nint Count;
	    public IntPtr* Context;
	}
	public static partial class JavaMarshal {
	    public static unsafe void Initialize(
	        delegate* unmanaged<
	            System.IntPtr,                  // sccsLen
	            StronglyConnectedComponent*,    // sccs
	            System.IntPtr,                  // ccrsLen
	            ComponentCrossReference*,       // ccrs
	            void> markCrossReferences);
	    public static GCHandle CreateReferenceTrackingHandle(object obj, IntPtr context);
	    public static IntPtr GetContext(GCHandle obj);
	}

Of note is the "data flow" of `context`:

  * `JavaMarshal.CreateReferenceTrackingHandle()` has a "`context`"
    parameter.

  * The `context` parameter to
    `JavaMarshal.CreateReferenceTrackingHandle()` is the return value
    of `JavaMarshal.GetContext()`

  * The `context` parameter to
    `JavaMarshal.CreateReferenceTrackingHandle()` is stored within
    `StronglyConnectedComponent.Context`.

  * The `markCrossReferences` parameter of `JavaMarshal.Initialize()`
    is called by the GC bridge and given a native array of
    `StronglyConnectedComponent` instances, which contains `Context`.

The short of it is that the proposed GC bridge doesn't contain direct
access to the `IJavaPeerable` instances in play.  Instead, it has
access to "context" which must contain the JNI Object Reference
information that the `markCrossReferences` callback needs access to.

Furthermore, the `context` pointer value *cannot change*, i.e. it
needs to be a native pointer value a'la **malloc**(3), ***not*** a
value which can be moved by the GC.  (The *contents* can change; the
pointer value cannot.))

While we're still prototyping this, what we currently believe we need
is the JNI object reference, JNI object reference type, and (maybe?)
the JNI Weak Global Reference value and "refs added" values.

Update `IJavaPeerable` to add a `JniObjectReferenceControlBlock`
property which can be used as the `context` parameter:

	partial interface IJavaPeerable {
	    IntPtr JniObjectReferenceControlBlock => 0;
	}

This supports usage of:

	IJavaPeerable   value   = …
	GCHandle        handle  = JavaMarshal.CreateReferenceTrackingHandle(
	    value,
	    value.JniObjectReferenceControlBlock
	);

It works!

Build:

	dotnet build -c Release -p:DotNetTargetFramework=net8.0 -t:Prepare
	dotnet build -c Release -p:DotNetTargetFramework=net8.0
	dotnet publish --self-contained -p:UseMonoRuntime=true -p:DotNetTargetFramework=net8.0 \
	  -p:UseAppHost=true -p:ErrorOnDuplicatePublishOutputFiles=false \
	  samples/Hello-Java.Base/Hello-Java.Base.csproj -r osx-x64

Run:

	JAVA_INTEROP_GREF_LOG=g.txt  ./samples/Hello-Java.Base/bin/Release/osx-x64/publish/Hello-Java.Base

`g.txt` contains:

	…
	*take_weak obj=0x10560dc70; handle=0x7f7f42f15148
	+w+ grefc 8 gwrefc 1 obj-handle 0x7f7f42f15148/G -> new-handle 0x7f7f43008401/W from thread '(null)'(1)
	take_weak_global_ref_jni
	-g- grefc 7 gwrefc 1 handle 0x7f7f42f15148/G from thread '(null)'(1)
	take_weak_global_ref_jni
	*try_take_global obj=0x10560dc70 -> wref=0x7f7f43008401 handle=0x0
	-w- grefc 7 gwrefc 0 handle 0x7f7f43008401/W from thread '(null)'(1)
	take_global_ref_jni
	Finalizing PeerReference=0x0/G IdentityHashCode=0x70dea4e Instance=0x59b98e2b Instance.Type=Hello.MyJLO

Implement MonoVM BC Bridge :: Java Marshaling API "thunk"

Commit 9e9daf4 suggested that we could
probably "thunk" the MonoVM API to implement the proposed
Java Bridge API..

Implemented the thunk!
jonathanpeppers added a commit to dotnet/android that referenced this pull request Jun 9, 2025
Originally from: #10125
Context: dotnet/java-interop#1339

These are changes to Mono's GC bridge implementation, such that the
same code can be used on both Mono and CoreCLR.

The base `JavaObject` type has changes such as:

```diff
--[NonSerialized] IntPtr                  handle;
--[NonSerialized] JniObjectReferenceType  handle_type;
--[NonSerialized] IntPtr                  weak_handle;
--[NonSerialized] int                     refs_added;
++unsafe  JniObjectReferenceControlBlock* jniObjectReferenceControlBlock;
```

And the new `JniObjectReferenceControlBlock` struct is defined as:

    internal struct JniObjectReferenceControlBlock {
        public	IntPtr  handle;
        public  int     handle_type;
        public  IntPtr  weak_handle;
        public  int     refs_added;
    }

* Each `JavaObject` allocation now has a `NativeMemory.AllocZeroed()`
  call for this struct.

* However, we only have to read a single field now using Mono's native
  embedding API:

```diff
--info->handle            = mono_class_get_field_from_name (info->klass, const_cast<char*> ("handle"));
--info->handle_type       = mono_class_get_field_from_name (info->klass, const_cast<char*> ("handle_type"));
--info->refs_added        = mono_class_get_field_from_name (info->klass, const_cast<char*> ("refs_added"));
--info->weak_handle       = mono_class_get_field_from_name (info->klass, const_cast<char*> ("weak_handle"));
++info->jniObjectReferenceControlBlock    = mono_class_get_field_from_name (info->klass, const_cast<char*> ("jniObjectReferenceControlBlock"));
```

We are hoping this results in ~the same performance as before, with
the flexibility to support multiple GC bridge implementations and
runtimes.

Co-authored-by: Marek Habersack <[email protected]>
@jonathanpeppers
Copy link
Member Author

jonathanpeppers and others added 4 commits June 9, 2025 09:28
We think this could be a potential race condition
A "funny" thing happened after 628aa39: a teardown assertion fired!

	% dotnet test bin/TestRelease-net8.0/Java.Interop-Tests.dll
	…
	# jonp: LoadJvmLibrary(/Users/runner/hostedtoolcache/Java_Temurin-Hotspot_jdk/17.0.15-6/x64/Contents/Home/lib/libjli.dylib)=140707427245328
	# jonp: JNI_CreateJavaVM=7113469040; JNI_GetCreatedJavaVMs=7113469120
	# jonp: executing JNI_CreateJavaVM=1a7feec70
	# jonp: r=0 javavm=1a9dc1830 jnienv=7fe37d0422b0
	TearDown failed for test fixture Java.InteropTests.JavaObjectArray_object_ContractTest
	  JNI global references: grefStartCount=324; gref=340
	  Expected: True
	  But was:  False

	TearDown : NUnit.Framework.AssertionException :   JNI global references: grefStartCount=324; gref=340
	  Expected: True
	  But was:  False

	StackTrace:    at Java.InteropTests.JavaObjectArray_object_ContractTest.EndCheckGlobalRefCount() in /Users/runner/work/1/s/tests/Java.Interop-Tests/Java.Interop/JavaObjectArrayTest.cs:line 158

	--TearDown
	   at NUnit.Framework.Assert.ReportFailure(String message)
	   at NUnit.Framework.Assert.ReportFailure(ConstraintResult result, String message, Object[] args)
	   at NUnit.Framework.Assert.That[TActual](TActual actual, IResolveConstraint expression, String message, Object[] args)
	   at NUnit.Framework.Assert.IsTrue(Boolean condition, String message, Object[] args)
	   at Java.InteropTests.JavaObjectArray_object_ContractTest.EndCheckGlobalRefCount() in /Users/runner/work/1/s/tests/Java.Interop-Tests/Java.Interop/JavaObjectArrayTest.cs:line 158
	   at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
	   at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
	  Skipped References_CreatedReference_GlobalRef [3 ms]
	  Skipped References_CreatedReference_LocalRef [< 1 ms]
	  Skipped DoesTheJmethodNeedToMatchDeclaringType [5 ms]
	# jonp: LoadJvmLibrary(/Users/runner/hostedtoolcache/Java_Temurin-Hotspot_jdk/17.0.15-6/x64/Contents/Home/lib/libjli.dylib)=140707427245328
	# jonp: JNI_CreateJavaVM=7113469040; JNI_GetCreatedJavaVMs=7113469120
	# jonp: executing JNI_CreateJavaVM=1a7feec70
	# jonp: r=-5 javavm=0 jnienv=0

Unfortunately, teardown assertions "don't count", i.e. CI was green,
even though the above is copied from the CI run.

The cause of the problem?  Having `JavaObject.Dispose()`
call `JniObjectReferenceControlBlock.Free()`.

The underlying problem is that
`JniRuntime.JniValueManager.DisposePeer()` uses `JavaObject.PeerReference`
*after* calling `JavaObject.Disposed()` (which calls
`JavaObject.Dispose(bool)`), yet `JavaObject.Dispose(bool)` released
the `JniObjectReferenceControlBlock` that backs `PeerReference`,
meaning we "lost" (leaked!) the GREF!

There are two possible solutions:

 1. Update `JniRuntime.JniValueManager.DisposePeer()` to use
    `JavaObject.PeerReference` *before* calling `JavaObject.Disposed()`

 2. Update `JavaObject.Dispose(bool)` to not release the
    `JniObjectReferenceControlBlock` that backs `PeerReference`.

I avoided (1) because I didn't want to have to audit and update
dotnet/android and all other potential callsites.

Which brings us to (2): if not in `Dispose(bool)`, then where?

The last thing that `JniRuntime.JniValueManager.DisposePeer()` and
`JniRuntime.JniValueManager.FinalizePeer()` do is call
`JavaObject.SetPeerReference(default)`.  *This*, then, is where
cleanup needs to happen.

Update `JavaObject` and `JavaException` to use the
`JniManagedPeerStates` field to track "have I been disposed?",
and then update `SetPeerReference()` to call
`JniObjectReferenceControlBlock.Free()` if the state is "disposed".

This fixes the leak.
@jonathanpeppers jonathanpeppers force-pushed the dev/peppers/control-block branch from f30b6d3 to 384bbfd Compare June 11, 2025 20:37
@jonathanpeppers jonathanpeppers marked this pull request as ready for review June 12, 2025 13:38
@grendello grendello merged commit 40d27eb into main Jun 12, 2025
2 checks passed
@grendello grendello deleted the dev/peppers/control-block branch June 12, 2025 16:00
jonathanpeppers added a commit to dotnet/android that referenced this pull request Jun 12, 2025
Originally from: #10125
Context: dotnet/java-interop#1339

These are changes to Mono's GC bridge implementation, such that the
same code can be used on both Mono and CoreCLR.

The base `JavaObject` type has changes such as:

```diff
--[NonSerialized] IntPtr                  handle;
--[NonSerialized] JniObjectReferenceType  handle_type;
--[NonSerialized] IntPtr                  weak_handle;
--[NonSerialized] int                     refs_added;
++unsafe  JniObjectReferenceControlBlock* jniObjectReferenceControlBlock;
```

And the new `JniObjectReferenceControlBlock` struct is defined as:

    internal struct JniObjectReferenceControlBlock {
        public	IntPtr  handle;
        public  int     handle_type;
        public  IntPtr  weak_handle;
        public  int     refs_added;
    }

* Each `JavaObject` allocation now has a `NativeMemory.AllocZeroed()`
  call for this struct.

* However, we only have to read a single field now using Mono's native
  embedding API:

```diff
--info->handle            = mono_class_get_field_from_name (info->klass, const_cast<char*> ("handle"));
--info->handle_type       = mono_class_get_field_from_name (info->klass, const_cast<char*> ("handle_type"));
--info->refs_added        = mono_class_get_field_from_name (info->klass, const_cast<char*> ("refs_added"));
--info->weak_handle       = mono_class_get_field_from_name (info->klass, const_cast<char*> ("weak_handle"));
++info->jniObjectReferenceControlBlock    = mono_class_get_field_from_name (info->klass, const_cast<char*> ("jniObjectReferenceControlBlock"));
```

We are hoping this results in ~the same performance as before, with
the flexibility to support multiple GC bridge implementations and
runtimes.

Other changes:

* Fix SIGSEGV when using the new control block

We were using it incorrectly - there's no need to fetch/set values of managed object fields.
The correct thing to do is to read/write them on the control block itself.

Co-authored-by: Marek Habersack <[email protected]>
@github-actions github-actions bot locked and limited conversation to collaborators Jul 13, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants