Memory Leak in GwtMockitoTestRunner$GwtMockitoClassLoader

[tcdavid]

When running a large suite of JUnit test classes using the @GwtMockitoTestRunner, we consistently encounter a "java/lang/OutOfMemoryError".

We used the Eclipse Memory Analyzer to inspect the resulting heap dump file.
The analysis identified the problem suspect as:
273 instances of "com.google.gwtmockito.GwtMockitoTestRunner$GwtMockitoClassLoader", loaded by "com.ibm.oti.vm.BootstrapClassLoader @ 0x8004b120" occupy 1,519,903,320 (94.34%) bytes.

We are running the following versions of the related libraries:
JUnit - junit:junit:4.12
Mockito - org.mockito:mockito-all:1.10.19
GWTMockito - com.google.gwt.gwtmockito:gwtmockito:1.1.5

To run the tests, we are simply annotating our test classes with
@RunWith(GwtMockitoTestRunner.class)

Is there something we need to do in a tearDown() method to make sure the GwtMockitoTestRunner releases the gwtMockitoClassLoader it created?

[uplight-dev]

Hi tcdavid,

Is there anything you or your team found during this time to fix this issue? Or +1 for an answer from the original developer of this library.

Thank you!

[ekuefler]

Hm this might be tricky to track down. Ideally GwtMockitoClassLoader shouldn't last longer than the test class, so if there're over 200 of them in your heap it's definitely either leaking somewhere or you're running a bunch of tests in parallel. Does the profiler you're using provide information on which classes are referencing GwtMockitoClassLoader? I think there are three possibilities:

• It's being referenced from GwtMockitoTestRunner, which is the class that should own it. This would imply that JUnit is keeping references to old runners around for some reason, which would be a bit surprising but probably not too hard to work around.
• It's being referenced from Thread. We pass the classloader as the context class loader for the thread, which will maintain a reference to it. We're supposed to clean up and switch back to the original context class loader when the test finishes though. So if this is the problem it implies both that we're not cleaning up properly and whatever you're using to run the tests is keeping threads around that it doesn't need.
• It's being referenced by some class loaded by the classloader. Every class maintains a reference to its classloader, so if you're somehow saving classes generated during each test (say, by putting them into a static list somewhere), that would cause the classloaders to be retained. It's also possible that GwtMockito is leaking something out to JUnit internally that would cause the reference to be held. This seems like the most likely culprit.

If you can tell which classes are holding on to the reference and whether they're in your code vs. GwtMockito/JUnit's that would go a long way towards tracking down the problem.

[schragmanuel]

We have the same Problem in our project. Memory usage is 5GB with 3500 tests. Number of tests is going to grow, so we need to solve the problem somehow. I also see 266 instances of GwtMockitoTestRunner in the heap dump. This is exactly the number of tests I have with the @RunWith Annotation. Would it help if I provide you the dump @ekuefler ? Because I don't see how to get closer to the source of the problem.

Thank you!

[Numerinico]

Hi there,

Same problem here, huge memory consumption, any workaround ?

Cheers,

[ekuefler]

If anyone has collected a heap dump that would definitely be useful for tracking down the problem.

[schragmanuel]

I collected two heap dumps with visual vm. Files are available on wetransfer The files are getting deleted within 7 days, so please download them as soon as possible.

[ekuefler]

Sweet, thanks! I downloaded the dumps and will be able to analyze them closer before too long.

[Numerinico]

Hi there, any news ? :)

[cinlloc]

@ekuefler it seems it's a bit of both your first and third point:

• It's being referenced from GwtMockitoTestRunner
• It's being referenced by some class loaded by the classloader

It seems the first problem is really a JUnit4 one, because I can reproduce it with BlockJUnit4Runner (without memory leak because no additional reference to a classloader in the object). As you said, workaround is possible (gwtMockitoClassLoader = null in finally clause for example).
The big matter is the second one. There is a bunch of objects that references the class loader, without saving any object like you said, so it should definitely be a leak of some kind.

[cinlloc]

For those interested I have a workaround, if you use Maven: use the maven surefire plugin, that allows to execute every test class in a different Java process.
Have to configure the plugin with these settings in your pom.xml:

<build>
  <plugins>
    <plugin>
      <groupId>org.apache.maven.plugins</groupId>
      <artifactId>maven-surefire-plugin</artifactId>
      <configuration>
        <forkCount>1</forkCount>
        <reuseForks>false</reuseForks>
      </configuration>
    </plugin>
  </plugins>
</build>

[LudoPL]

Hi,
The issue is linked to the use of "ThreadLocal" by Mockito (mentioned here and here).
When Mockito is used inside a unit test executed by the GwtMockitoTestRunner, instances of DefaultMockitoConfiguration and MockingProgressImpl objects are stored (as value) inside the ThreadLocalMap of the Thread executing the test.
These objects are instantiate through the GwtMockitoClassLoader created for the test, so they prevent the garbage collection of this GwtMockitoClassLoader.
The ThreadLocal instances used as keys in the ThreadLocalMap are referenced as static attributes of a class loaded by the GwtMockitoClassLoader, so even if WeakReference are used in the ThreadLocalMap, they won't be garbage collected too (before the GwtMockitoClassLoader is).
As a result, for each test the created GwtMockitoClassLoader stay in memory (which also prevent the garbage collection of the GwtMockitoTestRunner as the GwtMockitoClassLoader is an internal class of it).
I will try to propose a fix very soon, not sure it will be the proper way to do it but I have already done a POC that is working.

[ekuefler]

I've pushed a new snapshot version that contains this change: https://oss.sonatype.org/content/repositories/snapshots/com/google/gwt/gwtmockito/gwtmockito/1.1.9-SNAPSHOT/

Has anyone had a chance to try the new annotation yet and verify whether it's working as expected?