8319876: Reduce memory consumption of VM_ThreadDump::doit

[elifaslan1]

This pull request contains a backport of commit 8ec6b8de3bb3d7aeebdcb45d761b18cce3bab75e from the openjdk/jdk repository to JDK 17.

This backport significantly reduces RSS memory usage during ThreadMXBean.dumpAllThreads(boolean, boolean) execution.

This is not a clean backport due to differences between JDK 17 and upstream:

Threading Context: Changed ResourceMark rm(VMThread::vm_thread()) to ResourceMark rm;
• Similar to JDK 21 backport

Minor conflict resolved: _threads = nullptr; → _threads = NULL; to match JDK 17 conventions

GHA tested additionally ran tier1 tests on my local mac and passed.

---

Progress

• Change must be properly reviewed (1 review required, with at least 1 Reviewer)
• Change must not contain extraneous whitespace
• Commit message must refer to an issue
• JDK-8319876 needs maintainer approval

Issue

• JDK-8319876: Reduce memory consumption of VM_ThreadDump::doit (Enhancement - P4)

Reviewers

• Paul Hohensee (@phohensee - Reviewer)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk17u-dev.git pull/4006/head:pull/4006
$ git checkout pull/4006

Update a local copy of the PR:
$ git checkout pull/4006
$ git pull https://git.openjdk.org/jdk17u-dev.git pull/4006/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 4006

View PR using the GUI difftool:
$ git pr show -t 4006

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk17u-dev/pull/4006.diff

Using Webrev

Link to Webrev Comment

[bridgekeeper]

👋 Welcome back elifaslan1! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

[openjdk]

❗ This change is not yet ready to be integrated.
See the Progress checklist in the description for automated requirements.

[openjdk]

This backport pull request has now been updated with issue from the original commit.

[mlbridge]

Webrevs

• 00: Full (adef3e3e)

[phohensee]

Lgtm. Looks low risk, but I'd run tier2 on a bigger box to be on the safe side.

[openjdk]

⚠️ @elifaslan1 This change is now ready for you to apply for maintainer approval. This can be done directly in each associated issue or by using the /approval command.