[SPARK-53022][TESTS] Add MemoryConsumerBenchmark

[zhztheplayer]

What changes were proposed in this pull request?

Adds a MemoryConsumerBenchmark and several cases for the basic allocation / free APIs.

Why are the changes needed?

We once noticed that massive off-heap memory allocation calls to Spark’s memory APIs could slow down executions in our 3rd plugin. Adding a benchmark could help measuring the performance of the memory allocation API, and possibly help with improving performance of the memory code in the future.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

This is a patch to add a benchmark.

Was this patch authored or co-authored using generative AI tooling?

No.

[zhztheplayer]

The current benchmark result:

OpenJDK 64-Bit Server VM 21.0.7+6-b1038.54 on Linux 6.14.0-24-generic
Unknown processor
On-heap allocations:                      Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
use and free - TestMemoryManager                      0              0           0     243902.4           0.0       1.0X
use and free - UnifiedMemoryManager                   0              0           0      34364.3           0.0       0.1X

OpenJDK 64-Bit Server VM 21.0.7+6-b1038.54 on Linux 6.14.0-24-generic
Unknown processor
Off-heap allocations:                     Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
use and free - TestMemoryManager                      0              0           0     238095.2           0.0       1.0X
use and free - UnifiedMemoryManager                   0              0           0      40000.0           0.0       0.2X