Skip to content

Commit c326603

Browse files
MemoryAllocator
1 parent 1f89b02 commit c326603

File tree

3 files changed

+59
-3
lines changed

3 files changed

+59
-3
lines changed

docs/memory/MemoryAllocator.md

Lines changed: 34 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,36 @@
11
# MemoryAllocator
22

3-
`MemoryAllocator` is...FIXME
3+
`MemoryAllocator` is an [abstraction](#contract) of [memory allocators](#implementations) that [TaskMemoryManager](TaskMemoryManager.md) uses to [allocate](#allocate) and [release](#free) memory.
4+
5+
`MemoryAllocator` creates the available [MemoryAllocator](#implementations)s to be available under the names [HEAP](#HEAP) and [UNSAFE](#UNSAFE).
6+
7+
A [MemoryAllocator](#implementations) to use is selected when `MemoryManager` is [created](MemoryManager.md#tungstenMemoryAllocator) (based on [MemoryMode](MemoryManager.md#tungstenMemoryMode)).
8+
9+
## Contract
10+
11+
### <span id="allocate"> Allocating Contiguous Block of Memory
12+
13+
```java
14+
MemoryBlock allocate(
15+
long size)
16+
```
17+
18+
Used when:
19+
20+
* `TaskMemoryManager` is requested to [allocate a memory page](TaskMemoryManager.md#allocatePage)
21+
22+
### <span id="free"> Releasing Memory
23+
24+
```java
25+
void free(
26+
MemoryBlock memory)
27+
```
28+
29+
Used when:
30+
31+
* `TaskMemoryManager` is requested to [release a memory page](TaskMemoryManager.md#freePage) and [clean up all the allocated memory](TaskMemoryManager.md#cleanUpAllAllocatedMemory)
32+
33+
## Implementations
34+
35+
* <span id="HEAP"> HeapMemoryAllocator
36+
* <span id="UNSAFE"> UnsafeMemoryAllocator

docs/memory/MemoryManager.md

Lines changed: 24 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -183,6 +183,9 @@ tungstenMemoryMode: MemoryMode
183183

184184
`tungstenMemoryMode` tracks whether Tungsten memory will be allocated on the JVM heap or off-heap (using `sun.misc.Unsafe`).
185185

186+
!!! note "final val"
187+
`tungstenMemoryMode` is a `final val`ue so initialized once when `MemoryManager` is [created](#creating-instance).
188+
186189
`tungstenMemoryMode` is `OFF_HEAP` when the following are all met:
187190

188191
* [spark.memory.offHeap.enabled](../configuration-properties.md#spark.memory.offHeap.enabled) configuration property is enabled
@@ -198,5 +201,25 @@ Otherwise, `tungstenMemoryMode` is `ON_HEAP`.
198201

199202
`tungstenMemoryMode` is used when:
200203

201-
* `MemoryManager` is created (and initializes the [pageSizeBytes](MemoryManager.md#pageSizeBytes) and [tungstenMemoryAllocator](MemoryManager.md#tungstenMemoryAllocator) internal properties)
204+
* `MemoryManager` is [created](#creating-instance) (and initializes the [pageSizeBytes](#pageSizeBytes) and [tungstenMemoryAllocator](#tungstenMemoryAllocator) internal properties)
202205
* `TaskMemoryManager` is [created](TaskMemoryManager.md#tungstenMemoryMode)
206+
207+
## <span id="tungstenMemoryAllocator"> MemoryAllocator
208+
209+
```scala
210+
tungstenMemoryAllocator: MemoryAllocator
211+
```
212+
213+
`MemoryManager` selects the [MemoryAllocator](MemoryAllocator.md) to use based on the [MemoryMode](#tungstenMemoryMode).
214+
215+
!!! note "final val"
216+
`tungstenMemoryAllocator` is a `final val`ue so initialized once when `MemoryManager` is [created](#creating-instance).
217+
218+
MemoryMode | MemoryAllocator
219+
------------|----------------
220+
`ON_HEAP` | [HeapMemoryAllocator](MemoryAllocator.md#HEAP)
221+
`OFF_HEAP` | [UnsafeMemoryAllocator](MemoryAllocator.md#UNSAFE)
222+
223+
`tungstenMemoryAllocator` is used when:
224+
225+
* `TaskMemoryManager` is requested to [allocate a memory page](TaskMemoryManager.md#allocatePage), [release a memory page](TaskMemoryManager.md#freePage) and [clean up all the allocated memory](TaskMemoryManager.md#cleanUpAllAllocatedMemory)

docs/memory/TaskMemoryManager.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -311,7 +311,7 @@ Failed to allocate a page ([acquired] bytes), try again.
311311

312312
In the end, `allocatePage` tries to [allocate the page](#allocatePage) again (recursively).
313313

314-
## <span id="freePage"> Freeing Memory Page
314+
## <span id="freePage"> Releasing Memory Page
315315

316316
```java
317317
void freePage(

0 commit comments

Comments
 (0)