[FLINK-40282][docs] Expand and correct the concepts Glossary by nicusX · Pull Request #28863 · apache/flink

nicusX · 2026-08-01T17:03:12Z

What is the purpose of the change

The current Flink Glossary doc page has gaps, some inconsistencies, and ambiguities which may be confusing for new Flink users. It also focus on DataStream API exclusively.
This PR expands and fixes the Glossary page.

Brief change log

New entries (10)

Checkpoint, Savepoint: referenced by other entries but never defined
Watermark, Parallelism, Channel, Source Split: absent
Flink SQL Statement, Statement Set, StreamExchange Operator: first Table API / SQL
coverage in the glossary
Job Graph: promoted from an alias to a full entry
Added 2 pointer entries: StreamGraphs --> Logical Graph, Execution Graph --> Physical Graph

Corrected definitions

Job Graph no longer documented as a synonym of Logical Graph
Task and Sub-Task no longer conflated
Physical Graph fixed definition
Partition split into Logical Partitioning and Physical Partitioning sub-entries; also added note about for external system such as Kafka topic partitions
State Backend reframed as the storage itself rather than a configuration
Checkpoint Storage adds the durable-storage framing and a pointer to disaggregated state
(Runtime) Execution Mode was a bare link to another page; now describes STREAMING and
BATCH modes
Flink Application: Java or Python, DataStream or Table API (was "a Java Application").
Fixed 3 broken cross-links that used capitalized anchors which Hugo doesn't generate (#Record and #Function).

Expanded entries

Flink Job, Logical Graph, Operator (adds Source/Sink Operators and a note that the docs use
"Operator" loosely for Task/Sub-Task), Operator Chain, Function (Python, UDF, PTF), Flink Cluster,
Flink JobManager, Flink JobMaster, Flink TaskManager, Sub-Task

Consistency

Applied one capitalization rule: a defined term is capitalized when it refers to a glossary entry
and lowercase in generic English use.
Restored alphabetical order, disregarding the Flink prefix (5 entries were misplaced). The page
sets bookToc: false, so ordering is the only way to locate a term.
Retitled for consistency with sibling entries: ExecutionGraph --> Execution Graph,
JobGraph --> Job Graph, UID hash --> UID Hash
Anchors unchanged except #jobgraph --> #job-graph, which is not referenced outside of this page
Added "Also " lines recording the spellings readers meet in code, logs and the Web UI
Removed trailing whitespace.

Verifying this change

Docs changes only

Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): No
The public API, i.e., is any changed class annotated with @Public(Evolving): No
The serializers: No
The runtime per-record code paths (performance sensitive): No
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: No
The S3 file system connector: No

Documentation

Does this pull request introduce a new feature? No
If yes, how is the feature documented? not applicable

Was generative AI tooling used to co-author this PR?

Yes (please specify the tool below)

flinkbot · 2026-08-01T17:09:56Z

CI report:

37698da Azure: SUCCESS

Bot commands

The @flinkbot bot supports the following commands:

@flinkbot run azure re-run the last Azure build

alpinegizmo

There are a few points where I either disagree, or think some additional clarification is needed.

alpinegizmo · 2026-08-03T17:22:25Z

+Checkpoints are triggered automatically and periodically while the Job is running, and are used to
+recover from failures such as a TaskManager crash or a network problem: the Job restarts from the
+latest completed Checkpoint. They are designed for low overhead and run mostly asynchronously,
+without blocking record processing, apart from the short pause each Sub-Task takes to snapshot its


This characterization (the short pause ... to snapshot its state) is somewhat misleading, I think.

The synchronous phase of checkpointing a sub-task (during which record processing is blocked) takes care of some quick bookkeeping, plus snapshotting all non-keyed (operator) state and all on-heap timers, regardless of the state backend. Then the asynchronous phase takes care of snapshotting off-heap timers, keyed state, and in-flight buffers (for unaligned checkpoints). So the pause might not be short, but is less than the time required to snapshot all of a sub-task's state.

alpinegizmo · 2026-08-03T17:26:57Z

+with a variant of the Chandy-Lamport algorithm and written to [Checkpoint
+Storage](#checkpoint-storage).
+
+A Checkpoint contains the [State](#managed-state) of all stateful [Operators](#operator), but also


This makes it sound like kafka offsets, sink transaction metadata, etc aren't managed by Flink, but they are. They aren't keyed state, and they don't live in the state backend, but they are managed as operator state.

alpinegizmo · 2026-08-03T17:31:18Z

+
+#### Checkpoint Storage
+
+The durable location where the [State Backend](#state-backend) writes the snapshot it takes during a


Checkpoints include more than just the state managed by the state backend. There's also the so-called checkpoint metadata, which includes non-keyed state being managed by Flink.

alpinegizmo · 2026-08-03T17:44:34Z


-Managed State describes application state which has been registered with the framework. For
+Managed State describes Application State which has been registered with the framework. For
 Managed State, Apache Flink will take care about persistence and rescaling among other things.


Managed state includes both keyed state and non-keyed state (also known as operator state).

alpinegizmo · 2026-08-03T17:47:43Z

+
+*Logical Partitioning* is how records and State are divided in the [Logical
+Graph](#logical-graph) and the [Job Graph](#job-graph), in order to implement the semantics of an
+operation. A `GROUP BY` in SQL or a `keyBy()` in the DataStream API, for example, requires the data to


Perhaps expand to say

A JOIN or GROUP BY in SQL, or a keyBy in ...

alpinegizmo · 2026-08-03T17:53:53Z

+When a Job is *stopped with a Savepoint*, every [Sub-Task](#sub-task) stops right after its State has
+been snapshotted, which guarantees that no record is reprocessed when the Job is resumed.


"which guarantees that no record is reprocessed when the Job is resumed" -- nope, there is no such guarantee (unless something has recently changed that I'm unaware of).

alpinegizmo · 2026-08-03T17:54:33Z

+When a Job is *stopped with a Savepoint*, every [Sub-Task](#sub-task) stops right after its State has
+been snapshotted, which guarantees that no record is reprocessed when the Job is resumed.
+
+See [Checkpoints vs. Savepoints]({{< ref "docs/ops/state/checkpoints_vs_savepoints" >}}) for a


Savepoints are similar to checkpoints. See ...

alpinegizmo · 2026-08-03T17:57:40Z

-[state](#managed-state) is stored on each TaskManager (Java Heap of TaskManager or (embedded)
-RocksDB).
+For stream processing programs, the State Backend of a [Flink Job](#flink-job) holds the
+[State](#managed-state) that the Job is actively working with, local to each


the state backend only holds the keyed state, not all of the managed state

alpinegizmo · 2026-08-03T18:18:44Z

 "Vertex ID" and generated from a [UID](#uid).
 It is commonly exposed in logs, the REST API or metrics, and most importantly is how
-operators are identified within [savepoints]({{< ref "docs/ops/state/savepoints" >}}).
+Operators are identified within [Savepoints]({{< ref "docs/ops/state/savepoints" >}}).


Not just savepoints, but also checkpoints. I suggest introducing the term "snapshots" for situations like this.

alpinegizmo · 2026-08-03T18:19:46Z

+
+Watermarks are emitted at the Sources, based on a `WatermarkStrategy`. Each Source [Sub-Task](#sub-task) generates its own Watermarks independently. Event time advances independently in each [Physical Partition](#partition). When a Watermark reaches a Sub-Task, the Sub-Task advances its internal event-time clock and emits a new Watermark to its downstream Sub-Tasks.
+A Sub-Task with several input Channels takes the *minimum* of the event times of its
+inputs, which means a single lagging input holds back event time for the whole downstream graph.


... of its active inputs ...

[FLINK-40282][docs] Expand and correct the concepts Glossary

37698da

alpinegizmo requested changes Aug 3, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[FLINK-40282][docs] Expand and correct the concepts Glossary - #28863

[FLINK-40282][docs] Expand and correct the concepts Glossary#28863
nicusX wants to merge 1 commit into
apache:masterfrom
nicusX:FLINK-40282-docs-expand_glossary

nicusX commented Aug 1, 2026

Uh oh!

flinkbot commented Aug 1, 2026 •

edited

Loading

Uh oh!

alpinegizmo left a comment

Uh oh!

alpinegizmo Aug 3, 2026

Uh oh!

alpinegizmo Aug 3, 2026

Uh oh!

alpinegizmo Aug 3, 2026

Uh oh!

alpinegizmo Aug 3, 2026

Uh oh!

alpinegizmo Aug 3, 2026

Uh oh!

alpinegizmo Aug 3, 2026

Uh oh!

alpinegizmo Aug 3, 2026

Uh oh!

alpinegizmo Aug 3, 2026

Uh oh!

alpinegizmo Aug 3, 2026

Uh oh!

alpinegizmo Aug 3, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants


		#### Checkpoint Storage

		The durable location where the [State Backend](#state-backend) writes the snapshot it takes during a

		When a Job is stopped with a Savepoint, every [Sub-Task](#sub-task) stops right after its State has
		been snapshotted, which guarantees that no record is reprocessed when the Job is resumed.

Uh oh!

Conversation

nicusX commented Aug 1, 2026

What is the purpose of the change

Brief change log

New entries (10)

Corrected definitions

Expanded entries

Consistency

Verifying this change

Does this pull request potentially affect one of the following parts:

Documentation

Was generative AI tooling used to co-author this PR?

Uh oh!

flinkbot commented Aug 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

CI report:

Uh oh!

alpinegizmo left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

flinkbot commented Aug 1, 2026 •

edited

Loading