diff --git a/AGENTS.md b/AGENTS.md index 71f1ea3..8de19d7 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -106,8 +106,11 @@ backend → URL string or LibreChat artifact dict. Details: **Word** - One line-break model: every newline reaching the inline layer is a soft - break; two-space runs are assembled by `_soft_break_run()` in the block - dispatcher and stop before any block. Never give `expand_br_to_block_breaks()` + break; a line ending in two spaces or a `
` (`ends_with_soft_break()`) + continues into the next one — runs are assembled by `_soft_break_run()` in + the block dispatcher and stop at a blank line and before any block. Turn + `
` into newlines with `br_to_newlines()`, which reads `
` + newline + as one break, never with a bare `_BR_RE.sub()`. Never give `expand_br_to_block_breaks()` the renderer's numbering state (#110). Details in [word.md](docs/development/tools/word.md#one-line-break-model). diff --git a/docs/development/tools/word.md b/docs/development/tools/word.md index 524bfbb..86ef81e 100644 --- a/docs/development/tools/word.md +++ b/docs/development/tools/word.md @@ -61,12 +61,13 @@ markdown_content ├─ → collected, then applied to the next block ├─ other → skipped └─ anything else → _soft_break_run() → paragraph + inline parse - (a line ending in two spaces continues the paragraph, - stopping before any line that starts a block) + (a line ending in two spaces or a
continues the + paragraph, stopping before any line that starts a block) ▼ inline_formatting.parse_inline_formatting(text, paragraph) every text span ├─ decode a fixed list of safe HTML entities - ├─
, CR, literal "\n" → "\n"; split, rstrip, one per break + ├─
, CR, literal "\n" → "\n" (a
+ newline pair is one); split, + │ rstrip, one per break ├─ backslash escapes → private-use placeholders, restored after tokenising └─ inline_markdown.build_inline_pattern().split() → runs, hyperlinks ▼ @@ -83,7 +84,7 @@ line by line and paragraphs are appended as they are recognised. |--------|------| | `base_docx_tool.py` | Template load, metadata, TOC, header/footer, then hands off to the processor. The three entry points | | `markdown_processor.py` | Line walker (`process_markdown_content`), block dispatcher (`process_markdown_block`), soft breaks, code blocks, alignment blocks, comment directives, the running ordered-list count | -| `patterns.py` | Every compiled regex for block detection, plus `ordered_list_is_genuine()`, `normalize_newlines()`, `expand_br_to_block_breaks()`, `line_starts_block()`, `contains_block_markdown()` | +| `patterns.py` | Every compiled regex for block detection, plus `ordered_list_is_genuine()`, `normalize_newlines()`, `ends_with_soft_break()`, `br_to_newlines()`, `expand_br_to_block_breaks()`, `line_starts_block()`, `contains_block_markdown()` | | `inline_formatting.py` | `parse_inline_formatting()`: entities, escapes, soft breaks, run creation, hyperlinks | | `block_elements.py` | Tables, lists, images, horizontal lines, alignment detection | | `numbering.py` | Ordered-list restart through fresh `` instances; style-aware numbering resolution; indent re-assertion | @@ -253,11 +254,25 @@ Since #110 there is one rule for every entry point: the block layer decides which newlines separate blocks, and **every newline that reaches the inline layer is a soft break** (``). `parse_inline_formatting()` folds `
`, a CR and a literal `\n` into `\n`, splits on it and strips the -trailing-space marker from each segment. A line ending in two spaces is the -source spelling of the same break: `_soft_break_run()` in the block +trailing-space marker from each segment. A line ending in two spaces **or in +a `
`** is the source spelling of the same break +(`patterns.ends_with_soft_break()`): `_soft_break_run()` in the block dispatcher joins such lines with real newlines, for prose and for quotes, -and stops before any line for which `patterns.line_starts_block()` is true -so a list or table on the next line is never swallowed into the paragraph. +and stops at a blank line and before any line for which +`patterns.line_starts_block()` is true so a list or table on the next line is +never swallowed into the paragraph. `process_list_items()` asks the same +question of a list item and pulls a following line into the item unless it is +blank, a list marker or another block (`_continues_item()`). A heading is +always one line. + +A `
` that ends a line is left in the text the block layer joins, so the +inline layer sees `
` followed by `\n` — the same break spelled twice. +`patterns.br_to_newlines()`, which the inline layer and +`set_header_footer()` both use, turns that pair into one `\n` and every +other `
` into its own, so `

` plus Enter is still an empty line. +Before this a line ending in `
` became its own paragraph, with the +style's paragraph spacing after it, and the `
` vanished as a trailing +break — the usual way a model writes a signature block. `line_starts_block()` is also the predicate behind `contains_block_markdown()`, which routes template placeholder values. @@ -416,6 +431,7 @@ it on open. | `tests/test_docx_style_map.py`, `test_docx_style_numbering.py`, `test_docx_style_tag.py` | Style mapping, style-aware numbering, the `style` directive | | `tests/test_docx_escaped_newlines.py` | Literal `\n` and backslash escapes | | `tests/test_docx_soft_breaks.py` | The line-break model: `
`, trailing spaces, CR, runs stopping before blocks, quotes, cells, headers | +| `tests/test_docx_br_before_newline.py` | A `
` ending a line is one soft break: paragraphs, quotes, list items, placeholders, headers; a blank line after it is a new paragraph | | `tests/test_docx_templates.py`, `test_docx_placeholder_formatting.py`, `test_docx_conditionals.py` | Dynamic templates: placeholder replacement across runs, formatting preservation, conditionals | | `tests/test_docx_content_controls.py`, `test_docx_content_controls_phase2.py`, `test_docx_template_content_controls.py` | Dynamic templates: content controls filled by Tag | | `tests/test_docx_warnings.py` | The warnings channel: every code, the source line, de-duplication, the tool-boundary response shape | diff --git a/docs/markdown-reference.md b/docs/markdown-reference.md index 63000d5..786d846 100644 --- a/docs/markdown-reference.md +++ b/docs/markdown-reference.md @@ -76,11 +76,14 @@ Nesting and combinations work, e.g. `**bold with *italic* inside**`, `**~~bold s | blank line | New paragraph / next block | | `
` | **Soft line break** — a new line inside the *same* paragraph (`Street
City`) | | two trailing spaces | The same soft break. Invisible, and many editors strip them — prefer `
` | +| `
` at the end of a line | Still **one** soft break: the next line continues the same paragraph or list item (`Street
` ⏎ `City`). Follow the `
` with a blank line and the next line is a new paragraph | | a literal `\n` typed as text | Read as a real newline, but never write it deliberately | +A heading is always one line: a `
` or two spaces at its end do not pull the next line into it. + A soft break never swallows what follows it: a list, table, heading, quote or page break on the next line is still that block. A `
` written *before* a list (`Intro:
1. A
2. B`) is promoted to a real line break so the list renders as one — which is why a line-leading number that is really text (`1. máje 5`) needs the documented `1\.` escape. -> ⚠️ **One thing `
` cannot do:** resume a numbered list that started earlier in the document. `Poznámka
3. Třetí`, after a list that reached `2.`, stays one paragraph with "3." as text. Write that continuation on its own line (a blank line, or a line ending in two spaces) and it becomes item 3 — [list continuation](#word) across intervening content is unaffected, it just needs a real line break rather than a `
`. +> ⚠️ **One thing `
` cannot do:** resume a numbered list that started earlier in the document. `Poznámka
3. Třetí`, after a list that reached `2.`, stays one paragraph with "3." as text. Write that continuation on its own line (after a blank line, a line ending in two spaces, or a `
` followed by Enter) and it becomes item 3 — [list continuation](#word) across intervening content is unaffected, it just needs a real line break rather than a `
`. `header_text` / `footer_text` take `
` too; they are plain text, not markdown. diff --git a/docx_tools/block_elements.py b/docx_tools/block_elements.py index b021c33..a81be36 100644 --- a/docx_tools/block_elements.py +++ b/docx_tools/block_elements.py @@ -12,7 +12,10 @@ UNORDERED_LIST_CAPTURE_PATTERN, _ALIGN_INLINE_RE, _ALIGN_OPEN_RE, + _ALIGN_CLOSE_RE, _BR_PARAGRAPH_RE, + ends_with_soft_break, + line_starts_block, ) from .inline_formatting import parse_inline_formatting from .numbering import ( @@ -227,6 +230,26 @@ def add_table_to_doc(table_data, doc, col_alignments=None, borderless=False, # --------------------------------------------------------------------------- # Lists # --------------------------------------------------------------------------- +def _continues_item(lines, idx) -> bool: + """True if ``lines[idx]`` may continue the list item above it. + + The item above ended in a soft-break marker; the next line joins it unless it + is blank, or starts something of its own — any list marker (a sibling or + nested item; in a list a number is never read as a date), any other block + (:func:`~docx_tools.patterns.line_starts_block`), a comment or directive, or + the close of an alignment block. + """ + stripped = lines[idx].strip() + return bool( + stripped + and not ORDERED_LIST_PATTERN.match(stripped) + and not UNORDERED_LIST_PATTERN.match(stripped) + and not stripped.startswith('