More HTML parsing test failures in Python 3.14 beta 4

[waylan]

Note that 2 changes to the HTML Parser are included in Python 3.14 beta 4. (GH-135930 & GH-135664). Since beta 4 was released, an additional change has been made (GH-136908). I have not tested against the unreleased change.

........................................../home/runner/work/markdown/markdown/markdown/core.py:436: DeprecationWarning: codecs.open() is deprecated. Use open() instead.
  output_file = codecs.open(output, "w",
......................................................................................s......................................................................................................s.s...s.ss..s....s.........s...........................................................................ss...........................................................................................................................................................FF..............................ss...........................................................................................................................................................................................................................................................FF......................................................................................................................................................................................................................................................................................
======================================================================
FAIL: test_raw_unclosed_tag_in_code_span (test_syntax.blocks.test_html_blocks.TestHTMLBlocks.test_raw_unclosed_tag_in_code_span)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/runner/work/markdown/markdown/tests/test_syntax/blocks/test_html_blocks.py", line 668, in test_raw_unclosed_tag_in_code_span
    self.assertMarkdownRenders(
    ~~~~~~~~~~~~~~~~~~~~~~~~~~^
        self.dedent(
        ^^^^^^^^^^^^
    ...<15 lines>...
        )
        ^
    )
    ^
  File "/home/runner/work/markdown/markdown/markdown/test_tools.py", line 77, in assertMarkdownRenders
    self.assertMultiLineEqual(output, expected)
    ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
AssertionError: '<p><code>&lt;div</code>.</p>\n<p><div>\nhello\n</div></p>' != '<p><code>&lt;div</code>.</p>\n<div>\nhello\n</div>'
  <p><code>&lt;div</code>.</p>
- <p><div>
? ---
+ <div>
  hello
- </div></p>
?       ----
+ </div>


======================================================================
FAIL: test_raw_unclosed_tag_in_code_span_space (test_syntax.blocks.test_html_blocks.TestHTMLBlocks.test_raw_unclosed_tag_in_code_span_space)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/runner/work/markdown/markdown/tests/test_syntax/blocks/test_html_blocks.py", line 689, in test_raw_unclosed_tag_in_code_span_space
    self.assertMarkdownRenders(
    ~~~~~~~~~~~~~~~~~~~~~~~~~~^
        self.dedent(
        ^^^^^^^^^^^^
    ...<15 lines>...
        )
        ^
    )
    ^
  File "/home/runner/work/markdown/markdown/markdown/test_tools.py", line 77, in assertMarkdownRenders
    self.assertMultiLineEqual(output, expected)
    ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
AssertionError: '<p><code>&lt;div</code>.</p>\n<p><div>\nhello\n</div></p>' != '<p><code>&lt;div</code>.</p>\n<div>\nhello\n</div>'
  <p><code>&lt;div</code>.</p>
- <p><div>
? ---
+ <div>
  hello
- </div></p>
?       ----
+ </div>


======================================================================
FAIL: test_raw_unclosed_tag_in_code_span (test_syntax.extensions.test_md_in_html.TestDefaultwMdInHTML.test_raw_unclosed_tag_in_code_span)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/runner/work/markdown/markdown/tests/test_syntax/blocks/test_html_blocks.py", line 668, in test_raw_unclosed_tag_in_code_span
    self.assertMarkdownRenders(
    ~~~~~~~~~~~~~~~~~~~~~~~~~~^
        self.dedent(
        ^^^^^^^^^^^^
    ...<15 lines>...
        )
        ^
    )
    ^
  File "/home/runner/work/markdown/markdown/markdown/test_tools.py", line 77, in assertMarkdownRenders
    self.assertMultiLineEqual(output, expected)
    ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
AssertionError: '<p><code>&lt;div</code>.</p>\n<p><div>\nhello\n</div></p>' != '<p><code>&lt;div</code>.</p>\n<div>\nhello\n</div>'
  <p><code>&lt;div</code>.</p>
- <p><div>
? ---
+ <div>
  hello
- </div></p>
?       ----
+ </div>


======================================================================
FAIL: test_raw_unclosed_tag_in_code_span_space (test_syntax.extensions.test_md_in_html.TestDefaultwMdInHTML.test_raw_unclosed_tag_in_code_span_space)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/runner/work/markdown/markdown/tests/test_syntax/blocks/test_html_blocks.py", line 689, in test_raw_unclosed_tag_in_code_span_space
    self.assertMarkdownRenders(
    ~~~~~~~~~~~~~~~~~~~~~~~~~~^
        self.dedent(
        ^^^^^^^^^^^^
    ...<15 lines>...
        )
        ^
    )
    ^
  File "/home/runner/work/markdown/markdown/markdown/test_tools.py", line 77, in assertMarkdownRenders
    self.assertMultiLineEqual(output, expected)
    ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
AssertionError: '<p><code>&lt;div</code>.</p>\n<p><div>\nhello\n</div></p>' != '<p><code>&lt;div</code>.</p>\n<div>\nhello\n</div>'
  <p><code>&lt;div</code>.</p>
- <p><div>
? ---
+ <div>
  hello
- </div></p>
?       ----
+ </div>


----------------------------------------------------------------------
Ran 1056 tests in 1.071s

FAILED (failures=4, skipped=13)

This failure first appeared in #1546.

[waylan]

For reference, the history of the HTML parser lib can be found here. Python's tags are listed here. Beta 3 was released on June 17 and beta 4 on July 8.

[facelessuser]

Ugh, this is some weird behavior. So if we look at this example:

` <div `.

<div>
hello
</div>

It's thinking this is the start tag.

<div `.

<div>

This seems wrong on their part. I'm not sure how to work around this yet, but this seems invalid. I'm not sure though if it is something that happens coupled with our various changes or if this is what their stock implementation does.

[facelessuser]

I think I figured it out