Fix footnote ordering #1546
Fix footnote ordering #1546
Conversation
…heir references appear Fixes Python-Markdown#1367
|
The test failures for the |
While I understand that it may be easier to implement this way, I am unconvinced that it is imposable to implement in the inline processor. My concern is that it may match things it shouldn't. For example, what if a footnote is included in a code span? Some tests might convince me. In fact, this needs some new tests added to demonstrate that it is working as advertised. |
|
Thank you for the quick feedback. The code span example is definitely a valid concern. I can add some tests to demonstrate the functionality + a footnote reference inside a code span. I'll think about how to implement this with the inline processor. The problem is not detecting footnote references in the inline processor, it's the timing, since the tree processor which renders the footnotes runs before that. |
- Add changelog entry - Fix minor formatting issue
|
I've added a bunch of tests, including footnote references within code spans. See in particular Concerning the block vs inline discussion: One consideration I would to add is that my implementation does not remove or modify any of the footnote references/labels during block processing. It just registers the ids of any footnote references it finds in the blocks. I have not touched the parts of the code that remove substrings or render html elements. I merely reorder the dict that holds the ids and footnote content just before the footnotes are rendered. |
|
Sorry that I didn't mention this previously, but all new tests should be under the appropriate subdir of |
- Move new tests to proper file (tests/test_syntax/extensions/test_footnotes.py) - Adapt new tests to use markdown.test_tools.TestCase and style convention in test_footnotes.py - Remove a few of the new tests which turned out to be redundant, given the existing ones in test_footnotes.py
|
Ah, I see. No worries, I've moved and adapted the new tests. |
| def test_footnote_order(self): | ||
| """Test that footnotes occur in order of reference appearance.""" | ||
|
|
||
| self.assertMarkdownRenders( | ||
| 'First footnote reference[^first]. Second footnote reference[^last].\n\n' | ||
| '[^last]: Second footnote.\n[^first]: First footnote.', | ||
| '<p>First footnote reference<sup id="fnref:first"><a class="footnote-ref" ' | ||
| 'href="#fn:first">1</a></sup>. Second footnote reference<sup id="fnref:last">' | ||
| '<a class="footnote-ref" href="#fn:last">2</a></sup>.</p>\n' | ||
| '<div class="footnote">\n' | ||
| '<hr />\n' | ||
| '<ol>\n' | ||
| '<li id="fn:first">\n' | ||
| '<p>First footnote. <a class="footnote-backref" href="#fnref:first" ' | ||
| 'title="Jump back to footnote 1 in the text">↩</a></p>\n' | ||
| '</li>\n' | ||
| '<li id="fn:last">\n' | ||
| '<p>Second footnote. <a class="footnote-backref" href="#fnref:last" ' | ||
| 'title="Jump back to footnote 2 in the text">↩</a></p>\n' | ||
| '</li>\n' | ||
| '</ol>\n' | ||
| '</div>' | ||
| ) |
There was a problem hiding this comment.
This tests the new behavior (footnotes ordered by reference appearence).
| def test_footnote_reference_within_code_span(self): | ||
| """Test footnote reference within a code span.""" | ||
|
|
||
| self.assertMarkdownRenders( | ||
| 'A `code span with a footnote[^1] reference`.', | ||
| '<p>A <code>code span with a footnote[^1] reference</code>.</p>' | ||
| ) | ||
|
|
||
| def test_footnote_reference_within_link(self): | ||
| """Test footnote reference within a link.""" | ||
|
|
||
| self.assertMarkdownRenders( | ||
| 'A [link with a footnote[^1] reference](http://example.com).', | ||
| '<p>A <a href="http://example.com">link with a footnote[^1] reference</a>.</p>' | ||
| ) | ||
|
|
||
| def test_footnote_reference_within_footnote_definition(self): | ||
| """Test footnote definition containing another footnote reference.""" | ||
|
|
||
| self.assertMarkdownRenders( | ||
| 'Main footnote[^main].\n\n' | ||
| '[^main]: This footnote references another[^nested].\n' | ||
| '[^nested]: Nested footnote.', | ||
| '<p>Main footnote<sup id="fnref:main"><a class="footnote-ref" href="#fn:main">1</a></sup>.</p>\n' | ||
| '<div class="footnote">\n' | ||
| '<hr />\n' | ||
| '<ol>\n' | ||
| '<li id="fn:main">\n' | ||
| '<p>This footnote references another<sup id="fnref:nested"><a class="footnote-ref" ' | ||
| 'href="#fn:nested">2</a></sup>. <a class="footnote-backref" href="#fnref:main" ' | ||
| 'title="Jump back to footnote 1 in the text">↩</a></p>\n' | ||
| '</li>\n' | ||
| '<li id="fn:nested">\n' | ||
| '<p>Nested footnote. <a class="footnote-backref" href="#fnref:nested" ' | ||
| 'title="Jump back to footnote 2 in the text">↩</a></p>\n' | ||
| '</li>\n' | ||
| '</ol>\n' | ||
| '</div>' | ||
| ) | ||
|
|
||
| def test_footnote_reference_within_blockquote(self): | ||
| """Test footnote reference within a blockquote.""" | ||
|
|
||
| self.assertMarkdownRenders( | ||
| '> This is a quote with a footnote[^quote].\n\n[^quote]: Quote footnote.', | ||
| '<blockquote>\n' | ||
| '<p>This is a quote with a footnote<sup id="fnref:quote">' | ||
| '<a class="footnote-ref" href="#fn:quote">1</a></sup>.</p>\n' | ||
| '</blockquote>\n' | ||
| '<div class="footnote">\n' | ||
| '<hr />\n' | ||
| '<ol>\n' | ||
| '<li id="fn:quote">\n' | ||
| '<p>Quote footnote. <a class="footnote-backref" href="#fnref:quote" ' | ||
| 'title="Jump back to footnote 1 in the text">↩</a></p>\n' | ||
| '</li>\n' | ||
| '</ol>\n' | ||
| '</div>' | ||
| ) | ||
|
|
||
| def test_footnote_reference_within_list(self): | ||
| """Test footnote reference within a list item.""" | ||
|
|
||
| self.assertMarkdownRenders( | ||
| '1. First item with footnote[^note]\n1. Second item\n\n[^note]: List footnote.', | ||
| '<ol>\n' | ||
| '<li>First item with footnote<sup id="fnref:note">' | ||
| '<a class="footnote-ref" href="#fn:note">1</a></sup></li>\n' | ||
| '<li>Second item</li>\n' | ||
| '</ol>\n' | ||
| '<div class="footnote">\n' | ||
| '<hr />\n' | ||
| '<ol>\n' | ||
| '<li id="fn:note">\n' | ||
| '<p>List footnote. <a class="footnote-backref" href="#fnref:note" ' | ||
| 'title="Jump back to footnote 1 in the text">↩</a></p>\n' | ||
| '</li>\n' | ||
| '</ol>\n' | ||
| '</div>' | ||
| ) | ||
|
|
||
| def test_footnote_reference_within_html(self): | ||
| """Test footnote reference within HTML tags.""" | ||
|
|
||
| self.assertMarkdownRenders( | ||
| 'A <span>footnote reference[^1] within a span element</span>.\n\n[^1]: The footnote.', | ||
| '<p>A <span>footnote reference<sup id="fnref:1">' | ||
| '<a class="footnote-ref" href="#fn:1">1</a>' | ||
| '</sup> within a span element</span>.</p>\n' | ||
| '<div class="footnote">\n' | ||
| '<hr />\n' | ||
| '<ol>\n' | ||
| '<li id="fn:1">\n' | ||
| '<p>The footnote. <a class="footnote-backref" href="#fnref:1" ' | ||
| 'title="Jump back to footnote 1 in the text">↩</a></p>\n' | ||
| '</li>\n' | ||
| '</ol>\n' | ||
| '</div>' | ||
| ) |
There was a problem hiding this comment.
These test parsing of footnote references within other structures.
|
Sorry, don't review or merge this just yet. I've come up with a test case that messes up the order (when a footnote reference id inside e.g. a code span is identical to a proper footnote reference id elsewhere). It is an edge case, but I'd rather get this corrected before review. I'll provide a fix early next week. |
|
I've gone down the rabbithole with this and can report a few things:
It is indeed possible (trivial, even) to implement tracking the order of footnote references in an Inlineprocessor. What is not possible, however, is to then generate the list of footnotes in the order we have tracked, because the list of footnotes is produced by a Treeprocessor. To my knowledge, that is simply how the architecture is designed; Treeprocessors always run before Inlineprocessors. So instead of trying to generate the footnotes div in the correct order, I have explored different ways of reordering it in a Postprocessor. They all lead to a fair amount of reparsing that seems counterintuitive and excessive at this stage in the pipeline. I've had some succes with a regex-based reordering method that works on the generated footnotes div. But it fails So. Based on my work thus far, I will suggest that for this PR we go back to 0c2e4bf and proceed with review. This entails ignoring the edge case that ce45459 tests. It is a pretty remote possibility, but even if it happens, it will not produce any errors. It will simply result in an illogical order of footnotes. This is, in my view, much preferable to the current behavior where the order of footnote definitions dictates the order of footnotes. |
Actually, Inlineprocessors are wrapped in and run from markdown.treeprocessors.InlineProcessor, which is the first treeprocessor by default (as listed here). Therefore, it is possible to run a Treeprocessors either before or after inline processors by setting the priority either higher or lower than |
Footnotes are now displayed in the order they are referenced in the document text, not in the order they are defined.
Tracking footnote references is something the logically fits within inline processing. But it must happen before tree processing, because that is where the list of footnotes is rendered to html. For that reason, it is done within the block processor.
Fixes #1367