[SVCS-418] MFR Markdown update + Mathjax #272

AddisonSchiller · 2017-08-11T19:19:56Z

refs: https://openscience.atlassian.net/browse/SVCS-418

Purpose

Currently MFR and the OSF wiki pages do not use the same flavor of markdown to render .md files. This inconsitency could be a problem, and MFR should be upgraded to use the same renderer as the OSF wiki.

Summary of Changes

Added support for mathjax (via cdn), markdown-it and its noted extensions.
While the old renderer was solely python, this one is mostly in javascript.

Updating the markdown renderer to use markdown-it instead of original markdown. Includes support for various extensions such as:

@centerforopenscience/markdown-it-toc
markdown-it-highlightjs
markdown-it-ins-del
markdown-it-sanitizer
markdown-it-mathjax

Testing Notes/QA Notes

Nearly all markdown files should be able to be rendered. Code blocks should be highlighted, and inline math equations etc should display properly. Inline html should work. Here is an example equation that should render correctly:

$$\sum_{i=0}^n i^2 = \frac{(n^2+n)(2n+1)}{6}$$

HTML tags in .md files should also be tested thoroughly. This could be a problem due to the fact that html sanitization has to happen before being rendered. This means that html tags will not appear as inserted html, but as text. (this is contrary to what is happening on the wiki)

Try breaking it with unclosed html tags as well.

Other Notes

Once MFR has its javascript library packaging re-worked, this ticket will also need to be looked at. Currently we are grabbing mathjax from a CDN. During original testing I had it local, but the package is so large that it is not in good practice to include it in this commit ( as would be needed due to MFRs current js package set up)

This md renderer is not exactly like the osf wiki, there are a few issues:
Style isn't exact, I had trouble matching style.
Scroll bar issues. There are some issues with scrollbars appearing when they shouldn't (iframe problems)
Escape characters dont work exactly like they do on the wiki,
ie, < shows up as just < on the wiki. On MFR it will show up as &lt, which is a standard way of displaying a "Less than" sign

Updating the markdown renderer to use markdown-it instead of original markdown. Upgrade bleach

cslzchen

First pass done. As discussed, here is a few things in addition to the comments:

Fix firefox scrollbar issue.
Fix html tag sanitizing issue.
Try and figure out the difference between mfr and wiki when javascript is turned off

FYI, here is link for sanitization in OSF for wiki:
https://github.com/CenterForOpenScience/osf.io/blob/develop/addons/wiki/models.py#L72, which calls bleach with whitelist.

cslzchen · 2017-11-28T15:38:32Z

mfr/extensions/md/README.md

+
+to get all the libraries needed:
+
+```


change to ```bash

Add note that you do not need to use npm, can just get off of github as well.

cslzchen · 2017-11-28T15:38:44Z

mfr/extensions/md/README.md

+
+To add a new library that is not already set up to export to `root` can be a bit tricky, but the gist of it is, wrap the plugin in this code:
+
+```


add ```javascript

cslzchen · 2017-11-28T15:39:37Z

mfr/extensions/md/README.md

+
+And then modify it to work in this context. See other plugins for examples.
+
+Then, in md.js, you can add a plugin to the markdown renderer by adding a `.use(<window.<PLUGIN_NAME>)`


add comment on how to load into template.mako

cslzchen · 2017-11-28T15:41:13Z

mfr/extensions/md/README.md

+})(this, function () {
+
+
+return function(md){


indent nicer

cslzchen · 2017-11-28T15:46:25Z

mfr/extensions/md/render.py


    def render(self):
        """Render a markdown file to html."""
        with open(self.file_path, 'r') as fp:
-            body = markdown.markdown(fp.read(), extensions=[EscapeHtml()])
+            body = bleach.clean(fp.read())


comment on why we bleach before sending to template

cslzchen · 2017-11-28T15:58:51Z

mfr/extensions/md/README.md

+npm install [email protected]
+npm install [email protected]
+npm install [email protected]
+npm install [email protected]


Add links to github versions, so we can see differences in the files you modified.

cslzchen · 2017-11-28T16:11:11Z

mfr/extensions/md/templates/viewer.mako

 </div>

 <script src="/static/js/mfr.js"></script>
 <script src="/static/js/mfr.child.js"></script>
+<script src="${base}/js/markdown-it.min.js"></script>


Order matters, add comment for order.

cslzchen · 2017-11-28T16:12:02Z

mfr/extensions/md/templates/viewer.mako

-<link rel="stylesheet" href="static/css/default.css">
-<div style="word-wrap: break-word;" class="mfrViewer">
-${body}
+<link href='https://fonts.googleapis.com/css?family=Open+Sans:400,600,300' rel='stylesheet' type='text/css'>


mess with versions and re add the other osf file, see if it makes a difference.

cslzchen · 2017-11-28T16:13:21Z

mfr/extensions/md/templates/viewer.mako

-<link rel="stylesheet" href="static/css/default.css">
-<div style="word-wrap: break-word;" class="mfrViewer">
-${body}
+<link href='https://fonts.googleapis.com/css?family=Open+Sans:400,600,300' rel='stylesheet' type='text/css'>


style the file a bit better

cslzchen · 2017-11-28T16:34:18Z

mfr/extensions/md/static/js/md.js

+};
+
+var markdown = new MarkdownIt('commonmark', {
+    html: false,


check default on this, see if true makes it more like the osf side

It indeed does when proper sanitization is used 👍

Instead of bundling up assets, serve the smaller ones. Larger ones can be loaded from cdn.

more info on readme

cslzchen · 2017-12-07T21:18:57Z

Replaced by #305.

AddisonSchiller force-pushed the feature/markdown-upgrade branch 2 times, most recently from 727621f to 26e5b64 Compare August 15, 2017 18:20

AddisonSchiller changed the title ~~MFR Markdown update + Mathjax~~ [SVCS-418] MFR Markdown update + Mathjax Oct 5, 2017

cslzchen added Code Review Final Review labels Oct 12, 2017

felliott removed the Final Review label Nov 21, 2017

cslzchen added Final Review and removed Code Review labels Nov 21, 2017

cslzchen requested review from felliott and removed request for felliott November 21, 2017 16:52

cslzchen added Code Review and removed Final Review labels Nov 21, 2017

MFR Markdown update + Mathjax

49ae179

Updating the markdown renderer to use markdown-it instead of original markdown. Upgrade bleach

AddisonSchiller force-pushed the feature/markdown-upgrade branch from 92625c4 to 7cd21a8 Compare November 28, 2017 14:50

cslzchen requested changes Nov 28, 2017

View reviewed changes

Removing bundle and reworking

ee73df0

Instead of bundling up assets, serve the smaller ones. Larger ones can be loaded from cdn.

AddisonSchiller force-pushed the feature/markdown-upgrade branch 2 times, most recently from a90d297 to d7ef557 Compare November 29, 2017 20:06

Refactoring, making raw html work

7b69286

more info on readme

AddisonSchiller force-pushed the feature/markdown-upgrade branch from 4df6030 to 7b69286 Compare November 30, 2017 14:46

cslzchen mentioned this pull request Dec 7, 2017

[SVCS-418] Render MD Files at the Frontend #305

Open

2 tasks

cslzchen closed this Dec 7, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[SVCS-418] MFR Markdown update + Mathjax #272

[SVCS-418] MFR Markdown update + Mathjax #272

Uh oh!

AddisonSchiller commented Aug 11, 2017 •

edited

Loading

Uh oh!

cslzchen left a comment

Uh oh!

cslzchen Nov 28, 2017

Uh oh!

cslzchen Nov 28, 2017

Uh oh!

cslzchen Nov 28, 2017

Uh oh!

cslzchen Nov 28, 2017

Uh oh!

cslzchen Nov 28, 2017

Uh oh!

cslzchen Nov 28, 2017

Uh oh!

cslzchen Nov 28, 2017

Uh oh!

cslzchen Nov 28, 2017

Uh oh!

cslzchen Nov 28, 2017

Uh oh!

cslzchen Nov 28, 2017

Uh oh!

AddisonSchiller Nov 29, 2017

Uh oh!

cslzchen commented Dec 7, 2017

Uh oh!

Uh oh!


		To add a new library that is not already set up to export to `root` can be a bit tricky, but the gist of it is, wrap the plugin in this code:

		```


		And then modify it to work in this context. See other plugins for examples.

		Then, in md.js, you can add a plugin to the markdown renderer by adding a `.use(<window.<PLUGIN_NAME>)`

		})(this, function () {


		return function(md){

[SVCS-418] MFR Markdown update + Mathjax #272

[SVCS-418] MFR Markdown update + Mathjax #272

Uh oh!

Conversation

AddisonSchiller commented Aug 11, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Purpose

Summary of Changes

Testing Notes/QA Notes

Other Notes

Uh oh!

cslzchen left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cslzchen commented Dec 7, 2017

Uh oh!

Uh oh!

AddisonSchiller commented Aug 11, 2017 •

edited

Loading