Skip to content
This repository has been archived by the owner on Feb 17, 2021. It is now read-only.

Commit

Permalink
add a parse script for #18
Browse files Browse the repository at this point in the history
  • Loading branch information
ryanatkn committed Oct 6, 2019
1 parent 080726c commit 200ddc7
Show file tree
Hide file tree
Showing 2 changed files with 141 additions and 0 deletions.
3 changes: 3 additions & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,9 @@
"version": "1.0.0",
"description": "multitudes of Svelte v3+ links",
"main": "README.md",
"scripts": {
"parse": "node parse.js resources.json"
},
"repository": {
"type": "git",
"url": "git+https://github.com/ryanatkn/awesome-svelte-resources.git"
Expand Down
138 changes: 138 additions & 0 deletions parse.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,138 @@
/*
Parses README.md into a JSON data structure for use in other projects.
```json
{
tags: [
"components and libraries",
// ...
],
resources: [
{
"name": "`@sveltejs/svelte-virtual-list`",
"repo": "https://github.com/sveltejs/svelte-virtual-list",
"description": "A virtual list component for Svelte apps",
"tags": [
"components and libraries",
"layout and structure"
]
},
// ...
]
}
```
Usage:
```bash
npm run parse
# or
node parse.js some_filename.json SOME_README.md
```
The motivating use case is to deprecate `awesome-svelte-resources`
in favor of an official Svelte resources website.
https://github.com/ryanatkn/awesome-svelte-resources/issues/18
TODO consider these changes
- add an npm field if name is in `backticks`
- change "repo" field to "url" if `<sub>` isn't detected
*/

const fs = require('fs');

const args = process.argv.slice(2);
const outputPath = args[0];
const inputPath = args[1] || 'README.md';

const omittedTags = ['contents', 'license'];

const BEGIN_TAG_MATCHER = /^(#+) (.+)/;
const BEGIN_RESOURCE_MATCHER = /^\W*-/;

const parseReadme = readme => {
const lines = readme.split('\n');

const resources = [];
const tags = [];
const currentTags = [];
let skipThisTag = false;

// depth is 0 for top-level tags
const beginTag = (tag, depth) => {
tags.push(tag);
while (currentTags.length > depth) {
currentTags.pop();
}
currentTags.push(tag);
};

for (let i = 0; i < lines.length; i++) {
const line = lines[i];

// begin tag
const tagMatches = line.match(BEGIN_TAG_MATCHER);
if (tagMatches) {
const tag = tagMatches[2];
const depth = tagMatches[1].length - 2; // `- 2` because `##` is the top level
skipThisTag = depth < 0 || omittedTags.includes(tag);
if (skipThisTag) continue;
beginTag(tag, depth);
continue;
}

if (skipThisTag) continue;

// resource
const isResource = BEGIN_RESOURCE_MATCHER.test(line);
if (isResource) {
const resourceLines = [line];
// resources are only 1 or 2 lines right now, so this only handles that case
const nextLine = lines[i + 1];
if (
nextLine &&
nextLine.startsWith(' ') &&
!BEGIN_RESOURCE_MATCHER.test(nextLine)
) {
resourceLines.push(nextLine);
i++;
}
resources.push(createResource(resourceLines, currentTags));
continue;
}
}

return {tags, resources};
};

const createResource = (lines, currentTags) => {
const [line1, line2] = lines;
if (lines[2]) {
throw Error(`Resources are expected to have max 2 lines - ${line1}`);
}
const matches = line1.match(/\[(.+)\]\((.+)\)/);
const description = line2 && line2.trim();
return {
name: matches[1],
repo: matches[2],
description,
tags: currentTags.slice(),
};
};

const transform = parsed => parsed;

const main = () => {
const readme = fs.readFileSync(inputPath, 'utf8');
const parsed = parseReadme(readme);
const data = transform(parsed);
console.log(`parsed ${data.resources.length} resources`);
const outputData = JSON.stringify(data, null, 2);
fs.writeFileSync(outputPath, outputData, 'utf8');
console.log(`wrote to ${outputPath}`);
console.log('all done');
};

main();

4 comments on commit 200ddc7

@swyxio
Copy link

@swyxio swyxio commented on 200ddc7 Oct 7, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this would be a useful CLI for other repos to use :)

@ryanatkn
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting idea! It was easy to write because I kept such a strict format and it's hardcoded to those conventions. Do any specific projects come to mind? The idea of parsing lots of related readmes or awesome lists and putting a good UI on the data is enticing.

@thompcd
Copy link

@thompcd thompcd commented on 200ddc7 Oct 7, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is actually a very pleasant surprise. This is one of the first things that came to my mind when thinking of a Sapper version - I wanted to parse it to json and add a 'tags' field to entries to allow index/ search by category

@swyxio
Copy link

@swyxio swyxio commented on 200ddc7 Oct 7, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no specific project. but i guess itd be a cool thing for many awesome lists.

i have a similar list that could do with some conversion: https://github.com/sw-yx/spark-joy/ but i havent kept to your same format.

Please sign in to comment.