Skip to content

Update data storage format from annoying Markdown table rows, and automate updating README.md#137

Merged
glasss13 merged 35 commits intomainfrom
automations
Sep 6, 2025
Merged

Update data storage format from annoying Markdown table rows, and automate updating README.md#137
glasss13 merged 35 commits intomainfrom
automations

Conversation

@andrlime
Copy link
Copy Markdown
Member

@andrlime andrlime commented Jul 27, 2025

This PR automates updating the README.md file with YAML data files, and an OCaml parser that uses an existing ppx library. Here is why this change is good, on balance:

  1. Changes in the Markdown format can easily create huge merge conflicts that, despite being a single line of conflict, flag the entire table as a conflict and require annoying manual resolution.
  2. Updating a single string "blah blah" is easier than manually looking for which part of the Markdown table to update which is prone to error.
  3. More extensible - output format is pretty easily updatable in a single file, and a GitHub action auto updates the entire README.md
  4. If some formatting breaks, there's one source of truth instead of possibly 20+

Here is why this PR is dubious/bad
1. Needs some work on the OCaml side of things - that's to say, updating OCaml structs isn't exactly user friendly. The idea here is to take YAML files that contain the same info and just parse it into the structs. That should resolve the issue pretty quickly. Fixed via YAML parsing
2. GitHub action takes a few minutes to run because even though it caches dependencies, recompiles them or something. At most five minutes though, not too big of an issue. If this is too slow, someone can just run manually, force push, and merge. Takes two minutes with caching

@andrlime andrlime changed the title Test PR :) Update data storage format from annoying Markdown table rows, and automate updating README.md Jul 27, 2025
@andrlime andrlime force-pushed the automations branch 2 times, most recently from 3dd7cac to 79613e2 Compare July 27, 2025 19:15
@echavemann
Copy link
Copy Markdown
Member

huge.

Copy link
Copy Markdown
Member

@egelja egelja left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work! This is sick. Two small things.

name: "[ci] Lint and compile source code"

on:
pull_request:
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Id recommend only doing the git push on main, but running it on PR or push to main. It’s confusing seeing commits pushed on a PR. You can upload the generated readme to artifacts and link to it in a comment in the PR for preview functionality.

Also, this should only run when files in data/ are changed.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. On push, commits stuff. On PR, adds a comment with a link to a zipped artifact. A concern here I have is, we have branch protections on main. If we "push to main" via a merge, is the bot able to push to a protected branch? If not, I'd say pushing to the PR is still cleaner.
  2. Yup, added artifacts and comments
  3. Added a check for the data/ folder

Copy link
Copy Markdown
Member

@egelja egelja Jul 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@andrlime andrlime force-pushed the automations branch 2 times, most recently from d19f7ad to b635058 Compare July 28, 2025 15:08
@github-actions
Copy link
Copy Markdown
Contributor

@northwesternfintech northwesternfintech deleted a comment from github-actions bot Jul 28, 2025
@northwesternfintech northwesternfintech deleted a comment from github-actions bot Jul 28, 2025
@northwesternfintech northwesternfintech deleted a comment from github-actions bot Jul 28, 2025
@github-actions
Copy link
Copy Markdown
Contributor

@andrlime andrlime requested a review from egelja July 28, 2025 15:16
@github-actions
Copy link
Copy Markdown
Contributor

@github-actions
Copy link
Copy Markdown
Contributor

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Sep 6, 2025

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Sep 6, 2025

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Sep 6, 2025

@glasss13 glasss13 merged commit 812c0d5 into main Sep 6, 2025
@glasss13 glasss13 deleted the automations branch September 6, 2025 00:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants