feat(doc): move to structured format

[lewis6991]

This change introduces the files docs.lua and docgen.lua
which are used to generate docs.md.

docs.lua use a bespoke format designed so it can emit the docs.md
as close to the original version.

docgen.lua takes docs.lua as an input an emits docs.md.

To regenerate the docs.md, simply run lua docgen.lua.

Closes #632

[clason]

Closes #632

Having a structured "source of truth" for these would also allow generating meta files that can be ingested by structural analysis tools (lua-language-server, emmylua-rust-analyzer) as a maintained replacement of https://github.com/LuaCATS/luv.

(This can be automated with a GH workflow to ensure that docs and meta files are always updated and consistent.)

[truemedian]

What does moving from docs.md to a structured docs.lua gain that wouldn't be gained by moving all of the documentation to inline (using something like ldoc, and then potentially our own docgen so we maintain the current style), so the documentation for each function is as close to its source as possible, to decrease likelihood that the code and documentation become out of sync.

[lewis6991]

What does moving from docs.md to a structured docs.lua gain that wouldn't be gained by moving all of the documentation to inline

What do you mean by inline?

(using something like ldoc, and then potentially our own docgen so we maintain the current style)

Ldoc is pretty useless for what we want here. The goal is to also generate https://github.com/LuaCATS/luv which I've got a working script for, I can add that to this PR.

so the documentation for each function is as close to its source as possible, to decrease likelihood that the code and documentation become out of sync.

This PR already does this, and does so whilst making minimal changes to docs.md.

We could have https://github.com/LuaCATS/luv as source, but parsing that is non-trivial and realistically requires external tools. This PR only depends on Lua.

I'll also mention there has already been agreement to do exactly what this PR implements.

[truemedian]

What do you mean by inline?

Adjacent to the C code that implements the function.

Ldoc is pretty useless for what we want here.

The tool is, definitely; the format is well adjusted to documenting C modules though, especially if we allow diverging from the format. Or consider adapting another format like LuaCATS to be reasonable to use in C code.

This PR already does this

This documentation moves the source of truth for documentation from docs.md to docs.lua, not from docs.md to the implementations in the C code.

I'll also mention there has already been #632 (comment) to do exactly what this PR implements.

Don't get me wrong, I agree and this is a good step in the right direction, but this is a good chance to think deeper about how this can be implemented such that we never have to think about it again.

[lewis6991]

Having the docs in the C code would be ideal. However, I do not have a strategy on how to achieve that. E.g. how would you handle the async and sync variations of functions? Documenting the structures of all the different table inputs/outputs. Union types? Doc hierarchy? Constants?

My goals here are:

• for this repo to begin maintaining LuaCATs meta files for the API this project provides.
• changing docs.md as little as possible, that can be done in future changes.
• avoiding introducing any new external dependencies

If it helps, I wouldn't mind this PR being viewed as a stop-gap solution until something more sophisticated is introduced.

[squeek502]

Looks great to me, thanks for doing this! Very cool to see that the docs.md changes are basically all just straightforward fixes.