Improve package landing page

[aprotyas]

What?

This issue will target discussions about general improvements to a package's API documentation's landing page, housing the conversation in #20 (comment) somewhere outside that PR. Currently, the landing page simply draws the package description from the package manifest (package.xml), which is usually just a one-liner about the package; this description is followed by the API listing.

Ideally, the landing page for a package's API documentation should provide a longer description - and hence more context/information about the package, something usually found in an auxiliary documentation file, such as a README.

Screenshot of desired rcpputils documentation for visual reference

How?

My idea of improving the index page was to somehow inject/render the content of an optional, user-configurable documentation file in place of the package.description mapped into the index.rst template by the SphinxBuilder.

Implementation

There are a few methods to consider. To use any of these methods, the user would have to add a {use_mdfile_as_mainpage: <document>} option for SphinxBuilder in the package's rosdoc2.yaml file.

1. Using markdown parsers with include directives: Once Add markdown parsing support to rosdoc2 #21 is merged, the package description can be changed to an include directive with the specified file. This file would then just get parsed into rst format by the myst-parser. Problem: Relative file links will not work, because they link to exactly that file name in the host. On the other hand, the documents rendered by sphinx now don't even generate relative file links, let alone be broken - because documentation files are just rendered as program listings, similar to .cpp/.hpp files.

Proposed changes in sphinx_builder.py

    def generate_default_project_into_directory(self, directory):                                         
        if use_mdfile_as_mainpage:                                                                           
            package_description = f".. include:: {source-directory}/<documentation-file>"
        else:                                                                                       
            package_description = package.description                                               
        ...                                               
        template_variables = {                                                                      
            ...                                    
            'package_description': package_description,    

2. Programmatic use of markdown parsers to convert markdown to restructured text, which is then assigned to package_description. Problem: relative file links do not work still.

Proposed changes in sphinx_builder.py

    def generate_default_project_into_directory(self, directory):                                         
        if use_mdfile_as_mainpage:                                                                           
            package_description = parser.parse_md_to_rst({source-directory}/<documentation-file>)
        else:                                                                                       
            package_description = package.description                                               
        ...                                               
        template_variables = {                                                                      
            ...                                    
            'package_description': package_description,    

3. From docutils>=0.17, include the program_file_listing_<doc-file>.rst generated by exhale with a user-specified parser. Problem: My hunch is that relative file links will not work here either.

Proposed changes in sphinx_builder.py

    def generate_default_project_into_directory(self, directory):                                         
        if use_mdfile_as_mainpage:                                                                           
            package_description = f".. include:: api/program_listing_<doc_file>\n    :parser: <md-parser>"
        else:                                                                                       
            package_description = package.description                                               
        ...                                               
        template_variables = {                                                                      
            ...                                    
            'package_description': package_description,    

4. If the use_mdfile_as_mainpage flag is enabled, set the master_doc option in the default sphinx project's configuration to program_listing_<doc_file>.rst rather than index.rst. The landing page would render oddly in this case, because the program_listing_<doc_file>.rst is simply a markdown codeblock containing the content of the actual documentation file.

In summary, it looks like all of these methods suffer from the same problem of relative links. The latter pair may also suffer from unnecessary sphinx warnings about documents not being included in the toctree, but that warning is mostly harmless.

---

I would be open to input about how the landing page should actually be, and if this is an effort worth spending time on currently.

[clalancette]

First of all, thanks for starting this discussion. I agree that it would be really nice to improve the landing page.

However, I think we can do some more things automatically here. Looking at your original mockup, it looks like we have a structure that looks something like:

${package_name}
${package_name}: ${very short description}
${short description from package.xml}
Quality Declaration
${quality level from QUALITY_DECLARATION.md}
API
${human-readable description of APIs}
Features are described in more detail at docs/FEATURES.md
C++ API
...
Indices and Search
...

We already have the package_name and short description. We can relatively easily parse out the quality level from the QUALITY_DECLARATION.md, if it exists. That means the only things we don't have are the very short description, the human readable description of the APIs, and the link to the docs/FEATURES.md . Personally, I think all of those are optional. So I'd propose we start out by shooting for something like:

${package_name}
${short description from package.xml}
Quality Declaration
${quality level from QUALITY_DECLARATION.md}
C++ API
...
Indices and Search
...

That will give us a bit more information than we have now, without having to add something custom.

All of that said, I think it would be nice to also allow the user to include arbitrary information on that front page. I think we would want to allow the user to pass a rosdoc2.yaml configuration with the path to the file they want included on the front page, and then we include that file right after the Quality Declaration section. Whether we do that inclusion in the rosdoc2 code itself, inside of sphinx, or somewhere else isn't totally clear to me.

[aprotyas]

We can relatively easily parse out the quality level from the QUALITY_DECLARATION.md, if it exists.

Yes, I think extracting the quality level if one exists is a good idea.

That means the only things we don't have are the very short description, the human readable description of the APIs, and the link to the docs/FEATURES.md. Personally, I think all of those are optional.

Also, yes. Essentially, the README.md, and the docs folder are all optional content - and may not exhibit the same naming pattern across packages too. With this context, giving the user an optional configuration to point to arbitrary information (say, a README.md file) would make the most sense.

Whether we do that inclusion in the rosdoc2 code itself, inside of sphinx, or somewhere else isn't totally clear to me.

This change would probably be best housed in the rosdoc2_settings map in the sphinx configuration file (and not rosdoc2.yaml, which I feel is more about configuring/adding builders).

The problem of relative links being broken when the user-configurable file is rendered will probably still have to be addressed, but this shouldn't be a huge issue. I believe there is a way to not render links for a given page.

[jacobperron]

I don't know any technical details regarding rosdoc2, but it would be nice if it respected the configuration in an existing Doxyfile. For instance, I am able to setup the main landing page for my project with this setting: https://github.com/ros2/domain_bridge/blob/78c50f559bfb21bd5165f7272aa8cf40e32d81d5/Doxyfile#L9

But if I use rosdoc2 to build the documentation (instead of vanilla doxygen), then it does not have the desired affect.