Add support for custom help groups

[smeijer]

I think it'd be helpful if we had a way to group commands and options, to create sections in --help.

There's a prior issue to this; #78, but that's over 9 years old. I'm also aware of addHelpText (#1296), but when using that to group commands, it adds boilerplate due to the need of needing to "hide" commands, and later formatting and properly indenting help ourselves.

What I wish to achieve is something like:

Usage: magicbell [options] [command]

Work with MagicBell from the command line

Options:
  -V, --version             output the version number
  -p, --profile <string>    Profile to use (default: "default")
  -h, --help                display help for command

Commands:
  broadcasts                Manage broadcasts
  imports                   Manage imports
  listen                    Listen to events for a users inbox
  metrics                   Manage metrics
  notification-preferences  Manage notification preferences
  notifications             Send and retrieve notifications
  push-subscriptions        Manage push subscriptions
  subscriptions             Manage subscriptions
  users                     Manage all known users

Other commands:
  config                    Display or change config values
  login                     Login to your account
  logout                    Logout of your account
  help                      Display help for a command

This separates 'config commands' from 'core commands'. If you'd run stripe --help or gh --help you can see similar command grouping.

At this moment, I achieve the above via:

for (const command of commands) {
  program.addCommand(command);
}

for (const command of otherCommands) {
  program.addCommand(command, { hidden: true });
}

program.addHelpCommand(false);
const padding = Object.values([...commands, ...otherCommands]).reduce((acc, command) => {
  return Math.max(acc, command.name().length);
}, 0);

program.addHelpText(
  'after',
  `
Other commands:
  ${otherCommands.map((c) => `${c.name().padEnd(padding, ' ')}  ${c.description()}`).join('\n  ')}
  ${'help'.padEnd(padding, ' ')}  Display help for a command
`,
);

That's doable, but I think it'd be nice to have native support for it. Something like:

program.addCommand(command, { group: 'Other commands' });
program.addOption(option, { group: 'Global flags' });

The group can both serve as group title, and grouping key.

[shadowspawn]

Thanks for the example, and creative partial work-around.

I see option and command groups in "big" utilities and have wondered about grouping or separator support in Commander, but have not progressed to ideas.

I had one related old comment saved:

• Request to support for -- parameter separator. #374 (comment)

[shadowspawn]

I did some research looking for prior art for adding options groups in the help. I found a few variations:

Python argparse has add_argument_group(), which creates an intermediate object, and then you add options to the group rather than the command: https://docs.python.org/dev/library/argparse.html#argument-groups

Yargs has cmd.group(option_keys) or using group property on option.

command-line-args has a group property when adding options: https://github.com/75lb/command-line-usage/wiki/How-to-break-the-option-list-up-into-groups

Oclif options have a helpGroup property:

• https://oclif.io/docs/flags
• feat: help improvements and customizability oclif/core#184

[smeijer]

Yargs still has an open issue to group commands. The groups only work for flags there I believe.

In the meanwhile, I've improved my work-around a bit, by adding a group method to my commands, and overriding the formatHelp method. I really like this approach of groups and the custom help formatter. If I were to add support for a callback, we could easily invoke the callback, and pass it the sections, so users can easily customise the help output. This would also be useful for customisation of section headings.

As you might see, I've customised the help quite a bit now. I wasn't looking for that, that's just a side effect of writing the custom help formatter like this 🙂.

Also, if there's any interest in this work, I'd be happy to submit a pull-request. If not, no hard feelings. I'm happy enough with my current solution.

createCommand

source: https://github.com/magicbell-io/magicbell-js/blob/e019d664762ab10386596a34c785658d5790bc02/packages/cli/src/lib/commands.ts#L5-L29

export function createCommand(name?: string): ExtendedCommand {
  const command = new Command(name) as ExtendedCommand;

  command.helpOption('-h, --help', 'Show help for command');
  command.showHelpAfterError(true);
  command.addHelpCommand(false);

  command.configureHelp({
    sortSubcommands: true,
    showGlobalOptions: true,
    subcommandTerm: (cmd) => cmd.name(),
    // with callback support, this could look like formatHelp((sections) => generateHelpString(sections))
    formatHelp: formatHelp,
  });

  command.group = (group) => {
    (command as any)._group = group;
    return command;
  };

  return command;
}

formatHelp

source: https://github.com/magicbell-io/magicbell-js/blob/01e5fd93c3de85a61ce70a7dac5d81f007cc5d63/packages/cli/src/lib/help.ts

import { Command, Help } from 'commander';
import kleur from 'kleur';

export function formatHelp(cmd: Command, helper: Help) {
  const termWidth = helper.padWidth(cmd, helper);
  const helpWidth = helper.helpWidth || 80;

  const itemSeparatorWidth = 2; // between term and description
  const indent = ' '.repeat(2);
  const moveOptions = !cmd.parent && cmd.commands.length;

  function formatItem(term, description) {
    if (description) {
      const fullText = `${term.padEnd(termWidth + itemSeparatorWidth)}${description}`;
      return helper.wrap(fullText, helpWidth - indent.length, termWidth + itemSeparatorWidth);
    }

    return term;
  }

  function formatList(textArray) {
    const list = textArray.join('\n').replace(/^/gm, indent).trim();
    return list ? indent + list : '';
  }

  const sections = {
    description: '',
    usage: '',
    arguments: '',
    options: '',
    commands: [] as { title: string; list: string }[],
    globalOptions: '',
  };

  sections.description = helper.commandDescription(cmd);
  sections.usage = helper.commandUsage(cmd);
  sections.arguments = formatList(
    helper.visibleArguments(cmd).map((argument) => {
      return formatItem(helper.argumentTerm(argument), helper.argumentDescription(argument));
    }),
  );

  // Note: options might benefit from similar grouping as commands below, I just didn't need that (yet)
  sections.options = formatList(
    helper
      .visibleOptions(cmd)
      .filter((option) => {
        // move --help to global options
        if (cmd.parent && option.long === '--help') {
          cmd.parent.addOption(option);
          return false;
        }

        return true;
      })
      .map((option) => {
        return formatItem(helper.optionTerm(option), helper.optionDescription(option));
      }),
  );

  // Commands
  const commands: Record<string, Command[]> = {};
  for (const command of helper.visibleCommands(cmd)) {
    const group = ((command as any)._group || 'Commands').trim();
    commands[group] = commands[group] || [];
    commands[group].push(command);
  }

  sections.commands = Object.entries(commands).map(([title, commands]) => ({
    title,
    list: formatList(commands.map((cmd) => formatItem(helper.subcommandTerm(cmd), helper.subcommandDescription(cmd)))),
  }));

  sections.globalOptions = this.showGlobalOptions
    ? formatList(
        helper
          .visibleGlobalOptions(cmd)
          .filter((option) => {
            // don't return --version on sub commands
            return option.long !== '--version';
          })
          .map((option) => {
            return formatItem(helper.optionTerm(option), helper.optionDescription(option));
          }),
      )
    : '';

  // -------
  //  Everything below here could be wrapped in a callback, so formatHelp(cb) can be used as formatter
  // -------

  const output = [];
  output.push(kleur.bold('Usage'), indent + sections.usage, '');

  if (sections.arguments) {
    output.push(kleur.bold('Arguments'), sections.arguments, '');
  }

  if (sections.options && !moveOptions) {
    output.push(kleur.bold('Options'), sections.options, '');
  }

  if (sections.commands.length) {
    sections.commands.forEach((section) => {
      output.push(kleur.bold(section.title), section.list, '');
    });
  }

  if (sections.options && moveOptions) {
    output.push(kleur.bold('Options'), sections.options, '');
  }

  if (sections.globalOptions) {
    output.push(kleur.bold('Global Options'), sections.globalOptions, '');
  }

  return output.join('\n');
}

usage

source: https://github.com/magicbell-io/magicbell-js/blob/e019d664762ab10386596a34c785658d5790bc02/packages/cli/src/index.ts#L42-L51

for (const command of commands) {
  program.addCommand(command.group('Resource commands'));
}

for (const command of otherCommands) {
  program.addCommand(command.group('Other commands'));
}

Result:

[shadowspawn]

Interesting, thanks.

I was wondering about the order the command groups will appear in the help list. I think that implementation will give varying results depending on the alphabetical ordering of the commands? Probably the order they were first used is good enough. So could perhaps create an empty commands[group] with a scan through the commands in original order, then populate the groups in a scan through the visibleCommands.

[smeijer]

Oh interesting. I haven't thought about the order. It just "naturally" worked. But yeah, when I'd drop the broadcasts command, "Other commands" move to the top 😅 . Not a direct issue for me now, but I'll have to think on how to solve that. Maybe track the calls to .group('...') and use the order of creation?

[shadowspawn]

I was suggesting could create empty groups using the raw commands (unsorted, so in creation order), then populate the groups in a scan through the visibleCommands. And ignore zero-length groups that turned out to be all hidden. So can still all be done in help generation.

[shadowspawn]

I noticed there is a request for untitled help groups "hidden" in the referenced PR:

• Expand help customisation with .addHelpText #1296 (comment)

[shadowspawn]

Opened a draft Pull Request: #2328

Adds low-level .helpGroup() on Option and Command objects, and high level .optionsGroup() and .commandsGroup() which can use used to group following options and commands.

const devOptionsHeading = 'Development Options:';
const managementCommandsHeading = 'Management Commands:';

 // The high-level approach is use .optionsGroup() and .commandsGroup() before adding the options/commands.
 const docker1 = program.command('docker1')
   .optionsGroup(devOptionsHeading)
   .option('-d, --debug', 'add extra trace information')
   .option('-w, --watch', 'run and relaunch service on file changes');

 docker1.commandsGroup(managementCommandsHeading);
 docker1.command('images').description('manage images');
 docker1.command('volumes').description('manage volumes');

 // The low-level approach is using .helpGroup() on the Option or Command.
 const docker2 = program.command('docker2')
   .addOption(
     new Option('-d, --debug', 'add extra trace information'
  ).helpGroup(devOptionsHeading),
   )
   .addOption(
     new Option(
       '-w, --watch', 'run and relaunch service on file changes',
     ).helpGroup(devOptionsHeading),
   );

docker2
   .command('images')
   .description('manage images')
   .helpGroup(managementCommandsHeading);
 docker2
   .command('volumes')
   .description('manage volumes')
   .helpGroup(managemen