PR Summary

Very much a work in progress, but almost all tests pass (new cmdlet needs help for Get-Help) and json files are usable wherever .psd1 settings file can be used.

Json is a widely used format with lots of great existing tooling. It could offer some nice UX in future (could publish JSON Schemas to offer inline validation in tools such as VS Code etc).

Along with changes aimed at making Json (and potentially other formats) viable I wanted settings to be more discoverable. To that end I've:

• Proposed a new cmdlet New-ScriptAnalyzerSettingsFile which would allow creation of settings file in desired format. One option is -All which populates file with all rules and all configurable rule options (set to their defaults).

• Added a list of options to the RuleInfo class. For configurable built-in rules, the options, their types, and default value are populated.

  Get-ScriptAnalyzerRule -Name 'PSAvoidUsingPositionalParameters' | Select -expand Options
  
  Name             OptionType      DefaultValue
  ----             ----------      ------------
  CommandAllowList System.String[] {}
  Enable           System.Boolean  True

Additional change detail:

• Internal location of Settings Presets moved from Engine\Settings -> Engine\SettingsPresets so
  Settings folder can be used for all Settings logic and parsers. Presets are still shipped in the
  built module's Settings folder.

• Internal location of Command Data Files moved from Engine\Settings -> Engine\CommandDataFiles.
  Now separated out from the Settings Presets, as presets could now be in .json format - which is
  what previously distinguished the command data files from presets. They are shipped with the
  module in a new folder CommandDataFiles.

• GetShippedSettingsDirectory() abstracted to GetShippedModuleSubDirectory(string subDirectoryName)
  as there are now two folders, Settings, and CommandDataFiles, which consumers may care about.
  Usages updated in AvoidOverwritingBuiltInCmdlets and UseCompatibleCmdlets which load and use
  the Command Data Files.

• All state about settings data moved to SettingsData class. Settings class becomes a pure
  static helper. It's responsible for the supported file formats, preset resolution, auto-discovery,
  and orchestrating the deserialization of a settings file into a SettingsData and vice-versa.

• An interface ISettingsFormat describes a Settings Format. It can serialize a SettingsData to a
  string representation of the intended format and deserialize that string representation into a
  SettingsData. 2 Implementations of this are Psd1SettingsFormat and JsonSettingsFormat.

• The order of the Settings Formats in the Settings class (s_formats) defines the order in which
  formats are checked during Auto-Discovery and Preset Resolution. For instance if JsonSettingsFormat
  precedes Psd1SettingsFormat:

  • During Auto-Discovery, if both PSScriptAnalyzerSettings.json and PSScriptAnalyzerSettings.psd1
    exist, PSScriptAnalyzerSettings.json will be used.

  • During Preset resolution, if the preset CmdletDesign is used and both CmdletDesign.json and
    CmdletDesign.psd1 exist in the presets folder, the settings in CmdletDesign.json would be
    used. Note: In practice, this should be avoided. A test to ensure preset uniqueness should
    be added. Presets should only be shipped in a single format.

• When passing values in via -Settings:

  • When nothing supplied and no settings file in cwd:

    PS > Invoke-ScriptAnalyzer -Path . -Verbose
    VERBOSE: Settings object could not be resolved.

    Same as current behaviour. Auto discovery fails. Built-in defaults used

  • When nothing supplied and a psd1 settings file in cwd:

    PS > Invoke-ScriptAnalyzer -Path . -Verbose
    VERBOSE: Settings not provided. Will look for settings file in the given path C:\Scratch.
    VERBOSE: Found C:\Scratch\PSScriptAnalyzerSettings.psd1. Will use it to provide settings for this invocation.

    Different from current which omits the path. Note the space between path and period below
    which should contain the cwd path.

    PS > Invoke-ScriptAnalyzer -Path . -Verbose
    VERBOSE: Settings not provided. Will look for settings file in the given path .
    VERBOSE: Found C:\Scratch\PSScriptAnalyzerSettings.psd1. Will use it to provide settings for this invocation.

  • When nothing supplied and a json settings file in cwd:

    PS > Invoke-ScriptAnalyzer -Path . -Verbose
    VERBOSE: Settings not provided. Will look for settings file in the given path C:\Scratch.
    VERBOSE: Found C:\Scratch\PSScriptAnalyzerSettings.json. Will use it to provide settings for this invocation.

    Different from current which doesn't know about the json format.

    PS > Invoke-ScriptAnalyzer -Path . -Verbose
    VERBOSE: Settings object could not be resolved.

  • When incorrect type supplied:

    PS > Invoke-ScriptAnalyzer -Path . -Settings $false
    Invoke-ScriptAnalyzer: Settings should be either a file path, built-in preset or a hashtable.

    Differs from current behaviour which proceeds with built-in settings, noting only that settings
    cannot be found. This is inconsistent with when an invalid path is supplied when execution
    doesn't proceed.

  • When Preset name supplied:

    PS > Invoke-ScriptAnalyzer -Path . -Settings CmdletDesign -Verbose
    VERBOSE: Using settings preset CmdletDesign. File found at C:\Home\PSScriptAnalyzer\out\PSScriptAnalyzer\1.24.0\Settings\CmdletDesign.psd1.

    Differs from current behaviour - calls out that a preset was used.

  • When explicit valid psd1 file path supplied:

    PS > Invoke-ScriptAnalyzer -Path . -Settings C:\Scratch\PSScriptAnalyzerSettings.psd1 -Verbose
    VERBOSE: Using settings file at C:\Scratch\PSScriptAnalyzerSettings.psd1.

    Same as current behaviour.

  • When explicit valid json file path supplied:

    PS > Invoke-ScriptAnalyzer -Path . -Settings C:\Scratch\PSScriptAnalyzerSettings.json -Verbose
    VERBOSE: Using settings file at C:\Scratch\PSScriptAnalyzerSettings.json.

    Different from current behaviour which fails with:

    PS > Invoke-ScriptAnalyzer -Path . -Settings C:\Scratch\PSScriptAnalyzerSettings.json -Verbose
    VERBOSE: Using settings file at C:\Scratch\PSScriptAnalyzerSettings.json.
    Invoke-ScriptAnalyzer: Settings file 'C:\Scratch\PSScriptAnalyzerSettings.json' is invalid because it does not contain a hashtable.

  • When explicit invalid file path supplied:

    PS > Invoke-ScriptAnalyzer -Path . -Settings C:\Scratch\NoSuch.psd1 -Verbose
    Invoke-ScriptAnalyzer: Cannot resolve settings file path 'C:\Scratch\NoSuch.psd1'.

    Same as current behaviour.

• New Cmdlet New-ScriptAnalyzerSettingsFile. Intent is to make it easy to create settings files and discover available settings/options without having to directly resort to docs. It creates settings files based on presets (or All available settings set to their defaults). Creates file in nominated format at desired path. Path defaults to cwd and file format defaults to highest precedence file format in Settings.cs (currently .json). Has argument completers for FileFormat and Base.

  Gist of json and psd1 files created by cmdlet here.

Still to look at:

• Lots of testing. Plenty of coverage needed to ensure no regression. Testing around loading formats and that each format produces the same SettingsData object for equivalent input.
• Need to check on other (case sensitive) platforms that there are no issues.

PR Checklist

• PR has a meaningful title
  • Use the present tense and imperative mood when describing your changes
• Summarized changes
• Change is not breaking
• Make sure all .cs, .ps1 and .psm1 files have the correct copyright header
• Make sure you've added a new test if existing tests do not effectively test the code changed and/or updated documentation
• This PR is ready to merge and is not Work in Progress.
  • If the PR is work in progress, please add the prefix WIP: to the beginning of the title and remove the prefix when the PR is ready.