Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding assert_equals_golden and assert_output_equals_golden #67

Open
wants to merge 6 commits into
base: master
Choose a base branch
from

Commits on May 8, 2024

  1. Adding assert_equals_golden and assert_output_equals_golden

    Adding support for "golden files". Golden files are (often text) files that are checked into a repo and are used in testing for asserting strings against their content.
    They have a number of useful properties that make them a great alternative to single (possibly repetitive) assertions in files.
    Some benefits include:
      * WYSIWYG plaintext output assertions (separating asserted output from test case logic).
      * Test failure output that is searchable (asserted output is checked into repo as files).
      * Clear file diffs of test assertions during merge / code review.
      * Terse assertions in test cases (single assert instead of many verbose `assert_line` and `refute_line` for every line of output).
      * Reusable golden files (declared once, used for many test cases).
      * Clear intention (same exact expected output) when asserted against same goldens in multiple test cases.
      * Can be clearly diff'd across multiple lines in failure message(s).
      * Easily updated.
    
    This PR adds support to bats for golden files with the `assert_equals_golden` and `assert_output_equals_golden` test helper functions.
    `assert_equals_golden` takes 2 arguments, <actual> and <golden file path>, to assert <actual> is the same as the contents in <golden file path>.
    `assert_output_equals_golden` takes 1 argument, <golden file path>, to assert `output` is the same as the contents in <golden file path>.
    
    These functions support a number of features, including `--regexp` and automatic updating of golden files (via setting the `BATS_ASSERT_UPDATE_GOLDENS_ON_FAILURE` environment variable). Golden file assertions will properly account for empty lines or trailing newlines (e.g. when asserting against output obtained by `run --keep-empty-lines`).
    
    This PR adds `assert_equals_golden.bats` which contains test cases for the newly added functionality.
    
    Note that the output of these functions (that is asserted on in tests) has incorrect "lines" count.
    This is due to an incompatibility between `run --keep-empty-lines` and how various bats-support helper functions work.
    See bats-core/bats-support#11.
    CauhxMilloy committed May 8, 2024
    Configuration menu
    Copy the full SHA
    f121674 View commit details
    Browse the repository at this point in the history

Commits on May 13, 2024

  1. * Fixing incorrect error message output when assert_equals_golden f…

    …ailed (using `$output` instead of passed string).
    
    * Updating `assert_equals_golden` tests in `test/assert_equals_golden.bats` to invalidate `output` and use another variable instead (to better catch any similar issues).
    CauhxMilloy committed May 13, 2024
    Configuration menu
    Copy the full SHA
    71e2c5a View commit details
    Browse the repository at this point in the history

Commits on May 14, 2024

  1. Configuration menu
    Copy the full SHA
    8916f2d View commit details
    Browse the repository at this point in the history

Commits on May 16, 2024

  1. * Adding assert_file_equals_golden for easier assertions on files.

    * Adding new test cases for `assert_file_equals_golden`.
    CauhxMilloy committed May 16, 2024
    Configuration menu
    Copy the full SHA
    8d530de View commit details
    Browse the repository at this point in the history

Commits on May 21, 2024

  1. * Escaping \ for regexp golden files.

    * Adding `\` to list of special characters in doc strings.
    * Adding test cases to cover all special characters with `BATS_ASSERT_UPDATE_GOLDENS_ON_FAILURE`.
    * Cleaning up some unnecessary lines in a few existing `assert_equals_golden` tests.
    CauhxMilloy committed May 21, 2024
    Configuration menu
    Copy the full SHA
    988e355 View commit details
    Browse the repository at this point in the history

Commits on May 28, 2024

  1. * Adding always showing output/golden contents/diff when test fails (…

    …even if the golden ends up being updated).
    
    * Adding prefix of golden file path before error messages.
    * Simplifying return status from assert functions.
    * Updating tests to assert output with new changes.
      * Adding helper function to save tmp file path used in test cases (to then be used in assertion messages).
      * Adding a space after `\` in some test cases to handle/workaround `\\\n` in docstrings not being parsed properly by bash.
    CauhxMilloy committed May 28, 2024
    Configuration menu
    Copy the full SHA
    9501659 View commit details
    Browse the repository at this point in the history