Skip to content

Commit

Permalink
docs: update wording and examples in docs (#2)
Browse files Browse the repository at this point in the history
* docs: update docs
  • Loading branch information
Daniel-Aaron-Bloom authored May 3, 2024
1 parent 88bc4fe commit 7a743c9
Show file tree
Hide file tree
Showing 3 changed files with 158 additions and 54 deletions.
102 changes: 75 additions & 27 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,23 +1,33 @@
# tested-fixture
[![Crates.io](https://img.shields.io/crates/v/tested-fixture.svg)](https://crates.io/crates/tested-fixture)
[![Workflow Status](https://github.com/Daniel-Aaron-Bloom/tested-fixture/workflows/Rust/badge.svg)](https://github.com/Daniel-Aaron-Bloom/tested-fixture/actions?query=workflow%3A%22Rust%22)

# tested-fixture

Attribute macro for creating fixtures from tests

### Description

Sometimes a series of tests are progressive/incremental; One test is
targetted at verifying Step 1 works as expected, while another is focused
on ensuring Step 2 functions correctly. Common advice is to duplicate all
the work of Step 1 into Step 2's test, or to combine the tests into one
large test. However the former approach can significantly slow down tests,
while the latter can lead to large and unruly testing functions which are
difficult to maintain.

This crate takes a different approach by allowing a test to return a
fixture which can be used in subsequent tests, all through the use of a
single attribute macro [`tested_fixture`].
Sometimes a series of tests are progressive or incremental; that is to say
one test builds on another. A multi-stage test might have complicated
setup and verification processes for each step, but with clear boundaries
between stages (`test_1` verifies stage 1, `test_2` verifies stage 2, etc.
). The problem arises when stages want to share data (i.e. `test_2` wants
to start where `test_1` left off).

Common advice is to duplicate all the setup code across all tests, or
alternatively to combine the tests into one large test. However the former
approach can significantly slow down tests if setup is costly, and also
introduces significant test maintenance costs if setup procedures change.
The latter however can lead to large and unruly testing functions which are
difficult to maintain, and doesn't solve the problem when dependencies
cross multiple files (i.e. unit tests which test the full setup process for a
`Foo` are difficult to combine with unit tests which test the setup process
of a `Bar` which relies on a fully constructed `Foo`; should the "combined"
test live near `Foo` or `Bar`? What if the tests needs to access internals to
verify assertions?).

This crate provides an alternative approach by allowing a test to return a
fixture which can be used in subsequent tests. Tests can opt in to this
functionality by using a single attribute macro [`tested_fixture`].

### Usage

Expand All @@ -27,14 +37,24 @@ struct Foo {
// ...
}

struct State {
// ...
}

impl Foo {
fn step_1() -> Self {
Foo {
// Complicated setup...
}
}

fn step_2(&self) {
fn step_2(&self) -> State {
State {
// Complicated execution...
}
}

fn step_3(&self, v: &State) {
// Complicated execution...
}
}
Expand All @@ -56,10 +76,22 @@ fn step_2() {
foo.step_2();
// Complicated assertions verify step 2...
}

#[test]
fn step_3() {
let foo = Foo::step_1();
// (Some?) Complicated assertions verify step 1...

let state = foo.step_2();
// (Some?) Complicated assertions verify step 2...

foo.step_3(&state);
// Complicated assertions verify step 3...
}
```

As you can see, with a lot of steps, this can quickly get out of hand. To
clean it up is straightforward by switching `step_1` to use the
clean it up is straightforward by switching to use the
`tested_fixture` attribute instead of the normal `test`.

```rust
Expand All @@ -71,18 +103,25 @@ fn step_1() -> Foo {
foo
}

#[test]
fn step_2() {
STEP_1.step_2();
#[tested_fixture::tested_fixture(STEP_2_STATE)]
fn step_2() -> State {
let state = STEP_1.step_2();
// Complicated assertions verify step 2...
state
}

#[test]
fn step_3() {
STEP_1.step_3(&STEP_2_STATE);
// Complicated assertions verify step 3...
}
```

In the case where only `step_2` is run, `STEP_1` will be initialized on
Note that when only `step_2` is run, `STEP_1` will be initialized on
first access. Since the order of tests is not guaranteed, this actually can
occur even if both tests are run. But since results are cached, the
`step_1` test should still reproduce the same result regardless of if it is
the first access or not.
`step_1` test should still succeed (or fail) regardless of if it is run
first or not.

### Advanced usage

Expand All @@ -92,19 +131,28 @@ optional suffix can be used on tests returning a `Result` to specify that
only `Ok` return values should be captured. For example:

```rust
#[tested_fixture::tested_fixture(pub(crate) STEP_1: Foo)]
#[tested_fixture::tested_fixture(
/// Doc comment on the `STEP_1` global variable
pub(crate) STEP_1: Foo
)]
fn step_1() -> Result<Foo, &'static str> {
// ...
}
```

### Limitations

Ordinary test are able to return anything which implements
[`std::process::Termination`], which unlimited nestings of `Result`s. While
this crate does support returning nested `Result` wrappings, it only does
so up to a fixed depth. Additionally it does not support returning any other
types of `Termination`
Ordinary `#[test]` functions are able to return anything which implements
[`std::process::Termination`], including unlimited nestings of `Result`s.
While this crate does support returning nested `Result` wrappings, it only
does so up to a fixed depth. Additionally it does not support returning any
other `Termination` implementations besides `Result`.

As with all testing-related global state, it is recommended that tests don't
mutate the state, as doing so will increase the risk of flaky tests due to
changes in execution order or timing. Thankfully this is the default
behavior, as all fixtures defined by this crate are only accessible by
non-mutable reference.

Right now this crate does not support async tests.

Expand Down
3 changes: 1 addition & 2 deletions README.tpl
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
# {{crate}}
[![Crates.io](https://img.shields.io/crates/v/tested-fixture.svg)](https://crates.io/crates/tested-fixture)
{{badges}}

# {{crate}}

{{readme}}

## License
Expand Down
107 changes: 82 additions & 25 deletions src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,17 +2,28 @@
//!
//! ## Description
//!
//! Sometimes a series of tests are progressive/incremental; One test is
//! targetted at verifying Step 1 works as expected, while another is focused
//! on ensuring Step 2 functions correctly. Common advice is to duplicate all
//! the work of Step 1 into Step 2's test, or to combine the tests into one
//! large test. However the former approach can significantly slow down tests,
//! while the latter can lead to large and unruly testing functions which are
//! difficult to maintain.
//! Sometimes a series of tests are progressive or incremental; that is to say
//! one test builds on another. A multi-stage test might have complicated
//! setup and verification processes for each step, but with clear boundaries
//! between stages (`test_1` verifies stage 1, `test_2` verifies stage 2, etc.
//! ). The problem arises when stages want to share data (i.e. `test_2` wants
//! to start where `test_1` left off).
//!
//! This crate takes a different approach by allowing a test to return a
//! fixture which can be used in subsequent tests, all through the use of a
//! single attribute macro [`tested_fixture`].
//! Common advice is to duplicate all the setup code across all tests, or
//! alternatively to combine the tests into one large test. However the former
//! approach can significantly slow down tests if setup is costly, and also
//! introduces significant test maintenance costs if setup procedures change.
//! The latter however can lead to large and unruly testing functions which are
//! difficult to maintain, and doesn't solve the problem when dependencies
//! cross multiple files (i.e. unit tests which test the full setup process for a
//! `Foo` are difficult to combine with unit tests which test the setup process
//! of a `Bar` which relies on a fully constructed `Foo`; should the "combined"
//! test live near `Foo` or `Bar`? What if the tests needs to access internals to
//! verify assertions?).
//!
//! This crate provides an alternative approach by allowing a test to return a
//! fixture which can be used in subsequent tests. Tests can opt in to this
//! functionality by using a single attribute macro [`tested_fixture`].
//!
//! ## Usage
//!
Expand All @@ -22,14 +33,24 @@
//! // ...
//! }
//!
//! struct State {
//! // ...
//! }
//!
//! impl Foo {
//! fn step_1() -> Self {
//! Foo {
//! // Complicated setup...
//! }
//! }
//!
//! fn step_2(&self) {
//! fn step_2(&self) -> State {
//! State {
//! // Complicated execution...
//! }
//! }
//!
//! fn step_3(&self, v: &State) {
//! // Complicated execution...
//! }
//! }
Expand All @@ -51,10 +72,22 @@
//! foo.step_2();
//! // Complicated assertions verify step 2...
//! }
//!
//! #[test]
//! fn step_3() {
//! let foo = Foo::step_1();
//! // (Some?) Complicated assertions verify step 1...
//!
//! let state = foo.step_2();
//! // (Some?) Complicated assertions verify step 2...
//!
//! foo.step_3(&state);
//! // Complicated assertions verify step 3...
//! }
//! ```
//!
//! As you can see, with a lot of steps, this can quickly get out of hand. To
//! clean it up is straightforward by switching `step_1` to use the
//! clean it up is straightforward by switching to use the
//! `tested_fixture` attribute instead of the normal `test`.
//!
//! ```
Expand All @@ -66,18 +99,25 @@
//! foo
//! }
//!
//! #[test]
//! fn step_2() {
//! STEP_1.step_2();
//! #[tested_fixture::tested_fixture(STEP_2_STATE)]
//! fn step_2() -> State {
//! let state = STEP_1.step_2();
//! // Complicated assertions verify step 2...
//! state
//! }
//!
//! #[test]
//! fn step_3() {
//! STEP_1.step_3(&STEP_2_STATE);
//! // Complicated assertions verify step 3...
//! }
//! ```
//!
//! In the case where only `step_2` is run, `STEP_1` will be initialized on
//! Note that when only `step_2` is run, `STEP_1` will be initialized on
//! first access. Since the order of tests is not guaranteed, this actually can
//! occur even if both tests are run. But since results are cached, the
//! `step_1` test should still reproduce the same result regardless of if it is
//! the first access or not.
//! `step_1` test should still succeed (or fail) regardless of if it is run
//! first or not.
//!
//! ## Advanced usage
//!
Expand All @@ -87,23 +127,33 @@
//! only `Ok` return values should be captured. For example:
//!
//! ```
//! #[tested_fixture::tested_fixture(pub(crate) STEP_1: Foo)]
//! #[tested_fixture::tested_fixture(
//! /// Doc comment on the `STEP_1` global variable
//! pub(crate) STEP_1: Foo
//! )]
//! fn step_1() -> Result<Foo, &'static str> {
//! // ...
//! }
//! ```
//!
//! ## Limitations
//!
//! Ordinary test are able to return anything which implements
//! [`std::process::Termination`], which unlimited nestings of `Result`s. While
//! this crate does support returning nested `Result` wrappings, it only does
//! so up to a fixed depth. Additionally it does not support returning any other
//! types of `Termination`
//! Ordinary `#[test]` functions are able to return anything which implements
//! [`std::process::Termination`], including unlimited nestings of `Result`s.
//! While this crate does support returning nested `Result` wrappings, it only
//! does so up to a fixed depth. Additionally it does not support returning any
//! other `Termination` implementations besides `Result`.
//!
//! As with all testing-related global state, it is recommended that tests don't
//! mutate the state, as doing so will increase the risk of flaky tests due to
//! changes in execution order or timing. Thankfully this is the default
//! behavior, as all fixtures defined by this crate are only accessible by
//! non-mutable reference.
//!
//! Right now this crate does not support async tests.
#![warn(missing_docs)]
#![allow(clippy::test_attr_in_doctest)]

pub use tested_fixture_macros::tested_fixture;

Expand Down Expand Up @@ -173,7 +223,6 @@ pub mod helpers {
}
}


/// Helper trait for unwrapping fixtures
pub trait Unwrap<T>: Termination {
fn unwrap(self, context: &str) -> &'static T;
Expand Down Expand Up @@ -239,6 +288,14 @@ mod tests {
}
}

#[tested_fixture(
/// This is a test fixture
pub(crate) SETUP_0
)]
fn setup_with_attributes() -> HeavySetup {
HeavySetup::build(1)
}

#[tested_fixture(SETUP_1)]
fn setup() -> HeavySetup {
HeavySetup::build(1)
Expand Down

0 comments on commit 7a743c9

Please sign in to comment.