Improve diff reporter

[mindplay-dk]

In my opinion, the diff reporter really needs to work better for JS values and not just JSON.

I know you closed #121 as "won't fix", but diffs being constrained to comparisons for essentially JSON only is really problematic for me.

I repeatedly get into trouble - especially with dates, where fast-deep-equal finds a difference, but diff reporter can't display it.

I've been using this a lot lately:

https://www.npmjs.com/package/serialize-to-js

This doesn't solve the issue with prototypes etc. but that's not what I find myself running into most frequently.

It's things like Date, Set, Map, Buffer, RegExp and (circular) references - everyday things with no meaningful JSON representation.

It's very small, around ~300 LOC and no dependencies.

Unfortunately, it does not pretty print - it used to, but did so via another, much larger library.

It's maybe small enough to just own it though? Adding indentation and line-breaks, should be doable.

I might be willing to take a whack at it, if you're open to the idea?

[mindplay-dk]

I went over serialize-to-js, and while it is very small, it's also for the most part optimized for the machine/parser - not for the human reader. Making a human-readable version of this would require some leg work.

I've been poking around to see what else is available and found this:

https://github.com/facebook/jest/tree/main/packages/pretty-format

Granted, this has a larger footprint of around 20 KB - but it was designed for the human reader, and it covers a lot of ground, and is extensible via plugins. Most days, I'd be put off by the sheer size of this, but pretty printing in JS is clearly not a simple problem. It's a Facebook product, which is also off putting, but mostly for political reasons - from a community perspective, this being popular and well used is very positive, and it is MIT licensed.

I also poked through the internals of Mocha, Jasmine and Karma. The internals of Jasmine are deeply coupled to the framework and not available as a module. Mocha uses serialize-javascript, a "superset of JSON" that only supports a few additional types.

I also looked again at node-inspect-extracted, which I still think could work, although it does list some limitations with regards to Map and Set that I'm not crazy about.

I wouldn't take the option of forking serialize-to-js entirely off the table - having something small and simple is definitely always appealing to me, but I can also see that turning into an endless stream of feature requests if Zora takes off and becomes very popular.

Overall, I think pretty-format looks like a good option.

What do you think about opening the diff-reporter itself with an option to plug in a custom formatter?

Just something simple like (value: unknown) => string would enable us to experiment with different formatters by just plugging in a function. Might be a nice feature to have either way? And that's low hanging fruit, right? 🙂

[jwhitaker-swiftnav]

another option i came across is https://www.npmjs.com/package/browser-util-inspect - would seem plausible that zora could fork+embed this.

[mindplay-dk]

I've had pretty good results with pretty-format in my own reporter - it's been quite enjoyable to work with. I'm diffing the results in 5 different ways depending on the compared types:

It's not TAP, but could probably be adapted - and the dependencies are pretty big (around 50 KB) so maybe not in line with the philosophy of Zora, but feel free to borrow from this for your own reporter if it's helpful. 🙂

[lorenzofox3]

Very nice ! What is the protocol the reporter works on ? 50 KB for a reporter is ok as it remains decoupled from the testing library itself. In zora, reporters are "a la carte"

[mindplay-dk]

The protocol is this, so essentially just:

export type Fact = {
  label: string;
  pass: boolean;
  actual?: unknown;
  expected?: unknown;
  details: unknown[];
}

export type Check = {
  location: string;
  fact: Fact;
}

export type Result = {
  description: string;
  checks: Check[];
  time: number;
  error: Error | undefined;
}

It's divided into Fact and Check because those are produced at different times by different units.

My test-framework was heavily inspired by yours - I honestly didn't really want to create my own, but we had quite differing opinions about global state, if you recall those discussions? I eventually wanted to see, could I write a test-framework without any global state, comprised solely of functions - and this is the result. It started out as an exercise but somehow I ended up going deep on the reporting side, basically that's where I've spent most of my time. 😊

[lorenzofox3]

Don't worry. I take no offense: that's the beauty of open source.
I am glad I could somehow help you build a framework perfectly fitted for your needs.

I'll try to get a look if I can add your reporter to the list of options