Add/define versioning to the benchmark

[rmahdav]

How do we like do versioning? Maybe based on Transformers.js and LireRT.js versions?
Opening this issue to discuss the options and make a decision about it.

[danleh]

Recording offline discussion (please correct if I missed anything):

• For now, let's keep the version at below 1.0 (currently 0.0.1-alpha, see package.json) to clarify that there are no stability guarantees, since workloads and metrics are still expected to change frequently in the next months.
• To allow for for quickly checking which code is running e.g. on a hosted variant of the benchmark, I added the git commit hash to the landing page of the benchmark in Add information to index page #95.
• One intuition or reason to have versions is to signal to users which scores can be compared with each other (namely that one can compare the scores of the same version with each other, even if there were some commits in between). Since we are not yet that stable (i.e. we expect almost every commit to potentially affect metrics), it's not worth it going through the trouble of incrementing versions.
• Once we are closer to some form of stability (or a release), we can discuss incrementing the version (major or minor?) for every substantial library/framework upgrade, or maybe with every added/removed workload, but let's defer that for now.

[rmahdav]

@dtig @brendandahl @danleh

Let's restart our discussion here. I think it's time to make a decision on our versioning strategy so we can land significant changes to the repository that could have a huge impact on benchmark performance.

Looking ahead, the main changes we expect to see in the repo fall into three categories:

1. Version bumps for transformers.js
2. Version bumps for litert.js
3. Adding or updating use cases and models

For the third category (adding or updating use cases/models), I don't think we need major versioning (we can consider minor bumps). However, for the first two (engine upgrades), we definitely need clear version bumps so we can compare performance across different engine versions.

Instead of doing formal releases for these bumps, what if we use branches for different snapshots?

• The main branch would always be the latest, bleeding-edge version.
• Older versions (tied to specific engine versions) would remain accessible via dedicated snapshot branches.

This approach gives us the version tracking we need without the overhead of a full release cycle. Let me know what you think!

[danleh]

(Disclaimer / general note: I am not against versioning per-se, I just want to keep the overhead at this relatively early stage as small as possible. E.g., not having to update a changelog file, releasing an artifact somewhere, or hosting older versions.)

Looking ahead, the main changes we expect to see in the repo fall into three categories: ...

Agreed, but I can imagine one further reason for a version bump: Fundamentally changing the scoring (e.g., such as adding sub iterations, which we did recently.)

for the first two (engine upgrades), we definitely need clear version bumps

Bumping the version on framework changes sounds good to me. However, at this point I would not bump the major version (i.e., from 0.0.1 to 1.0.0) simply because we are not at a "post 1.0" state yet. Once we have reached 1.0, bumping the major version sounds fine.

Where do we "mark" or track the version? Simplest to me seems the version field in the top-level package.json file

webai-compute-benchmark/package.json

Line 3 in 8c932cb

"version": "0.0.1",

That also has the benefit of being used for the benchmark title and score heading after #124