Skip to content

Releases: o19s/quepid

8.2.0 -- 2025-07-23 - Welcome quepidapp.com as the domain name!

23 Jul 19:28
@epugh epugh
v8.2.0
c61a478
This commit was signed with the committer’s verified signature.
epugh Eric Pugh
GPG key ID: E54EAD255F4A9678
Verified
Learn about vigilant mode.
Compare
Choose a tag to compare
8.2.0 -- 2025-07-23 - Welcome quepidapp.com as the domain name! Latest
Latest

What's Changed

  • The biggest thing folks may notice is that we've moved from the domain name quepid.com to quepidapp.com. This has required lots of updates in documentation, build tooling, and various links scattered around the Quepid codebase.

  • Lots of refinement to the Continuous Experimentation support released in 8.1.0. Quepid will run cases marked "Nightly" and store the score. It also stores the last FIVE snapshots including document fields for additional analysis. We are limiting Snapshots to 5 to make sure we don't destroy the database ;-), and could evaluate that later. When you load the case, we still rerun all the queries again, however there is some preliminary work about just loading the last snapshot for these nightly cases: #1323.

  • Lots of improvements around the LLM-as-a-Judge. Encrypted LLM keys in the database, nicer UI when testing out prompts, ability to evaluate images in the judging process.

  • Brand new approach to documenting APIs! OpenAPI 3.1 specification compliant docs, and you can test out API calls right there in Quepid, or get the curl equivalent. Thank you Apipie for your previous service, and I'm excited about the OasRails project. The lead developer, Andrés has been great to work with.

Features

  • Nightly Evaluation Support. #1288 and #1289.

  • You can now generate an API key for a user from the commandline: bundle exec thor user:add_api_key EMAIL. Thanks @frutik for the contribution in #1408

  • Nightly build and push to DockerHub of Quepid image. Thanks @frutik for #1369.

  • A new Scorer! Welcome NDCG_CUT@10! NDCG is properly calculated on all the documents that have been rated for a query. However, often we want to only look at the top 10, and cut off evaluation there, ignoring all the rest. That is when you want NDCG_CUT@10. If you had been using NDCG@10 then you may want to swap.

  • API to get the latest snapshot for a case. Supporting a custom evaluation frontend built by @sjarmak. Thanks @david-fisher for encouragement. #1312 by @epugh.

Improvements

  • LLM-as-a-Judge can now look at images as part of the evaluation! Thanks @khrabrovart for #1344 and for writing docs. Thanks [email protected] for making this happen.

  • When testing out LLM Judge prompt display a nice spinning icon since these can be long! #1395 by @epugh. Did some vibe coding!

  • Support Ollama based LLMs for judging. Thanks @frutik for the initial work in #1258 that resulted in #1275.

  • Nicer UI around importing books. Let's make getting data in and out easier. cf0673c and #1332 by @epugh. Thanks @atarora for design input on this.

  • Optimize the performance of editing books. 9605a60 by @epugh. Thanks [email protected].

Bugs Addressed

  • When comparing snapshots, if you had basic auth set up then those credentials were not passed in. Thanks @kiratraynor for finding this, fixed in #1411 by @epugh.

  • Judgements in a Book don't HAVE to have a user id, so fix the UI: 178594f by @epugh.

  • Fix Quepid API to support the Jupyter Inter Rater Reliablity notebook. #1294 by @epugh.

What's Changed

Read more

Contributors

epugh, frutik, and 9 other contributors
Assets 2
Loading

8.1.0 - Polished version of 8!

10 Mar 15:09
@epugh epugh
v8.1.0
fa0156b
This commit was signed with the committer’s verified signature.
epugh Eric Pugh
GPG key ID: E54EAD255F4A9678
Verified
Learn about vigilant mode.
Compare
Choose a tag to compare
8.1.0 - Polished version of 8!

8.1.0 -- 2024-02-28

We've had lots of folks trying new features of Quepid, and that has led to some nice polish.
This is the release for people who don't want the bleeding edge version ;-).

What's Changed

  • Special characters like $ and @ in Basic Auth passwords are now supported. Thanks to @atarora for opening #1245 and the debugging work from @david-fisher.
  • UX Work.
  • We have added filtering and pagination to all of the various screens like Seach Endpoints, Communal Scorers, Users, Judgements etc. This makes these pages load MUCH faster and you can search for the data you need.
  • Consistency across web pages. Outside of the core AngularJS app, we had a mish mash of UI's. Now they all have the same style header, footer, and UX treatment using card layouts.
  • Fixed the nDCG calculation to look at ALL ratings, not just the top k when doing calculation.
  • Fixed a goof where the new ERR scorer wasn't added as a communal scorer for everyone.
  • LLM as a Judge is better. Cleaned up the refine UI for prompt development.
  • Fix a bug where the Judge couldn't be launched from the UI.

Details

Full Changelog: v8.0.1...v8.1.0

Contributors

epugh, david-fisher, and 3 other contributors
Loading

8.0.1 means now you can deploy Quepid 8!

19 Feb 20:09
@epugh epugh
v8.0.1
4931635
This commit was signed with the committer’s verified signature.
epugh Eric Pugh
GPG key ID: E54EAD255F4A9678
Verified
Learn about vigilant mode.
Compare
Choose a tag to compare
8.0.1 means now you can deploy Quepid 8!

8.0.1 -- 2024-02-19

Whelp, that didn't take long!

We broke support for running Quepid on non standard ports like :3000. #1229 by @epugh fixes this.

What's Changed

Full Changelog: v8.0.0...v8.0.1

Contributors

epugh and dependabot
Loading

8.0.0 Release - Valentine's Day Edition 💘

14 Feb 20:02
@david-fisher david-fisher
v8.0.0
38119f8
This commit was signed with the committer’s verified signature.
epugh Eric Pugh
GPG key ID: E54EAD255F4A9678
Verified
Learn about vigilant mode.
Compare
Choose a tag to compare
8.0.0 Release - Valentine's Day Edition 💘

8.0.0 -- 2024-02-14

It's Valentine's Day 💘, so it seems appropriate to release the next major version of Quepid, the tool to give your queries some ❤️.

Exciting News

Judge Judy is now appearing on the Quepid channel. It is now possible to create one or more LLM-based AI judges in Quepid! Set your OpenAI key, add your examples or customizations to the judgement prompt, push the big button, and let the magic box do the rest.

Continuous Experimentation is now a reality in Quepid. You can now schedule automatic evaluation runs on a case on a daily basis. It is now possible to see the most up to date performance results, and track changes over time without loading up Quepid! You can even configure alerts to be emailed to you if your Case score drops!

Scaling Up. Quepid now supports many thousands of Queries in a single Case! Our target is 5000 queries in a single Case, and by adopting background processing and async feedback in the user interface, we can now scale up to handle this. Oh, and query performance has been drastically improved. One long time user with 1000 queries reports that loading the case has dropped from 20 minutes to three minutes ;-). Cases with a large number of queries can have problems completing, due to timeouts or 429 (too busy, try again) errors. Thanks to @mkr the concurrency control for handling query execution has been updated to prevent Quepid hanging up forever waiting for a query to complete.

Expected Reciprocal Rank (ERR) is now available as a community scorer. This scorer is appropriate for multi-valued judgment ranges when evaluating search journeys like known item search.

New User Manual. We've embraced the Diátaxis framework for writing documentation, and consolidated into a single platform the mix of documentation that has been published in various locations.

Bugs Addressed

If you don't have P@10 defined, the migrations blow up. Thanks @frutik for finding the bug. #1093

Basic auth that includes '%' in username or password breaks reloading from http to https. Thanks @brucks24 for #1139

Emoji filled queries like kfc 🍟➕🍔➕🍗 now can be stored in Quepid! Thanks @shuttie for reporting this bug. #1046

Other Changes

  • Simpler Deployment. We've updated to Rails 8, and no longer need Redis. We're back to just a server and a MySQL Database ;-).
  • Nicer import book page. Gave it some UX love. Kudos to @wrigleyDan
  • We use the built into Rails health check end point at /healthcheck. Please check your setup.
  • Change tracking of scores to record which scorer produced the score. This is a breaking change, as previous versions don't know what scorer generated a score. For historical scores, the current case scorer is assumed to be the only scorer used on that case. Soon you will be able to run multiple scores and pick one as your "North Star" metric.
  • We thought having really fine grained permissions would be important to Quepid, but what we've seen is that it doesn't matter. Indeed, if anything, the permissions structure is confusing to users. Additionally, we've mostly cared "are you an Admin?" to decide if certain things can be done or not. So it's gone.
  • We care about privacy. We've dropped Google Analytics in favour of our own first party analytics. We've also dropped the Intercom integration.

What's Changed

  • undo the "fit" that made us only run a single test by @epugh in #1076
  • Updates oct 2024 by @epugh in #1071
  • Add check for the existence of scorer before updating. by @epugh in #1093
  • Migration to Rails 8 by @epugh in #1084
  • Rails 8 is out, and we drag along a swap from annotate to annotaterb by @epugh in #1098
  • Strip out SASS by @epugh in #1113
  • swap to released version of Bullet by @epugh in #1102
  • Tweaks to text. by @epugh in #1119
  • Hard-limit search concurrency to 10 by @mkr in #1103
  • Restore proper support for notifying users about Cookies being required policy by @epugh in #1127
  • Update jshint structure by @epugh in #1135
  • Be smart about if we have no score yet, and share that info. by @epugh in #1132
  • ship a first cut of being smarter by @epugh in #1133
  • Frogs forever by @epugh in #1136
  • Change the close icon to something that renders larger and more visible by @epugh in #1140
  • Warn on incompatible characters in basicauth password by @epugh in #1143
  • Wire in deleting the case when you abandon it. by @epugh in #1144
  • First party analytics by @epugh in #1100
  • Fix up the routing logic to keep you in the rating experience. by @epugh in #1145
  • Perf optimizations. (again!) by @epugh in #1148
  • On to 3.3.6 of Ruby by @epugh in #1157
  • Add err scorer by @david-fisher in #1162
  • Backout counter_cache by @epugh in #1165
  • Review database structures by @epugh in #1167
  • Improve links/buttons when creating a Book by @wrigleyDan in #1175
  • Meet Judge Judy, she is your AI powered SME by @epugh in #985
  • Missing indexes on Search Endpoints by @epugh in #1173
  • Spike 5000 queries round deux by @epugh in #1039
  • Remove permissions architecture in favour of roles by @epugh in #1185
  • Restore the proper encoding, lost somewhere by @epugh in #1188
  • Look at database consistency issues by @epugh in #1189
  • Testing the flow of everything.... by @epugh in #1192
  • fix RRE export format by @epugh in #929
  • Properly nest returning judgements for a book under the book route by @epugh in #1187
  • OpenAPI query limits being hit by @epugh in #1194
  • Can we use Blazer to monitor users? by @epugh in #1205
  • FInally remove intercom by @epugh in #1206
  • Validate search endpoints by @epugh in #1209
  • Simplify book ux by @epugh in #1200
  • Need to track which scorer provided the score! by @epugh in #1210
  • Manage all Case scores, and let you delete the werid scores. by @epugh in #1213
  • Fancier handling of scores and ratings and judgements and fix search endpionts and all the other things that make it hard for @david-fisher to write changelog notes. by @epugh in #1215
  • nicer filtering makes loading books faster... by @epugh in #1216
  • Document why updating a rating doesn't update the specifics in a query_doc_pair by @epugh in #1219
  • Remove owner concept by @wrigleyDan in #1181
  • Strip out remaining references to team ownership. by @epugh in #1221
  • Testing of Quepid8 Functions by @epugh in #1222
  • Revamp Query.query_text to handle emoji better by @epugh in #1223

Full Changelog: v7.18.0...v8.0.0

Contributors

epugh, mkr, and 5 other contributors
Loading

v7.18.1

07 Dec 02:24
@epugh epugh
v7.18.1
34e9271
This commit was signed with the committer’s verified signature.
epugh Eric Pugh
GPG key ID: E54EAD255F4A9678
Verified
Learn about vigilant mode.
Compare
Choose a tag to compare
v7.18.1

7.18.1 - 2024-12-06

We've been hard at work on Quepid8, however we have two items that demand a point release:

  • Bug fix when looking up potential Querqy rule use and we hadn't finished running the query. cb9e3f4 by @epugh.

  • Prevent run away querying! A user of Quepid has a proxy between Quepid and the search engine, and we were overloading it. 7cb1f55 by @mkr!

Full Changelog: v7.18.0...v7.18.1

Contributors

epugh and mkr
Loading

7.18.0

04 Oct 19:43
@epugh epugh
v7.18.0
2d4f3c0
This commit was signed with the committer’s verified signature.
epugh Eric Pugh
GPG key ID: E54EAD255F4A9678
Verified
Learn about vigilant mode.
Compare
Choose a tag to compare
7.18.0

7.18.0 - 2024-10-04

Wow! It's been three months since the last release of Quepid, so it's about time.
This release brings Quepid up to Rails 7.2 standard, which gets us ready for some of the interesting things in Rails 8, specifically simplifying what you need to run Quepid.
There is a good path to removing the dependency on Redis, and maybe even MySQL for small setups!

Improvements

  • Shrink the Quepid Docker image by 1 GB. This was accomplished by the migration to Rails 7.2 for Quepid in #1058, fixing #1004 by @epugh.

  • Expanded Quepid's API for working with Search Endpoints. Thanks @clintbxtreme for opening #1056, and this was fixed in #1057 by @epugh.

  • Using SMUI and want to see if your rule was triggered? Now you can, we have a Querqy icon! Previously you would have to dig into the query parsing explanation to know. #1050 by @clamar-VM was fixed by #1053 by @epugh.

Bugs

  • When rating images that have a prefix defined, the prefix was being lost. Thanks @clamar-VM for finding this, fixed in @#1054 by @epugh.

  • We got too strict with our cookies, and Quepid quit working with Keycloak. Thanks @drieschel for #1049.

What's Changed

  • So long docker-compose. Welcome docker compose. by @epugh in #1052
  • Fix login and redirect problem with Keycloak by @drieschel in #1049
  • introduce a icon letting you know querqy rule matched a query by @epugh in #1053
  • adding the prefix when setting up links by @epugh in #1054
  • Update nginx Docker tag to v1.27.1 by @renovate in #1045
  • Update dependency terser to v1.2.3 by @renovate in #1025
  • Extend search endpoints controller to allow delete, complete update, … by @epugh in #1057
  • Big combo! Rail 7.2 Dockerfile used for Dockerfile.prod, plus bump R… by @epugh in #1058

New Contributors

Full Changelog: v7.17.1...v7.18.0

Contributors

epugh, drieschel, and 3 other contributors
Loading

7.17.1

26 Jun 21:07
@epugh epugh
v7.17.1
bc9ce71
Compare
Choose a tag to compare
7.17.1

7.17.1 - 2024-06-26

Lots of small things ;-).

Improvements

  • Proxied connections can now handle non ASCII characters like café. #1043 by @epugh. Thanks @miguelmesas for opening up the ticket that tipped me off to this.

  • p@10 scorer now is smarter to handle non binary judgements. #991 by @david-fisher. Sorry this took so long to get merged!

  • RAG style queries are MUCH longer. @miguelmesas opened up which is fixed by #1042 and #1037 by @epugh.

  • Improve the bulk query API to support 1000's of queries. #1037 by @epugh.

  • Small tweaks to the layout of the Search Endpoints to move the Destroy action away from the edit action. Thanks @OkkeKlein for reporting #900.

Bugs

  • Resetting skipped and can't judge query/doc pairs is fixed. Thanks @pfries for opening #1040. Quite a few other Turbo Drive related navigation issues fixed.

Contributors

epugh, pfries, and 3 other contributors
Loading

7.17.0 - Breaking the tyranny of the web request/response cycle

14 Jun 16:35
@epugh epugh
v7.17.0.0
5c9384a
Compare
Choose a tag to compare
7.17.0 - Breaking the tyranny of the web request/response cycle

7.17. - 2024-06-14

This is a big one! We have broken the tyranny of the web request/response lifecycle by embracing some powerful Rails technologies:

  • ActionCable: We now support two way communication from the backend server to your front end browser via Web Sockets. This lets us notify not just the person who starts a long running process of updates, but also any OTHER user who is interested in a specific topic. For example, if someone uploads a Book with many thousands of judgements, then everyone who is part of that team can see the progress for them selves.

  • ActiveJob: This let's us run background jobs. Historically we only used it for tracking of events (like you visited a Case), however now we are using it for processing large data sets in the background.

  • ActiveStorage: We need to better handle LARGE datasets in Quepid. ActiveStorage lets us build massive export files and store them in the database as a binary until you decide to download the data. Likewise, importing large data files that take time to process is now supported by being able to store them in Quepid.

  • Hotwire: HTML over the Wire lets us render HTML on the backend using our familiar Rails MVC, but magically stream the updates to the front end. This lets us add interactivity without requiring us to build out a Single Page Javascript application.

These changes are going to enable a future for Quepid that let's it scale up signficantly and be a richer environment for measuring relevance and working with search!

Okay, now on to our more detailed list of changes:

Features

  • You can now run your Query Evaluations OUTSIDE of Quepid, and store the scores INSIDE Quepid. This gives you a place to share this information with folks in your team. See the example Python scripting here: https://github.com/o19s/quepid/tree/main/docs/examples/external_eval. We also enhanced the API documentation. #1034 by @epugh.

  • HTML Over the Wire! Now we only load HTML that has changed, which improves page percieved performance. Added new export and import of Books as background jobs with notifications to users over Web Sockets. You then can download the resulting file without waiting for the data processing. Loading aspects of the home page asyncrhonsly so the page loads super fast, and then content renders in as available. #992 by @epugh.

Improvements

  • Update to Rails 7.1 standards. #1033 by @epugh.

  • Nicer Admin Homepage visual layout. #1031 by @epugh.

  • Revamped performance of the Home page to not recalculate the case summaries if you've already cached them in the browser. Much faster page! #1022 and #1029 by @epugh.

  • New Case Wizard is smarter about existing Search Endpoints pane. #1012 by @epugh.

  • Lots of updates to all the dependencies!

Bugs

  • Swapping between HTTP and HTTPS end points would cause issues with your Search Endpoint configuration. #1035 by @epugh fixes #1005.

  • We state that judging a query doc pair only requires three jdugements, but didn't actually make that work. #1032 by @epugh and @wrigleyDan fixes #1019 by @wrigleyDan.

  • Fixed utf encoding in mysql for query_doc_pairs. #1017 by @epugh fixes #1013 by @wrigleyDan.

  • Pass basic auth credentials through when you view a specific document or use Quepid as a proxy to your search engine. #1015 by @epugh fixes #1014 by @david-fisher.

Contributors

epugh, david-fisher, and wrigleyDan
Loading

v7.16.1

09 Apr 19:54
@epugh epugh
v7.16.1
45abb9d
Compare
Choose a tag to compare
v7.16.1

7.16.1 - 2024-04-09

Bugs

  • You can delete a user who has existing judgements, and then that messes up the main Books page because their judgements hang around. Now you are warned about this, and you can choose to make their judgements anonymous. (or just Lock their user account). #999 by @epugh.

Full Changelog: v7.16.0...v7.16.1

Contributors

epugh
Loading

7.16.0

28 Mar 20:49
@epugh epugh
v7.16.0
95658c2
Compare
Choose a tag to compare
7.16.0

7.16.0 - 2024-03-28

Features

  • Now have a dropdown for Books similar to Cases that leverages our existing tracking of what books you have viewed. #986 by @epugh.

  • We now run scorers over queries that have no ratings and queries that are ZSR. This lets us have smarter logic about how you want to score those situations. #993 by @epugh.

Improvements

  • One less query to determine what a user has access to via being an owner or a member of a team while ensuring no duplicate cases/books show up. #982 by @epugh with input from @reid-rigo.

  • Remove extranous faraday logs when running tests. #983 by @epugh with input form @reid-rigo fixes #964.

  • Nicer onboarding using TMDB dataset for Algolia users. #987 by @chuckmeyer.

Bugs

  • Book Importing was broken. Plus now we have nicer formatted error message when you validate a book to import. #989 by @epugh.

Full Changelog: v7.15.1...v7.16.0

Contributors

epugh, reid-rigo, and chuckmeyer
Loading