-
Notifications
You must be signed in to change notification settings - Fork 10.6k
Add “Selecting a Model Based on Stripe Conversion” notebook #1873
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
@shyamal-anadkat @Lupie here is the PR for your review! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
Adds a new notebook demonstrating how to choose an LLM based on real Stripe payment conversion, plus configuration entries to surface it in the cookbook site.
- Registers the new notebook in
registry.yaml
- Introduces the notebook file under
examples/stripe_model_eval/
- Adds authors metadata in
authors.yaml
Reviewed Changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.
File | Description |
---|---|
registry.yaml | New entry for “Selecting a Model Based on Stripe Conversion” |
examples/stripe_model_eval/selecting_a_model_based_on_stripe_conversion.ipynb | Full notebook content for the new guide |
authors.yaml | Added YAML entries for two authors (third still missing) |
Comments suppressed due to low confidence (3)
registry.yaml:2111
- The author
shyamal-anadkat
is listed here but not defined inauthors.yaml
. Add a matching entry inauthors.yaml
to avoid broken author links.
- - shyamal-anadkat
examples/stripe_model_eval/selecting_a_model_based_on_stripe_conversion.ipynb:152
- The notebook references
diagram.svg
but that file isn’t included in this PR. Please add the SVG or update the path so the diagram displays correctly.
+ "<img src=\"diagram.svg\" width=\"600\">"
registry.yaml:2105
- The indentation for the new list entry doesn’t match the surrounding items (which use leading spaces before the dash). Align this entry to the same indent level so it’s recognized correctly.
+- title: Selecting a Model Based on Stripe Conversion – A Practical Eval for Startups
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm! thanks
Great, thanks! |
Summary
Adds a new notebook “Selecting a Model Based on Stripe Conversion: A Practical Eval for Startups.”
The post walks through HyperWrite’s real-world A/B-test workflow for choosing an LLM using Stripe payment conversion as the success metric.
Motivation
Benchmark scores may fail to predict whether users will pay.
This guide shows startups how to evaluate and pick a cheaper or higher-performing model based on revenue-linked conversion data, not offline metrics. It fills a gap in the cookbook by providing:
The pattern is directly applicable to any OpenAI-powered app that charges through Stripe.
For new content
When contributing new content, read through our contribution guidelines, and mark the following action items as completed: