Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions ci/local/polis-api.sh
Original file line number Diff line number Diff line change
Expand Up @@ -41,12 +41,31 @@ check_project "$IMPORTANCE_PROJECT"
# Keep aggregate test pools below PostgreSQL's default connection limit.
# Pin this here so local and hosted checks use the same database budget.
export RUST_TEST_THREADS=2
# The first pass runs on test databases as PostgreSQL creates them. An
# interrupted earlier run may have left the second pass's shape installed.
cargo run --locked -p polis-api --example older_database_shape -- remove
cargo test --locked -p polis-api --features db-tests
# Compare timely replies with full selections from the same run. Execute
# the optimized queue test in both local and hosted CI.
cargo test --release --locked -p polis-api --features db-tests sustained_votes_keep_delivering -- --nocapture
)

(
cd "$CHECK_ROOT/queue-rs"
export POLIS_API_TEST_DATABASE_URL
export POLIS_AUTH_TEST_DATABASE_URL
export RUST_TEST_THREADS=2
# The second pass: the same tests, unchanged, on test databases in the shape
# of an installation whose database predates the migrations (json math
# documents, another column order, a few other columns). While
# polis-api/tests/older_database_shape.sql is installed in the test
# database and in template1, the tables it lists take that shape as the
# tests create them. It is removed again whatever the tests answer.
trap 'cargo run --locked -p polis-api --example older_database_shape -- remove' EXIT
cargo run --locked -p polis-api --example older_database_shape -- install
POLIS_API_TEST_DATABASE_SHAPE=older cargo test --locked -p polis-api --features db-tests
)

(
cd "$CHECK_ROOT/queue-rs"
cargo build --locked --release -p polis-api
Expand Down
44 changes: 44 additions & 0 deletions docs/local-checks.md
Original file line number Diff line number Diff line change
Expand Up @@ -115,6 +115,50 @@ These are sampled peaks, not upper bounds. At a shell open-file limit of 256,
default parallelism also fails tests with "Too many open files"; two threads pass. Two threads leave headroom without requiring a larger test
database; test selection, assertions and concurrency within each test are unchanged.

The API entry point runs its full Rust database test suite twice. The first
pass uses test databases as PostgreSQL creates them, so every table a test
builds has the shape the migrations give it. The second pass runs the same
tests, unchanged, on the shape of an installation whose database predates the
migrations: such a database stores the math documents as `json` where the
migrations build `jsonb`, keeps some columns in another order, has a few
columns the migrations never create and lacks two they do. A statement can
depend on any of that without saying so (a function that exists for `jsonb`
only, a `SELECT *` read by position), and only a database in that shape shows
it. `queue-rs/polis-api/tests/older_database_shape.sql` records the shape of
the thirteen tables that differ and that the API reads, and reshapes a table
with one of those names when it is created; its header says exactly what is
and is not reproduced. Between the passes the entry point installs it in the
test database and in that server's `template1` (every database a test creates
is a copy of `template1`), sets `POLIS_API_TEST_DATABASE_SHAPE=older` for the
second pass, and removes it afterwards, also when the tests fail. Installing
needs a superuser, and `template1` is shared by the whole server, so
`POLIS_API_TEST_DATABASE_URL` must name a server that belongs to this run. One
test in each pass checks that its databases have the shape the pass names, so
a shape left installed by an interrupted run fails the first pass instead of
changing it silently; the first pass also begins by removing it. To run one
pass by hand:

```sh
cd queue-rs
cargo run --locked -p polis-api --example older_database_shape -- install
POLIS_API_TEST_DATABASE_SHAPE=older RUST_TEST_THREADS=2 \
cargo test --locked -p polis-api --features db-tests
cargo run --locked -p polis-api --example older_database_shape -- remove
```

A test that needs a table in one particular column order, in both passes,
creates it under another name and renames it; a test that loads a table by
position declares fewer columns than the migrations' table or names the
columns it inserts. The release-mode queue test runs in the first pass only.

A migration that adds a column to one of the thirteen tables also adds it to
`older_database_shape.sql`, as the last row of that table and marked `later`
(the file's header says why). Until it does,
`every_migration_applies_and_the_listed_tables_take_the_older_shape` fails in
both passes and says so. Most tables the tests declare have fewer columns than
the migrations' tables: in the second pass those keep the order the test gave
them, and only their `jsonb` math documents become `json`.

CLI differential needs
its `CLI_DIFF_*_IMAGE` variables and an independent `SETUP_ADMISSION_PROJECT`.
Mixed identity uses all six `MIXED_*_PORT` / `POLIS_MIXED_WORKER_PORT` values.
Expand Down
3 changes: 3 additions & 0 deletions queue-rs/polis-api/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -225,6 +225,9 @@ cargo clippy --locked -p polis-api --all-targets -- -D warnings
cargo test --locked -p polis-api
POLIS_API_TEST_DATABASE_URL=postgresql://postgres@127.0.0.1:25462/scratch \
cargo test --locked -p polis-api --features db-tests # a throwaway Postgres
# `make check-polis-api` runs these database tests a second time on the shape
# of an installation whose database predates the migrations
# (tests/older_database_shape.sql; see docs/local-checks.md).
cd ..
queue-rs/polis-api/conformance/nginx-routing.sh # needs Docker
# Byte-for-byte replay (needs Docker and the local characterization images;
Expand Down
46 changes: 46 additions & 0 deletions queue-rs/polis-api/examples/older_database_shape.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
//! Installs or removes `tests/older_database_shape.sql` on the test server
//! named by `POLIS_API_TEST_DATABASE_URL`: in that database, where tests
//! create their schemas, and in `template1`, which every database a test
//! creates is copied from. While it is installed, every table the crate's
//! database tests create under one of the names that file lists comes out in
//! the shape of an installation whose database predates the migrations.
//!
//! cargo run --locked -p polis-api --example older_database_shape -- install
//! cargo run --locked -p polis-api --example older_database_shape -- remove
//!
//! `template1` is shared by the whole server, so use this only on a server
//! that belongs to the test run. `remove` is safe to run when nothing is
//! installed.
use anyhow::{Context, Result, bail};
use postgres::{Client, Config, NoTls};

const INSTALL: &str = include_str!("../tests/older_database_shape.sql");
const REMOVE: &str = "DROP EVENT TRIGGER IF EXISTS older_database_shape;
DROP SCHEMA IF EXISTS older_database_shape CASCADE;";

fn main() -> Result<()> {
let action = std::env::args().nth(1).unwrap_or_default();
let sql = match action.as_str() {
"install" => INSTALL,
"remove" => REMOVE,
_ => bail!("usage: older_database_shape install|remove"),
};
let url = std::env::var("POLIS_API_TEST_DATABASE_URL")
.context("POLIS_API_TEST_DATABASE_URL names the test database")?;
let tests: Config = url
.parse()
.context("POLIS_API_TEST_DATABASE_URL is not a PostgreSQL URL")?;
let mut template = tests.clone();
template.dbname("template1");
for (name, config) in [("the test database", tests), ("template1", template)] {
// Quiet the notices of DROP ... IF EXISTS.
let mut client: Client = config
.connect(NoTls)
.with_context(|| format!("connecting to {name}"))?;
client
.batch_execute(&format!("SET client_min_messages = warning; {sql}"))
.with_context(|| format!("older database shape: {action} in {name}"))?;
}
println!("older database shape: {action} done in the test database and in template1");
Ok(())
}
3 changes: 3 additions & 0 deletions queue-rs/polis-api/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2070,3 +2070,6 @@ mod health_tests {

#[cfg(all(test, feature = "db-tests"))]
mod startup_wiring_tests;

#[cfg(all(test, feature = "db-tests"))]
mod older_database_shape_tests;
22 changes: 17 additions & 5 deletions queue-rs/polis-api/src/math_bundle.rs
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,15 @@ impl Cache {
}
}
pub fn read(c: &mut postgres::Client, zid: i32, env: &str) -> anyhow::Result<Option<Read>> {
let rows=c.query("SELECT m.math_tick,b.math_tick,p.math_tick,t.math_tick,CASE WHEN octet_length(m.data::text)<=65536 THEN m.data::text END,CASE WHEN octet_length(b.data::text)<=65536 THEN b.data::text END,p.data IS NOT NULL AND p.data<>'null'::jsonb FROM math_main m LEFT JOIN math_bidtopid b ON b.zid=m.zid AND b.math_env=m.math_env LEFT JOIN math_ptptstats p ON p.zid=m.zid AND p.math_env=m.math_env LEFT JOIN math_ticks t ON t.zid=m.zid AND t.math_env=m.math_env WHERE m.zid=$1 AND m.math_env=$2 LIMIT 2",&[&zid,&env])?;
// The three `data` columns are jsonb in a database built from the
// migrations and json in installations older than them. `::text` exists
// for both. Whether the participant statistics are a JSON null is asked
// of the column's own type: only that arm of the CASE is evaluated, and
// its cast is to the type the column already has, so no document is
// converted. On jsonb this is the comparison with 'null'::jsonb it
// replaces. A column of another type gives SQL NULL here for a stored
// document, and the read is refused.
let rows=c.query("SELECT m.math_tick,b.math_tick,p.math_tick,t.math_tick,CASE WHEN octet_length(m.data::text)<=65536 THEN m.data::text END,CASE WHEN octet_length(b.data::text)<=65536 THEN b.data::text END,p.data IS NOT NULL AND CASE pg_typeof(p.data) WHEN 'pg_catalog.jsonb'::regtype THEN jsonb_typeof(p.data::jsonb) WHEN 'pg_catalog.json'::regtype THEN json_typeof(p.data::json) END<>'null' FROM math_main m LEFT JOIN math_bidtopid b ON b.zid=m.zid AND b.math_env=m.math_env LEFT JOIN math_ptptstats p ON p.zid=m.zid AND p.math_env=m.math_env LEFT JOIN math_ticks t ON t.zid=m.zid AND t.math_env=m.math_env WHERE m.zid=$1 AND m.math_env=$2 LIMIT 2",&[&zid,&env])?;
if rows.len() > 1 {
return Ok(None);
}
Expand All @@ -104,10 +112,14 @@ pub fn read(c: &mut postgres::Client, zid: i32, env: &str) -> anyhow::Result<Opt
};
let mapping = r.try_get::<_, Option<String>>(5)?;
let matching = (1..=3).all(|i| r.try_get::<_, Option<i64>>(i).ok() == Some(Some(tick)));
if !matching
|| mapping.as_deref() == Some("null")
|| !r.try_get::<_, Option<bool>>(6)?.unwrap_or(false)
{
// SQL NULL only for statistics stored in a column that is neither jsonb
// nor json: the expression is false, never NULL, without a document.
let Some(statistics) = r.try_get::<_, Option<bool>>(6)? else {
return Ok(None);
};
// A json column keeps a document's text as written, so its null may be
// surrounded by white space; a jsonb column always gives `null`.
if !matching || mapping.as_deref().map(str::trim) == Some("null") || !statistics {
return Ok(Some(Read::Incoherent(tick)));
}
let Some(mapping) = mapping else {
Expand Down
188 changes: 188 additions & 0 deletions queue-rs/polis-api/src/math_membership_tests.rs
Original file line number Diff line number Diff line change
Expand Up @@ -211,3 +211,191 @@ async fn wait_for_generation(
)
})
}

/// The three math documents are jsonb in a database built from the migrations
/// and json in an installation whose database predates them. The whole
/// generation read gives the same answer on both, and refuses statistics kept
/// in a column of any other type.
#[test]
fn json_and_jsonb_math_columns_give_the_same_generation() {
let url = std::env::var("POLIS_API_TEST_DATABASE_URL").unwrap();
let mut c = postgres::Client::connect(&url, postgres::NoTls).unwrap();
let schema = format!("math_bundle_types_{}", std::process::id());
c.batch_execute(&format!(
"CREATE SCHEMA {schema}; SET search_path={schema};
CREATE TABLE math_main(zid integer,math_env text,data text,math_tick bigint);
CREATE TABLE math_bidtopid(zid integer,math_env text,data text,math_tick bigint);
CREATE TABLE math_ptptstats(zid integer,math_env text,data text,math_tick bigint);
CREATE TABLE math_ticks(zid integer,math_env text,math_tick bigint);"
))
.unwrap();
let main = r#"{"base-clusters":{"id":[42,99]},"group-clusters":[{"members":[42]}],"n":2}"#;
let spaced = "{ \"group-clusters\" : [ { \"members\" : [ 99 ] } ] ,\n\t\"base-clusters\" : { \"id\" : [ 42 , 99 ] } }";
// (zid, main, mapping, statistics, ticks of mapping, statistics, math_ticks)
type Case = (
i32,
&'static str,
Option<&'static str>,
Option<&'static str>,
[i64; 3],
);
let cases: Vec<Case> = vec![
(
1,
main,
Some(r#"{"bidToPid":[[0],[2]]}"#),
Some("{}"),
[7; 3],
),
(
2,
spaced,
Some("{ \"bidToPid\" : [ [ 0 , 1 ] , [ 2 ] ] }"),
Some(" { \"a\" : 1 } "),
[7; 3],
),
(
3,
main,
Some(r#"{"bidToPid":[[0],[2]]}"#),
Some("null"),
[7; 3],
),
(
4,
main,
Some(r#"{"bidToPid":[[0],[2]]}"#),
Some(" null\n"),
[7; 3],
),
(5, main, Some(r#"{"bidToPid":[[0],[2]]}"#), None, [7; 3]),
(6, main, Some("null"), Some("{}"), [7; 3]),
(7, main, Some("\tnull "), Some("{}"), [7; 3]),
(8, main, None, Some("{}"), [7; 3]),
(
9,
main,
Some(r#"{"bidToPid":[[0],[2]]}"#),
Some("[1]"),
[7, 6, 7],
),
(
10,
main,
Some(r#"{"bidToPid":[[0],[2]]}"#),
Some("0"),
[7; 3],
),
(
11,
main,
Some(r#"{"bidToPid":[[0],[2]]}"#),
Some("\"null\""),
[7; 3],
),
(
12,
"[1]",
Some(r#"{"bidToPid":[[0],[2]]}"#),
Some("false"),
[7; 3],
),
];
let mut reads: Vec<Vec<String>> = vec![];
for data_type in ["jsonb", "json"] {
// The types are set here, whatever shape the test database gives a
// newly created table.
c.batch_execute(&format!(
"TRUNCATE math_main, math_bidtopid, math_ptptstats, math_ticks;
ALTER TABLE math_main ALTER COLUMN data TYPE {data_type} USING data::{data_type};
ALTER TABLE math_bidtopid ALTER COLUMN data TYPE {data_type} USING data::{data_type};
ALTER TABLE math_ptptstats ALTER COLUMN data TYPE {data_type} USING data::{data_type};"
))
.unwrap();
for (zid, main, mapping, statistics, ticks) in &cases {
c.execute(
&format!("INSERT INTO math_main VALUES($1,'a',$2::text::{data_type},7)"),
&[zid, main],
)
.unwrap();
if let Some(mapping) = mapping {
c.execute(
&format!("INSERT INTO math_bidtopid VALUES($1,'a',$2::text::{data_type},$3)"),
&[zid, mapping, &ticks[0]],
)
.unwrap();
}
if let Some(statistics) = statistics {
c.execute(
&format!("INSERT INTO math_ptptstats VALUES($1,'a',$2::text::{data_type},$3)"),
&[zid, statistics, &ticks[1]],
)
.unwrap();
}
c.execute(
"INSERT INTO math_ticks VALUES($1,'a',$2)",
&[zid, &ticks[2]],
)
.unwrap();
}
assert_eq!(
c.query_one(
"SELECT pg_typeof(m.data)::text, pg_typeof(b.data)::text, pg_typeof(p.data)::text
FROM math_main m, math_bidtopid b, math_ptptstats p LIMIT 1",
&[]
)
.map(|row| [row.get::<_, String>(0), row.get(1), row.get(2)])
.unwrap(),
[data_type; 3]
);
reads.push(
(1..=13)
.map(|zid| format!("{:?}", math_bundle::read(&mut c, zid, "a").unwrap()))
.collect(),
);
}
assert_eq!(reads[0], reads[1], "jsonb and json differ");
let complete = |read: &str| read.starts_with("Some(Complete(");
for (index, expected) in [
(1, "complete"),
(2, "complete"),
(3, "incoherent"),
(4, "incoherent"),
(5, "incoherent"),
(6, "incoherent"),
(7, "incoherent"),
(8, "incoherent"),
(9, "incoherent"),
(10, "complete"),
(11, "complete"),
(12, "refused"),
(13, "absent"),
] {
let read = reads[1][index - 1].as_str();
let found = if complete(read) {
"complete"
} else if read == "Some(Incoherent(7))" {
"incoherent"
} else if read == "Some(Absent)" {
"absent"
} else if read == "None" {
"refused"
} else {
read
};
assert_eq!(found, expected, "conversation {index}: {read}");
}
// The members of the first two generations, read from a json column.
for (zid, pid, bid) in [(1, 2, Some(99)), (2, 1, Some(42)), (2, 2, Some(99))] {
let Some(Read::Complete(bundle)) = math_bundle::read(&mut c, zid, "a").unwrap() else {
panic!("conversation {zid} is a complete generation");
};
assert_eq!(bundle.bid(pid), bid);
}
// Statistics in a column that is neither jsonb nor json are refused.
c.batch_execute("ALTER TABLE math_ptptstats ALTER COLUMN data TYPE text")
.unwrap();
assert!(math_bundle::read(&mut c, 1, "a").unwrap().is_none());
c.batch_execute(&format!("DROP SCHEMA {schema} CASCADE"))
.unwrap();
}
16 changes: 15 additions & 1 deletion queue-rs/polis-api/src/next_priorities.rs
Original file line number Diff line number Diff line change
Expand Up @@ -201,8 +201,22 @@ fn refresh(c: &mut Client, zid: i32, env: &str, cached: Option<Entry>) -> Result
// Explicit parameter types batch Parse/Bind/Execute into one exchange.
// CASE does not evaluate or transfer the JSONB projection on a cache hit.
// A missing priority field is distinct from a reused generation (SQL NULL).
//
// `data` is jsonb in a database built from the migrations and json in
// installations older than them. Nothing here remembers which: the
// server resolves `->` and the inner CASE against the column it is
// reading, each time the statement runs. Only the arm of the column's
// own type is evaluated, and its cast is to the type the column already
// has, so it does nothing; the other arm's cast, which would convert the
// whole document, never runs. A column of any other type has no `->` and
// fails the statement.
let rows = c.query_typed(
"select math_tick, case when math_tick = $3 then null else jsonb_typeof(data) end, \
"select math_tick, case when math_tick = $3 then null else \
case pg_typeof(data) \
when 'pg_catalog.jsonb'::regtype then jsonb_typeof(data::jsonb) \
when 'pg_catalog.json'::regtype then json_typeof(data::json) \
end \
end, \
case when math_tick = $3 then null else (data->'comment-priorities')::text end \
from math_main where zid=$1 and math_env=$2 and math_tick>=0",
&[(&zid, Type::INT4), (&env, Type::TEXT), (&tick, Type::INT8)],
Expand Down
Loading
Loading