Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 0 additions & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,6 @@ ipnet = { version = "2", features = ["serde"] }
http = "1.4"
indoc = "2"
itertools = "0.14"
jiff = { version = "0.2", default-features = false, features = ["std"] }
log = "0.4"
typed-reqwest = { version = "0.2.0", path = "typed-reqwest" }
mmigration = { path = "mmigration" }
Expand Down
17 changes: 17 additions & 0 deletions pg-ephemeral/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,20 @@
now carry the image reference as `Box<ociman::Reference>` rather than inline.
Code that pattern-matches these variants needs to dereference the binding.
`Display` output is unchanged.
- `wait_available_timeout` now uses PostgreSQL's time-unit syntax, the one
timeout parameters such as `statement_timeout` accept: an integer directly
followed by one of `us`, `ms`, `s`, `min`, `h` or `d`, such as `30s`, `500ms`
or `2min`. Every accepted value means exactly what it means to PostgreSQL:
it is rounded half to even to whole milliseconds, `0` disables the timeout,
and the maximum is `2147483647ms` (about 24.8 days). Values outside this
syntax now fail to load, including `1m` (write `1min`), compound values such
as `1m30s`, a space between number and unit, and units such as `w`.
- The `wait_available_timeout` fields of `Definition` and `config::Instance`,
and the timeout parameters of `Container::wait_available` and
`Container::set_superuser_password`, are now `definition::Timeout`, whose
`Disabled` variant waits without limit. The `Definition::wait_available_timeout`
builder accepts `impl Into<definition::Timeout>`, so existing calls passing a
`std::time::Duration` compile unchanged.

### Added

Expand All @@ -26,6 +40,9 @@

Parameters feed the seed cache key, so a top-level parameter invalidates the
cached images of every instance.
- `time_unit::Value`, parsing the subset of PostgreSQL's time-unit parameter
syntax used by `wait_available_timeout`, and `definition::Timeout`, giving
such a value the meaning of a PostgreSQL millisecond timeout parameter.

### Changed

Expand Down
1 change: 0 additions & 1 deletion pg-ephemeral/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,6 @@ env_logger.workspace = true
git-proc.workspace = true
hex.workspace = true
indexmap = { version = "2.13", features = ["serde"] }
jiff.workspace = true
log.workspace = true
nom.workspace = true
nom-language.workspace = true
Expand Down
29 changes: 28 additions & 1 deletion pg-ephemeral/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,7 @@ cache = { type = "none" }
| `cache_registry` | OCI registry prefix for cache images (e.g. `"ghcr.io/myorg"`). See [Sharing cache across machines](#sharing-cache-across-machines). |
| `parameters` | PostgreSQL server parameters inherited by every instance. See [Server parameters](#server-parameters). |
| `ssl_config` | SSL configuration with `hostname` field ([example](https://github.com/mbj/mrs/tree/main/pg-ephemeral/examples/08-ssl)) |
| `wait_available_timeout` | How long to wait for PostgreSQL to accept connections (e.g. `"30s"`) |
| `wait_available_timeout` | How long to wait for PostgreSQL to accept connections (e.g. `"30s"`, default `"10s"`). See [Wait timeout](#wait-timeout). |

### Backend selection

Expand Down Expand Up @@ -155,6 +155,33 @@ When `ssl_config` is set, pg-ephemeral controls the `ssl`, `ssl_cert_file`, `ssl
and `ssl_ca_file` parameters itself; setting any of them in `parameters` — at any layer —
is rejected at launch.

### Wait timeout

`wait_available_timeout` is written in PostgreSQL's own
[syntax for time-valued parameters](https://www.postgresql.org/docs/current/config-setting.html#CONFIG-SETTING-NAMES-VALUES),
the one `statement_timeout` accepts, restricted to its unambiguous form: an integer
directly followed by one of the units `us`, `ms`, `s`, `min`, `h` or `d`.

```toml
wait_available_timeout = "30s"

[instances.slow]
wait_available_timeout = "2min"
```

A value means exactly what it would mean to `statement_timeout`:

* It is rounded half to even to whole milliseconds, so `1500us` waits `2ms`.
* `0` in any unit, or a value that rounds to it such as `400us`, disables the timeout:
pg-ephemeral waits until PostgreSQL accepts connections, however long that takes.
* The maximum is `2147483647ms`, about 24.8 days.

Spellings outside that form are rejected, including some PostgreSQL itself accepts:
a space between number and unit (`30 s`), a number without unit, fractions (`1.5s`),
signs, leading zeros (PostgreSQL reads `010s` as octal, 8 seconds), and hexadecimal.
So are spellings common in other tools that PostgreSQL rejects too: `1m` (write `1min`),
compound values such as `1m30s`, and units such as `w`.

### Seed types

Seeds run in declaration order inside the container. Whenever a seed step exits,
Expand Down
63 changes: 9 additions & 54 deletions pg-ephemeral/src/config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ pub struct Instance {
pub superuser: pg_client::User,
pub image: Image,
pub cross_container_access: bool,
pub wait_available_timeout: std::time::Duration,
pub wait_available_timeout: crate::definition::Timeout,
}

impl Instance {
Expand All @@ -42,7 +42,9 @@ impl Instance {
database: pg_client::Database::POSTGRES,
image,
cross_container_access: false,
wait_available_timeout: std::time::Duration::from_secs(10),
wait_available_timeout: crate::definition::Timeout::After(
std::time::Duration::from_secs(10),
),
}
}

Expand Down Expand Up @@ -324,31 +326,6 @@ pub struct SslConfigDefinition {
pub hostname: pg_client::config::HostName,
}

/// Deserializes a duration from its unit-suffixed form, such as `30s`,
/// `1m30s` or `500ms`, as well as ISO 8601 (`PT30S`).
mod duration {
pub(super) fn deserialize<'de, D>(
deserializer: D,
) -> Result<Option<std::time::Duration>, D::Error>
where
D: serde::Deserializer<'de>,
{
let Some(text) =
<Option<std::string::String> as serde::Deserialize>::deserialize(deserializer)?
else {
return Ok(None);
};

let signed = text
.parse::<jiff::SignedDuration>()
.map_err(serde::de::Error::custom)?;

std::time::Duration::try_from(signed)
.map(Some)
.map_err(serde::de::Error::custom)
}
}

#[derive(Debug, serde::Deserialize, PartialEq)]
#[serde(deny_unknown_fields)]
pub struct InstanceDefinition {
Expand All @@ -359,8 +336,7 @@ pub struct InstanceDefinition {
#[serde(default)]
pub seeds: indexmap::IndexMap<SeedName, SeedConfig>,
pub ssl_config: Option<SslConfigDefinition>,
#[serde(default, with = "duration")]
pub wait_available_timeout: Option<std::time::Duration>,
pub wait_available_timeout: Option<crate::definition::Timeout>,
}

impl InstanceDefinition {
Expand Down Expand Up @@ -431,7 +407,9 @@ impl InstanceDefinition {
.wait_available_timeout
.or(self.wait_available_timeout)
.or(defaults.wait_available_timeout)
.unwrap_or(std::time::Duration::from_secs(10));
.unwrap_or(crate::definition::Timeout::After(
std::time::Duration::from_secs(10),
));

Ok(Instance {
application_name: None,
Expand All @@ -457,8 +435,7 @@ pub struct Config {
#[serde(default)]
parameters: pg_client::parameter::Map,
ssl_config: Option<SslConfigDefinition>,
#[serde(default, with = "duration")]
wait_available_timeout: Option<std::time::Duration>,
wait_available_timeout: Option<crate::definition::Timeout>,
instances: Option<std::collections::BTreeMap<InstanceName, InstanceDefinition>>,
}

Expand Down Expand Up @@ -946,25 +923,3 @@ mod test {
);
}
}

#[cfg(test)]
mod duration_tests {
use super::*;

#[test]
fn parses_unit_suffixed_durations() {
for (text, expected) in [
("30s", std::time::Duration::from_secs(30)),
("1m30s", std::time::Duration::from_secs(90)),
("2m", std::time::Duration::from_secs(120)),
("500ms", std::time::Duration::from_millis(500)),
("1h", std::time::Duration::from_secs(3600)),
("PT30S", std::time::Duration::from_secs(30)),
] {
let config = Config::load_toml(format!("wait_available_timeout = \"{text}\"\n"))
.unwrap_or_else(|error| panic!("{text}: {error}"));

assert_eq!(Some(expected), config.wait_available_timeout, "{text}");
}
}
}
43 changes: 26 additions & 17 deletions pg-ephemeral/src/container.rs
Original file line number Diff line number Diff line change
Expand Up @@ -403,16 +403,24 @@ impl Container {
})
}

pub async fn wait_available(&self, timeout: std::time::Duration) -> Result<(), Error> {
pub async fn wait_available(&self, timeout: crate::definition::Timeout) -> Result<(), Error> {
let config = self.client_config.to_sqlx_connect_options().unwrap();

let start = std::time::Instant::now();
let max_duration = timeout;
let sleep_duration = std::time::Duration::from_millis(100);

let mut last_error: Option<sqlx::Error> = None;

while start.elapsed() <= max_duration {
loop {
if let crate::definition::Timeout::After(duration) = timeout
&& start.elapsed() > duration
{
return Err(Error::ConnectionTimeout {
timeout: duration,
source: last_error,
});
}

log::trace!("connection attempt");
match sqlx::ConnectOptions::connect(&config).await {
Ok(connection) => {
Expand All @@ -434,11 +442,6 @@ impl Container {
}
tokio::time::sleep(sleep_duration).await;
}

Err(Error::ConnectionTimeout {
timeout: max_duration,
source: last_error,
})
}

pub async fn exec_schema_dump(
Expand Down Expand Up @@ -829,12 +832,23 @@ impl Container {
Ok(())
}

async fn wait_for_container_socket(&self, timeout: std::time::Duration) -> Result<(), Error> {
async fn wait_for_container_socket(
&self,
timeout: crate::definition::Timeout,
) -> Result<(), Error> {
let start = std::time::Instant::now();
let max_duration = timeout;
let sleep_duration = std::time::Duration::from_millis(100);

while start.elapsed() <= max_duration {
loop {
if let crate::definition::Timeout::After(duration) = timeout
&& start.elapsed() > duration
{
return Err(Error::ConnectionTimeout {
timeout: duration,
source: None,
});
}

if self
.container
.exec("pg_isready")
Expand All @@ -850,11 +864,6 @@ impl Container {
}
tokio::time::sleep(sleep_duration).await;
}

Err(Error::ConnectionTimeout {
timeout: max_duration,
source: None,
})
}

/// Set the superuser password using peer authentication via Unix domain socket.
Expand All @@ -864,7 +873,7 @@ impl Container {
pub async fn set_superuser_password(
&self,
password: &pg_client::config::Password,
timeout: std::time::Duration,
timeout: crate::definition::Timeout,
) -> Result<(), Error> {
self.wait_for_container_socket(timeout).await?;

Expand Down
45 changes: 41 additions & 4 deletions pg-ephemeral/src/definition.rs
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,43 @@ pub enum SeedApplyError {
EnvVariableValue(#[from] cmd_proc::EnvVariableValueError),
}

/// How long to wait, with the meaning PostgreSQL gives a millisecond timeout
/// parameter such as `statement_timeout`: the value is rounded half to even to
/// whole milliseconds, a value rounding to zero disables the timeout, and the
/// timeout is at most 2147483647ms.
#[derive(Clone, Copy, Debug, PartialEq, Eq, serde::Deserialize)]
#[serde(try_from = "crate::time_unit::Value")]
pub enum Timeout {
Disabled,
After(std::time::Duration),
}

impl From<std::time::Duration> for Timeout {
fn from(duration: std::time::Duration) -> Self {
Self::After(duration)
}
}

#[derive(Debug, thiserror::Error, PartialEq)]
#[error("timeout {value} is outside the valid range of 0ms .. 2147483647ms")]
pub struct TimeoutRangeError {
pub value: crate::time_unit::Value,
}

impl TryFrom<crate::time_unit::Value> for Timeout {
type Error = TimeoutRangeError;

fn try_from(value: crate::time_unit::Value) -> Result<Self, Self::Error> {
match value.to_milliseconds() {
Some(0) => Ok(Self::Disabled),
Some(milliseconds @ 1..) => Ok(Self::After(std::time::Duration::from_millis(
u64::from(milliseconds.unsigned_abs()),
))),
_ => Err(TimeoutRangeError { value }),
}
}
}

/// Errors from cache sync operations ([`Definition::pull_cache`] /
/// [`Definition::push_cache`]).
#[derive(Debug, thiserror::Error)]
Expand Down Expand Up @@ -95,7 +132,7 @@ pub struct Definition {
pub superuser: pg_client::User,
pub image: crate::image::Image,
pub cross_container_access: bool,
pub wait_available_timeout: std::time::Duration,
pub wait_available_timeout: Timeout,
pub remove: bool,
/// Optional user-facing identifier for the resulting container.
///
Expand Down Expand Up @@ -138,7 +175,7 @@ impl Definition {
database: pg_client::Database::POSTGRES,
image,
cross_container_access: false,
wait_available_timeout: std::time::Duration::from_secs(10),
wait_available_timeout: Timeout::After(std::time::Duration::from_secs(10)),
remove: true,
session_name: None,
transparent_workdir: None,
Expand Down Expand Up @@ -359,9 +396,9 @@ impl Definition {
}

#[must_use]
pub fn wait_available_timeout(self, timeout: std::time::Duration) -> Self {
pub fn wait_available_timeout(self, timeout: impl Into<Timeout>) -> Self {
Self {
wait_available_timeout: timeout,
wait_available_timeout: timeout.into(),
..self
}
}
Expand Down
1 change: 1 addition & 0 deletions pg-ephemeral/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ pub mod image;
pub mod label;
pub mod seed;
pub mod session;
pub mod time_unit;

pub use config::{Config, Instance};
pub use container::Container;
Expand Down
Loading
Loading