Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions tests/load/.env.example
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ PROMETHEUS_RETENTION_SIZE=2GB
RESTART_POLICY=no
COZY_STACK_BIND_ADDRESS=127.0.0.1
COZY_STACK_PORT=8080
COZY_ADMIN_PORT=16060
COZY_INSTANCE_DOMAIN=load.localhost:8080
COZY_BASE_URL=http://load.localhost:8080
COZY_CLIENT_SOFTWARE_ID=twake-drive-load-local
Expand Down
10 changes: 10 additions & 0 deletions tests/load/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,16 @@ COZY_STACK_URL ?= http://$(or $(COZY_STACK_BIND_ADDRESS),127.0.0.1):$(or $(COZY_
# Where the host reaches the Stack when k6 uses a Compose-internal BASE_URL: the bundled Stack's port
# when BASE_URL is that Stack, as make cozy-prepare writes it to .env.
STACK_URL ?= $(if $(filter $(COZY_BASE_URL),$(BASE_URL)),$(COZY_STACK_URL))
# Organization fixtures seed the bundled Stack through its loopback admin port, in the load-org-sharing
# context (compose.yaml, config/cozy/cozy.yaml). Shared DEV seeds through the Cloudery (SEEDING.md).
ifeq ($(LOAD_TARGET)$(SEED_CLOUDERY_TOKEN_FILE),local)
SEED_ADMIN_URL ?= http://127.0.0.1:$(or $(COZY_ADMIN_PORT),16060)
SEED_STACK_URL ?= $(COZY_STACK_URL)
SEED_ADMIN_PASSWORD ?= cozy
SEED_DOMAIN_SUFFIX ?= localhost:8080
SEED_CONTEXT_NAME ?= load-org-sharing
export SEED_ADMIN_URL SEED_STACK_URL SEED_ADMIN_PASSWORD SEED_DOMAIN_SUFFIX SEED_CONTEXT_NAME
endif
COZY_CLIENT_SOFTWARE_ID ?= twake-drive-load-local
COZY_TOKEN_EXPIRATION ?= 24h
HOST_UID ?= $(shell id -u)
Expand Down
6 changes: 4 additions & 2 deletions tests/load/ORG-DRIVE-FANOUT.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,10 @@ metadata is fabricated.

## Server prerequisites

Use a disposable Stack deployment. Configure a dedicated context before creating
the fixture, for example:
Use a disposable Stack deployment. The bundled Stack (`make cozy-prepare`) has
the context below and its admin API on 127.0.0.1:16060, and Make points the
seeder at it ([SEEDING.md](SEEDING.md)). Elsewhere, configure a dedicated
context before creating the fixture, for example:

```yaml
contexts:
Expand Down
19 changes: 12 additions & 7 deletions tests/load/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,17 +33,18 @@ separate future scenarios.
- `smoke` runs every step on its shortest ladder, in about ten minutes after the
first build; `capacity` runs the full ladders ([Campaigns](#campaigns)).
- Locally, the campaign first builds and starts the bundled Stack from this
checkout (`make cozy-prepare`). The group fan-out runs on shared DEV only, and
the [mixed workload](#mixed-workload) by hand, until the campaign covers them.
checkout (`make cozy-prepare`), then seeds one organization for the ramp's
members and the group fan-out, and removes it at the end. The
[mixed workload](#mixed-workload) still runs by hand, until the campaign covers it.
- Shared DEV needs `SEED_CLOUDERY_TOKEN_FILE`, `SEED_CLOUDERY_MANIFEST` and
`DEV_INSTANCE`, holds the shared DEV lock and uses the budgets agreed with the
DEV owner, in `campaigns/capacity.sh` ([Shared DEV](#shared-dev)).
- Each run writes `data/results/campaigns/<run id>-<name>/summary.md`, with a
Grafana link per step, and exits 0 when completed, 1 on a proven correctness
failure and 2 when inconclusive.
- One runner alone: `make cozy-prepare`, then `make upload-capacity`,
`upload-regression` or `shared-drive-ramp`; `make org-drive-fanout` also needs
an organization fixture ([ORG-DRIVE-FANOUT.md](ORG-DRIVE-FANOUT.md)). Add
`upload-regression`, `shared-drive-ramp` or `org-drive-fanout` (which seeds
its own organization, [ORG-DRIVE-FANOUT.md](ORG-DRIVE-FANOUT.md)). Add
`LOAD_TARGET=dev` for shared DEV.

## How it is built
Expand Down Expand Up @@ -309,7 +310,8 @@ Shared drives can be tested in two modes:
With `ACTORS_DOMAIN`, the ramp runs as members of an organization fixture
(`make seed-org SEED_USERS=2 SEED_CONTEXT_NAME=load-org-sharing`, read from
its [manifest](SEEDING.md#fixture-manifest)) on a local Stack, using the
seeder's `SEED_ADMIN_URL`, `SEED_ADMIN_PASSWORD` and `SEED_STACK_URL`. The
seeder's `SEED_ADMIN_URL`, `SEED_ADMIN_PASSWORD` and `SEED_STACK_URL`
(the bundled Stack's by default). The
organization instance owns a fresh drive and creates the test directory;
its first member uploads through a group-only grant; its second member
observes through a read-only grant. Each uses its own URL and token. Before
Expand Down Expand Up @@ -619,8 +621,11 @@ $ make campaign CAMPAIGN=capacity LOAD_TARGET=dev # shared DEV, from the load
```

The steps are the 100 KiB upload capacity on the personal drive, the
shared-drive ramp and, on shared DEV, the group fan-out: the local Stack has no
organization fixture for it yet. Locally, the steps keep going after a failure.
shared-drive ramp and the group fan-out. Locally, one organization fixture,
seeded before the ramp and removed after the fan-out, gives the ramp its writer
and reader members and the fan-out its recipients; `run.log` names it, for
`make seed-org-cleanup SEED_DOMAIN=...` after an interrupted run. Locally, the
steps keep going after a failure.
A definition is a `campaign()` function calling `upload_capacity`,
`shared_drive_ramp`, `org_drive_fanout`, or `run_step`.

Expand Down
4 changes: 4 additions & 0 deletions tests/load/SEEDING.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,10 @@ make seed-scale SEED_FILES=1000 SEED_DRIVES=10
# Seed instance: seed-<32-hex-id>.load.example.test:8443
```

On the bundled Stack (`LOAD_TARGET=local`, no Cloudery token), Make defaults
these settings to its loopback admin port (`COZY_ADMIN_PORT`, 16060), its host
port, the `load-org-sharing` context and the local admin passphrase.

`SEED_ADMIN_URL` and `SEED_STACK_URL` are explicit HTTP(S) connection origins,
without credentials, paths, queries or fragments. Public requests connect to
`SEED_STACK_URL` while sending the generated instance domain as the exact `Host`
Expand Down
25 changes: 18 additions & 7 deletions tests/load/campaigns/capacity.sh
Original file line number Diff line number Diff line change
@@ -1,13 +1,17 @@
# Every runner's capacity, the highest level it sustains, on the local Stack or on shared DEV
# (make campaign CAMPAIGN=capacity LOAD_TARGET=dev). smoke.sh runs the same steps on short ladders.
# Locally, members is the organization fixture's size: the largest fan-out batch, and at least the
# ramp's writer and reader.
if [ "$CAMPAIGN" = smoke ]; then
upload=(MAX_VUS=2 BASELINE_ITERATIONS=4 CONFIRMATION_ITERATIONS=2)
ramp=(RAMP_LEVELS=1,2 RAMP_STEP_DURATION=30s)
fanout=(FANOUT_LEVELS=1,2)
members=2
else
upload=(BASELINE_ITERATIONS=50 CONFIRMATION_ITERATIONS=20)
ramp=(RAMP_LEVELS=1,2,4,8,16)
fanout=(FANOUT_LEVELS=1,2,4,8,16)
members=16
fi
# Shared DEV budgets, agreed with the DEV owner: raise them only with that agreement. The ladders come
# last, so a smoke ladder stays below them. DEV needs DEV_INSTANCE and the Cloudery token and manifest
Expand All @@ -19,14 +23,21 @@ if [ "${SHARED_DEV:-}" = 1 ]; then
fi

campaign() {
# Locally, every step uses the bundled Stack and the instance and token this writes to .env.
if [ "${SHARED_DEV:-}" != 1 ]; then
run_step cozy-prepare prepare '' cozy-prepare || return
fi
upload_capacity upload-capacity FILE_SIZE=100K "${upload[@]}"
shared_drive_ramp shared-drive-ramp "${ramp[@]}"
# The fan-out needs an organization: shared DEV has the Cloudery one, the local Stack none yet.
if [ "${SHARED_DEV:-}" = 1 ]; then
upload_capacity upload-capacity FILE_SIZE=100K "${upload[@]}"
shared_drive_ramp shared-drive-ramp "${ramp[@]}"
org_drive_fanout org-drive-fanout "${fanout[@]}"
return
fi
# Locally, every step uses the bundled Stack and the instance and token this writes to .env.
run_step cozy-prepare prepare '' cozy-prepare || return
upload_capacity upload-capacity FILE_SIZE=100K "${upload[@]}"
# One organization fixture: the ramp's writer and reader members, and the fan-out's recipients.
local org
org=seed-$(od -An -N16 -tx1 /dev/urandom | tr -d ' \n').${SEED_DOMAIN_SUFFIX:-localhost:8080}
echo "Organization fixture: $org (make seed-org-cleanup SEED_DOMAIN=$org removes it)"
run_step seed-org prepare '' seed-org SEED_DOMAIN="$org" SEED_USERS="$members"
shared_drive_ramp shared-drive-ramp ACTORS_DOMAIN="$org" "${ramp[@]}"
org_drive_fanout org-drive-fanout FANOUT_ORG_DOMAIN="$org" "${fanout[@]}"
run_step seed-org-cleanup cleanup '' seed-org-cleanup SEED_DOMAIN="$org"
}
2 changes: 2 additions & 0 deletions tests/load/compose.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -120,6 +120,8 @@ services:
LOCAL_GROUP_ID: ${HOST_GID:-1000}
ports:
- "${COZY_STACK_BIND_ADDRESS:-127.0.0.1}:${COZY_STACK_PORT:-8080}:8080"
# The admin API, for seeding organization fixtures: loopback only, whatever the Stack's address.
- "127.0.0.1:${COZY_ADMIN_PORT:-16060}:6060"
expose:
- "6060"
volumes:
Expand Down
10 changes: 10 additions & 0 deletions tests/load/config/cozy/cozy.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -37,3 +37,13 @@ rabbitmq:
default:
enabled: true
url: amqp://cozy:cozy@rabbitmq:5672/

# Organization fixtures (ORG-DRIVE-FANOUT.md): members auto-accept their organization's drives.
contexts:
load-org-sharing:
common_contacts: true
max_members_per_sharing: 1001.0
sharing:
auto_accept_trusted: true
auto_accept_trusted_contacts: false
trusted_domains: []
7 changes: 7 additions & 0 deletions tests/load/src/scripts/org-drive-fanout.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -325,6 +325,13 @@ test('recipient canonical scheme mismatch is inconclusive, not a ready cohort',
equal(f.calls.filter(r => r.method === 'DELETE').length, 0)
})

test('an admin password that is a common word, as on the bundled Stack, leaves the envelope valid', async t => {
const f = await fixture(t)
const result = await f.run({ SEED_ADMIN_PASSWORD: 'cozy' })
equal(result.code, 0, result.stderr)
ok(validateEnvelope(result.report.result), JSON.stringify(result.report.result))
})

test('unrecognized roster fields cannot leak an upstream secret into the report', async t => {
const f = await fixture(t, r => {
if (r.headers.host === domain(1) && r.url.pathname === '/sharings/drive-1') r.transform = value => {
Expand Down
5 changes: 3 additions & 2 deletions tests/load/src/scripts/org-drive-fanout.ts
Original file line number Diff line number Diff line change
Expand Up @@ -152,8 +152,9 @@ async function main(): Promise<void> {
for (const secret of secrets) text = text.replaceAll(secret, '[redacted]')
return text
}
const render = () => JSON.stringify(report, (_key, value: unknown) =>
typeof value === 'string' ? redact(value) : value, 2) + '\n'
// The envelope's identity is the harness's own constant: a common-word password must not rewrite it.
const render = () => JSON.stringify(report, (key, value: unknown) =>
typeof value === 'string' && !['schema', 'producer', 'workload'].includes(key) ? redact(value) : value, 2) + '\n'
try {
check(process.argv.length === 2, 'Configuration belongs in environment variables, not arguments')
const levelsRaw = process.env.FANOUT_LEVELS ?? '1,2,4,8,16'
Expand Down
18 changes: 13 additions & 5 deletions tests/load/src/scripts/run-campaign.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -255,19 +255,27 @@ test('a shared DEV campaign stops after a failed or inconclusive step', async t
const setting = (command: string | undefined, name: string): string | undefined =>
command?.split(' ').filter(arg => arg.startsWith(`${name}=`)).at(-1)?.slice(name.length + 1)

test('smoke and capacity run every runner on their own ladders, the fan-out only on shared DEV', async t => {
test('smoke and capacity run every runner on their own ladders, locally on one organization fixture', async t => {
const plays = {
'upload-capacity': { exit: 0, report: upload },
'shared-drive-ramp': { exit: 0, report: ramp('lower_bound') },
'org-drive-fanout': { exit: 0, report: { ...fanout, capacity: { kind: 'lower_bound', lower: 4 }, status: 'passed' } },
}
for (const [campaign, vus, levels] of [['smoke', '2', '1,2'], ['capacity', undefined, '1,2,4,8,16']] as const) {
for (const [campaign, vus, levels, members] of [['smoke', '2', '1,2', '2'],
['capacity', undefined, '1,2,4,8,16', '16']] as const) {
const run = await runCampaign(t, '', plays, undefined, { campaign })
equal(run.code, 0, run.log)
// Locally, the bundled Stack is prepared before any runner.
deepEqual(run.targets, ['cozy-prepare', 'upload-capacity', 'shared-drive-ramp'])
// Locally, the bundled Stack is prepared before any runner, and the organization removed after the last.
deepEqual(run.targets, ['cozy-prepare', 'upload-capacity', 'seed-org', 'shared-drive-ramp', 'org-drive-fanout',
'seed-org-cleanup'])
deepEqual([setting(run.commands[1], 'LOAD_TARGET'), setting(run.commands[1], 'FILE_SIZE'),
setting(run.commands[1], 'MAX_VUS'), setting(run.commands[2], 'RAMP_LEVELS')], ['local', '100K', vus, levels])
setting(run.commands[1], 'MAX_VUS'), setting(run.commands[3], 'RAMP_LEVELS'),
setting(run.commands[2], 'SEED_USERS'), setting(run.commands[4], 'FANOUT_LEVELS')],
['local', '100K', vus, levels, members, levels])
const org = setting(run.commands[2], 'SEED_DOMAIN')!
match(org, /^seed-[a-f0-9]{32}\./)
deepEqual([setting(run.commands[3], 'ACTORS_DOMAIN'), setting(run.commands[4], 'FANOUT_ORG_DOMAIN'),
setting(run.commands[5], 'SEED_DOMAIN')], [org, org, org])
if (!hasFlock()) continue
// On shared DEV, the agreed budgets apply and a smoke ladder stays below them.
const dev = await runCampaign(t, '', plays, undefined, { campaign, env: {
Expand Down
18 changes: 15 additions & 3 deletions tests/load/src/scripts/seed-organization-make.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,19 +7,21 @@ import { test, type TestContext } from 'node:test'
import { fileURLToPath } from 'node:url'

const cwd = fileURLToPath(new URL('../../', import.meta.url))
function run(t: TestContext, target: string, settings: Record<string, string> = {}) {
function run(t: TestContext, target: string, settings: Record<string, string> = {},
keys = ['SEED_USERS', 'SEED_FILES', 'SEED_DRIVES']) {
const root = mkdtempSync(join(tmpdir(), 'seed-org-make-'))
t.after(() => rmSync(root, { recursive: true, force: true }))
const bin = join(root, 'bin')
mkdirSync(bin)
const executable = join(bin, 'tsx')
writeFileSync(executable, `#!/usr/bin/env node
const keys = ['SEED_USERS', 'SEED_FILES', 'SEED_DRIVES']
const keys = JSON.parse(process.env.FAKE_KEYS)
console.log(JSON.stringify({ args: process.argv.slice(2),
env: Object.fromEntries(keys.filter(k => process.env[k] !== undefined).map(k => [k, process.env[k]])) }))
`)
chmodSync(executable, 0o700)
const env = Object.fromEntries(Object.entries(process.env).filter(([key]) => !key.startsWith('SEED_')))
const env = { ...Object.fromEntries(Object.entries(process.env).filter(([key]) => !/^(SEED_|COZY_|LOAD_TARGET)/.test(key))),
FAKE_KEYS: JSON.stringify(keys) }
const result = spawnSync('make', ['--no-print-directory', '-s', '-o', 'node_modules/.load-ready',
`NODE_BIN=${bin}`, target, ...Object.entries(settings).map(([key, value]) => `${key}=${value}`)],
{ cwd, env, encoding: 'utf8' })
Expand Down Expand Up @@ -52,3 +54,13 @@ test('Make keeps an explicit organization cleanup domain in one argument', t =>
deepEqual(result.args, ['src/scripts/seed-scale-data.ts', 'cleanup-org', domain])
deepEqual(run(t, 'seed-verify', { SEED_DOMAIN: domain }).args, ['src/scripts/seed-scale-data.ts', 'verify', domain])
})

test('Locally, organization fixtures default to the bundled Stack; shared DEV and the Cloudery never do', t => {
const keys = ['SEED_ADMIN_URL', 'SEED_STACK_URL', 'SEED_ADMIN_PASSWORD', 'SEED_DOMAIN_SUFFIX', 'SEED_CONTEXT_NAME']
deepEqual(run(t, 'seed-org', { COZY_STACK_PORT: '18080' }, keys).env, {
SEED_ADMIN_URL: 'http://127.0.0.1:16060', SEED_STACK_URL: 'http://127.0.0.1:18080', SEED_ADMIN_PASSWORD: 'cozy',
SEED_DOMAIN_SUFFIX: 'localhost:8080', SEED_CONTEXT_NAME: 'load-org-sharing',
})
deepEqual(run(t, 'seed-org', { LOAD_TARGET: 'dev' }, keys).env, {})
deepEqual(run(t, 'seed-org', { SEED_CLOUDERY_TOKEN_FILE: 'token.json' }, keys).env, {})
})
Loading