v2.5.0

Features

New API endpoint for filtering tasks on metadata

This release adds a new endpoint for filtering tasks based on metadata field_name and a regexp pattern for the value. The endpoint enables a Metaflow client feature for looking up task ancestry.

Add arm64 support for Dockerfile

This release also adds support for arm64 for the release Dockerfile. The official image remains amd64 for the time being, but the changes enable trying out a local container of the release services on arm64 hardware.

What's Changed

• fix: incompatibility with imported DAGNode and service dag generation by @saikonen in #450
• feature: add multiarch support for Dockerfile by @saikonen in #452
• feature: add task filtering based on metadata by @saikonen in #449
• release: 2.5.0 by @saikonen in #453
• fix: update dockerhub credential secrets by @saikonen in #455

Full Changelog: v2.4.13...v2.5.0

v2.4.13

Improvements

Database table trigger versioning

This release changes the naming of table triggers that the ui_backend sets up to include a timestamp as a version. This allows the backend to recreate table triggers in case any changes need to be deployed, or skip in case of a version match.

⚠️ After deploying this version of the service, do note that in the case of a service rollback, the database will have duplicate table triggers due to older service versions not having any cleanup code for these. In such a case, clean up the remaining triggers manually

Reduced Postgres notify messaging

This release reduces the volume of pg_notify messages that the database sends on table events. There should not be any effect on the average deployment, but for very large deployments running on services like Amazon Aurora, this solves significant performance issues.

Fixes runs starting with a failed status in certain cases

This release fixes an issue with runs starting with a failed status if the Metaflow client version being used had an uncommon version string.

What's Changed

• fix: linting issues by @saikonen in #433
• feature: version table triggers by @saikonen in #432
• fix: log paths failure by @saikonen in #430
• Bump azure-identity from 1.10.0 to 1.16.1 in /services/ui_backend_service by @dependabot in #425
• fix: reduce postgres notify messaging by @saikonen in #409
• fix: correctly sort log lines based on utc_tstamp by @saikonen in #435
• fix: make trigger cleanup schema aware by @saikonen in #436
• fix: version string parse failure leads to runs starting without a heartbeat by @saikonen in #443
• fix: run status stuck as failed with heartbeat changes by @saikonen in #441
• feature: treat client version parse failures as heartbeat-enabled by default by @saikonen in #444
• fix: incorrect table name passing in notify broadcast by @saikonen in #446
• release: 2.4.13 by @saikonen in #447

New Contributors

• @dependabot made their first contribution in #425

Full Changelog: v2.4.12...v2.4.13

v2.4.12

Upgrade metaflow-ui to v1.3.13

v2.4.11

Improvements

More memory efficient handling of large logs

This release alleviates some of the issues with memory consumption when processing extremely large logs ranging in gigabytes.

What's Changed

• feature: massive logs streaming by @saikonen in #424
• bump version to 2.4.11 by @saikonen in #428

Full Changelog: v2.4.9...v2.4.11

v2.4.10

Full Changelog: v2.4.9...v2.4.10

v2.4.9

What's Changed

• [cache-memory-leak] Fix Memory leak in cache server by @valayDave in #416
• update dev images to 3.11.7-bookworm by @saikonen in #418
• [card-cache-service] optimize caching by @valayDave in #417
• Fix: cache issues by @saikonen in #292
• bump version to 2.4.9 by @saikonen in #419

Full Changelog: v2.4.8...v2.4.9

v2.4.8

What's Changed

• [Ready for review] Update SearchAPI to search for foreach variable name and value. by @darinyu in #411
• fix: make search api changes backwards compatible by @saikonen in #414
• Release/2.4.8 by @saikonen in #415

New Contributors

• @darinyu made their first contribution in #411

Full Changelog: v2.4.7...v2.4.8

v2.4.7

Features

Support realtime cards

This release pins the version of Metaflow UI to v1.3.9 which adds support for realtime cards

Full Changelog: v2.4.6...v2.4.7

v2.4.6

Features

API additions for Metaflow UI

This release adds Metaflow UI API endpoints required for the upcoming realtime cards feature.

Improvements

Issue with goose script for migrations

Fixes a typo that caused the run_goose.py to fail when enabling ssl for the database connection.

What's Changed

• fix: remove legacy virtualenv from Dockerfile by @saikonen in #399
• Bump version to 2.4.6 by @saikonen in #400
• chore: revert bundle fix for further discussion by @saikonen in #401
• [realtime-cards] backend API Methods by @valayDave in #402
• [realtime-cards] fix bug in mf version by @valayDave in #410
• fix: change ssl_mode to sslmode by @rswigginton in #408
• Release/2.4.6 by @saikonen in #412

New Contributors

• @rswigginton made their first contribution in #408

Full Changelog: v2.4.5...v2.4.6

v2.4.5

Updating UI version to 1.3.7