-
Notifications
You must be signed in to change notification settings - Fork 1.9k
Description
Bug summary
When creating a deployment from yaml definition, the build step does not pick up the config variable PREFECT_DEFAULT_DOCKER_BUILD_NAMESPACE
to adjust the name of the build image. When using the python flow.deploy()
method, the build image includes the namespace as intended. Referencing this portion of the docs: https://docs.prefect.io/v3/deploy/infrastructure-examples/docker#automatically-build-a-custom-docker-image-with-a-local-dockerfile
First
prefect config set PREFECT_DEFAULT_DOCKER_BUILD_NAMESPACE=my-private-repository.com
With a flow entrypoint of flow.py:hello_world
With yaml deployment:
# build section allows you to manage and build docker images
build:
- prefect_docker.deployments.steps.build_docker_image:
id: build_image
requires: prefect-docker>=0.3.1
image_name: 'test-image'
tag: latest
dockerfile: "./Dockerfile"
# push section allows you to manage if and how this project is uploaded to remote locations
push:
- prefect_docker.deployments.steps.push_docker_image:
requires: prefect-docker>=0.3.1
image_name: '{{ build_image.image_name }}'
tag: '{{ build_image.tag }}'
# the deployments section allows you to provide configuration for deploying flows
deployments:
- name: sample_deployment
version: '{{ build_image.tag }}'
tags:
- test
description: Sample ECS deployment for unit testing
entrypoint: flow.py:hello_world
parameters: {}
work_pool:
name: my-work-pool
job_variables:
image: '{{ build_image.image }}'
work_queue_name:
schedules: []
concurrency_limit:
pull:
- prefect.deployments.steps.set_working_directory:
directory: /opt/prefect/sample_ecs_deployment
Running prefect deploy flow.py:hello_world
should yield an image: my-private-repository.com/test-image:latest
Instead I get an image test-image:latest
When I use the python deploy method via
flow.from_source(
source = "./",
entrypoint = "flow.py:hello_world",
).deploy(
name=name,
work_pool_name="ecs-worker-pool",
image=DockerImage(
name="test-image",
tag="latest",
dockerfile="Dockerfile",
),
)
I successfully get the output image my-private-repository.com/test-image:latest
built locally and then pushed to my private repository.
Version info
Version: 3.2.9
API version: 0.8.4
Python version: 3.12.9
Git commit: 27eb408c
Built: Fri, Feb 28, 2025 8:12 PM
OS/Arch: linux/x86_64
Profile: ephemeral
Server type: server
Pydantic version: 2.10.6
Additional context
The custom image built through the python deploy method has an error as without the pull
step from the yaml deployment (seems to be no equivalent in the python method), the working directory of the deployed container inherits the local path context from the docker build... but this deserves a separate issue and is related to the discussion here: https://linen.prefect.io/t/16246009/ulva73b9p-is-it-possible-to-manually-set-a-working-directory
As a workaround I plan to adjust to build the images externally.