Skip to content

Yaml deployment doesn't use PREFECT_DEFAULT_DOCKER_BUILD_NAMESPACE for private repository name #17604

@jlwhelan28

Description

@jlwhelan28

Bug summary

When creating a deployment from yaml definition, the build step does not pick up the config variable PREFECT_DEFAULT_DOCKER_BUILD_NAMESPACE to adjust the name of the build image. When using the python flow.deploy() method, the build image includes the namespace as intended. Referencing this portion of the docs: https://docs.prefect.io/v3/deploy/infrastructure-examples/docker#automatically-build-a-custom-docker-image-with-a-local-dockerfile

First

prefect config set PREFECT_DEFAULT_DOCKER_BUILD_NAMESPACE=my-private-repository.com

With a flow entrypoint of flow.py:hello_world

With yaml deployment:

# build section allows you to manage and build docker images
build:
- prefect_docker.deployments.steps.build_docker_image:
    id: build_image
    requires: prefect-docker>=0.3.1
    image_name: 'test-image'
    tag: latest
    dockerfile: "./Dockerfile"

# push section allows you to manage if and how this project is uploaded to remote locations
push:
- prefect_docker.deployments.steps.push_docker_image:
    requires: prefect-docker>=0.3.1
    image_name: '{{ build_image.image_name }}'
    tag: '{{ build_image.tag }}'

 # the deployments section allows you to provide configuration for deploying flows
deployments:
- name: sample_deployment
  version: '{{ build_image.tag }}'
  tags:
  - test
  description: Sample ECS deployment for unit testing
  entrypoint: flow.py:hello_world
  parameters: {}
  work_pool:
    name: my-work-pool
    job_variables:
      image: '{{ build_image.image }}'
    work_queue_name:
  schedules: []
  concurrency_limit:
pull:
- prefect.deployments.steps.set_working_directory:
    directory: /opt/prefect/sample_ecs_deployment

Running prefect deploy flow.py:hello_world should yield an image: my-private-repository.com/test-image:latest
Instead I get an image test-image:latest

When I use the python deploy method via

flow.from_source(
    source = "./",
    entrypoint = "flow.py:hello_world",
).deploy(
    name=name,
    work_pool_name="ecs-worker-pool",
    image=DockerImage(
        name="test-image",
        tag="latest",
        dockerfile="Dockerfile",
    ),
)

I successfully get the output image my-private-repository.com/test-image:latest built locally and then pushed to my private repository.

Version info

Version:             3.2.9
API version:         0.8.4
Python version:      3.12.9
Git commit:          27eb408c
Built:               Fri, Feb 28, 2025 8:12 PM
OS/Arch:             linux/x86_64
Profile:             ephemeral
Server type:         server
Pydantic version:    2.10.6

Additional context

The custom image built through the python deploy method has an error as without the pull step from the yaml deployment (seems to be no equivalent in the python method), the working directory of the deployed container inherits the local path context from the docker build... but this deserves a separate issue and is related to the discussion here: https://linen.prefect.io/t/16246009/ulva73b9p-is-it-possible-to-manually-set-a-working-directory

As a workaround I plan to adjust to build the images externally.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions