Examples showing Infrastructure blocks and Deployment configuration in the latest Prefect 2.0 release (General Availability and later)

tbd

1. Process

tbd soon
tbd soon

2. DockerContainer

Using the DockerContainer block with extra environment variables and image pull policy:

import prefect
from prefect import task, flow
from prefect import get_run_logger
from prefect.deployments import Deployment
from prefect.infrastructure.docker import DockerContainer, ImagePullPolicy
from prefect.packaging import DockerPackager
from prefect.software import PythonEnvironment


@task
def say_hi():
    logger = get_run_logger()
    logger.info("Hello from the Health Check Flow! 👋")


@task
def log_platform_info():
    import platform
    import sys
    from prefect.orion.api.server import ORION_API_VERSION

    logger = get_run_logger()
    logger.info("Host's network name = %s", platform.node())
    logger.info("Python version = %s", platform.python_version())
    logger.info("Platform information (instance type) = %s ", platform.platform())
    logger.info("OS/Arch = %s/%s", sys.platform, platform.machine())
    logger.info("Prefect Version = %s 🚀", prefect.__version__)
    logger.info("Prefect API Version = %s", ORION_API_VERSION)


@flow
def healthcheck():
    hi = say_hi()
    log_platform_info(wait_for=[hi])


Deployment(
    name="docker",
    flow=healthcheck,
    infrastructure=DockerContainer(
        image_pull_policy=ImagePullPolicy.IF_NOT_PRESENT,
        env=dict(PREFECT_LOGGING_LEVEL="DEBUG"),
    ),
)

Using DockerContainer block with DockerPackager and the same flow as above:

from prefect.packaging import DockerPackager

Deployment(
    flow=healthcheck,
    name="docker_packager",
    packager=DockerPackager(
        base_image="prefecthq/prefect:2.0b12-python3.9",
        python_environment=PythonEnvironment(
            python_version="3.9",
            pip_requirements=["requests"],
        ),
    ),
    infrastructure=DockerContainer(
        env=dict(PREFECT_LOGGING_LEVEL="DEBUG"),
    ),
)

Using DockerContainer block with a custom AWS ECR image:

Deployment(
    flow=crypto,
    name="aws_ecr_docker_block",
    infrastructure=DockerContainer(
        image="123456789.dkr.ecr.us-east-1.amazonaws.com/prefect2:latest",
        env=dict(PREFECT_LOGGING_LEVEL="DEBUG"),
    ),
)

3. KubernetesJob

Using the KubernetesJob block with extra environment variables and image pull policy:

from prefect.infrastructure.kubernetes import KubernetesJob, KubernetesImagePullPolicy

Deployment(
    name="k8s",
    flow=healthcheck,
    infrastructure=KubernetesJob(
        image_pull_policy=KubernetesImagePullPolicy.IF_NOT_PRESENT,
        env=dict(PREFECT_LOGGING_LEVEL="DEBUG"),
    ),
)
tbd soon
2 Likes