When building a docker image using M1 chip, I'm getting an error: "standard_init_linux.go:228: exec user process caused: exec format error"

Archive
, , , , ,
1

Question copied from a Slack thread:

"hello, guys! I am having some issues building and deploying docker images using Prefect. My Docker agent runs on amd64 architecture and runs on an EC2 instance. But I am using a mac with an M1 chip to build the docker image using Docker storage, and it fails with the following error:

failed to get destination image "xxx": 
image with reference xxx was found 
but does not match the specified platform: 
wanted linux/amd64, actual: linux/arm64/v8

I have build_kwargs on the Docker storage as:

storage = Docker(
        registry_url="osai",
        image_name="enrichment-flow",
        image_tag="mal",
        dockerfile=f"{BASEDIR}/.enrichment/Dockerfile.enrich",
        build_kwargs={"buildargs": {"PYTHON_VERSION": "3.8.12"}, 
                      "platform": ["linux/amd64"]},
        python_dependencies=[
            "fasttext==0.9.2"
        ],
        files={
            f"{BASEDIR}/queries": "/opt/prefect/queries/",
            f"{BASEDIR}/enrichment": "/opt/prefect/enrichment/",
        },
        env_vars={
            "PYTHONPATH": "/opt/prefect/enrichment:/opt/prefect/queries",
            "PREFECT__LOGGING__LEVEL": "ERROR",
        },
    )

Can anybody please help me :pray:?"

Solution

Since the Docker agent is running on a platform that supports images built for the linux/amd64 platform, when building a Docker storage (e.g. on flow registration), you need to explicitly pass the platform as linux/amd64, otherwise by default your images built on M1 chip will be built for platform linux/arm64.

Based on Low-level API — Docker SDK for Python 6.0.1 documentation, the platform should be defined as a string, rather than a list:

storage = Docker(...,
             build_kwargs={"buildargs": {"PYTHON_VERSION": "3.8.12"}, 
                           "platform": "linux/amd64"},
...
)

For more on amd64 vs. arm64, check out Does Prefect support Docker images with arm64 platform?