How to use DockerFlowRunner with S3 storage? I'm getting an error "botocore.exceptions.ClientError: An error occurred (403) when calling the HeadObject operation: Forbidden"

Archive
, , , , , , , , ,
7

There is no such thing as .gcp folder, GCS uses a different convention for credentials.

If anything you could try .gsutil creds:

image
image778×740 59.7 KB

Can you try with S3 and .aws for now?

GCS Service Accounts

For GCS you would need to mount the JSON service account file as a volume:

DeploymentSpec(
    name="docker-example",
    flow=my_docker_flow,
    flow_runner=DockerFlowRunner(
        image="prefecthq/prefect:2.0b2-python3.9",
        volumes=["/Users/anna/repos/packaging-prefect-flows/gcs_sa.json:/opt/prefect/gcs_sa.json"],  # ADJUST IT TO MATCH YOUR GCS JSON PATH
    ),
)
1 Like