Feedback about getting started with Orion deployments and DockerFlowRunner

[ahuang11]

While trying to get the DockerFlowRunner running with the following:
https://orion-docs.prefect.io/tutorials/docker-flow-runner/

And receiving support:

How to use DockerFlowRunner with S3 storage? I'm getting an error "botocore.exceptions.ClientError: An error occurred (403) when calling the HeadObject operation: Forbidden"

There is no such thing as .gcp folder, GCS uses a different convention for credentials. If anything you could try .gsutil creds: [image] Can you try with S3 and .aws for now? GCS Service Accounts For GCS you would need to mount the JSON service account file as a volume: DeploymentSpec( name="docker-example", flow=my_docker_flow, flow_runner=DockerFlowRunner( image="prefecthq/prefect:2.0b2-python3.9", volumes=["/Users/anna/repos/packaging-prefect-flows/gcs_sa.js…

I'm getting storage errors when running flows from deployments using DockerFlowRunner (FileNotFoundError: [Errno 2] No such file or directory: '/var/folders/tb/xxx/T/prefect/deployment_UUID')

I tried following this: https://orion-docs.prefect.io/tutorials/docker-flow-runner/ The first time, I didn’t run prefect config set PREFECT_API_URL=http://127.0.0.1:4200/api The second time, I ran that, but I used local storage, but I saw this line: " * A remote Storage configuration, not Local Storage or Temporary Local Storage." The third time, I ran with GCS and meticulously followed the tutorial, but I still get the following: 10:26:02.698 | INFO | prefect.flow_runner.docker - Flow …

I had trouble with:

1. Running the docker flow tutorial as is (had to iterate many times over Discourse to clarify and add what was missing; thank you so much to Anna for your prompt responses!) I think expounding on credentials and custom docker image is needed.
2. Having three separate terminal windows to:
   a. prefect orion start
   b. prefect agent start UUID
   c. prefect deployment run orion-test-docker/docker_custom_local_image
   (Would really like a quick-start)
3. The terminal windows became outdated upon updating prefect storage set-default… and gcloud set project (I think this problem is exaggerated with having three separate terminal windows)
4. How I couldn’t set default values based on the label “Name” column, e.g. prefect storage set-default integrations-dev, had to lookup UUID with prefect storage ls then use the corresponding UUID.
5. Same for prefect agent start <UUID> rather than prefect agent start <LABEL-NAME>, and when trying to use The error message actually refers to “name” rather than “UUID”: 14:05:28.169 | WARNING | prefect.agent - No work queue found named 'tut-work-queue’.
6. When creating Local Storage, I found that if I used the user key ~, e.g. STORAGE PATH: ~/.prefect it doesn’t expand it. Anna mentioned that the docs state to use an absolute path. I think it’d be nice to either clarify that in the CLI upon calling prefect storage create, or supporting / erroring out when ~ or $HOME is used