Create an S3 block
First, go to the Prefect UI and create an S3 block:
Create a deployment from CLI
This will assume that you want to run your flow in a local process, i.e. using a
Process block. To use a different type of infrastructure, check recipes for Docker and Kubernetes.
prefect deployment build flows/hello.py:hello --name dev --tag dev --storage-block s3/dev
prefect deployment apply deployment.yaml
You should now see the entire project copied into your S3 bucket
When you run the
apply commands, you should expect output similar to:
Start an agent
prefect agent start --tag dev
You should see that your agent picked up the run:
You’re all set!
Hi! Just started going through this tutorial and I got to say It feels wrong to enter AWS service account credentials in clear text in the Prefect UI, especially using the Prefect Cloud. Wasn’t the value proposition that Prefect (as in the company) does not have access to the code which is being deployed? As soon as I upload my AWS service account credentials to your cloud I feel that goal somewhat compromised.
I have seen scattered documentation about using Blocks in Code instead of the UI but I can’t really make sense of it. Could you point me in the right direction here? I basically want a way to configure S3 storage without uploading my service account credentials in clear text to the Prefect Cloud.
Thanks a lot!
it’s not clear text, it’s a Secret field that is encrypted in transit and at rest. Check out the file systems block code for more detailed on how we handle it
Perfect, thanks that explains it