Topics tagged infrastructure

Topic	Replies	Views	Activity
Main topic: Flow Deployment Patterns & Getting Started Recipes DataflowOps for Prefect 2.0 prefect-2-0 , deployment , ci-cd , event-driven , infrastructure , real-time-streaming , infrastructure-as-code , execution-layer-deployment-tutorial , list-of-resources , tbd This list of resources will point you to any topic of your interest regarding how to build a reliable flow deployment process suitable to your use cases, your data stack, and your team. 1. Getting started: install P…	3	1295	September 22, 2022
Security Advisory: OpenSSL Vulnerability Announcements prefect-1-0 , prefect-2-0 , infrastructure , security , ssl , open-ssl , advisory	0	57	November 3, 2022
Create an Infrastructure From a Python Object Tutorials & Resources prefect-2-0 , deployment , infrastructure	0	120	September 30, 2022
Tricks for reclaiming memory in Prefect flows and preventing Out-Of-Memory errors (OOM) Deployments & DevOps prefect-1-0 , prefect-2-0 , infrastructure , memory_request , resource-clean-up , oom-error	2	600	September 23, 2022
Overview of Infrastructure Blocks in CLI-based deployments Deployments & DevOps prefect-2-0 , docker , infrastructure , getting-started	0	196	September 13, 2022
Deploying an MVP on Heroku, Scalingo or Dokku Tutorials & Resources prefect-2-0 , deployment , infrastructure	1	70	September 16, 2022
Out-of-memory (OOM) error after all tasks completed Usage Questions prefect-2-0 , agent , infrastructure	8	162	September 8, 2022
Creating and Deploying a custom Kubernetes Infrastructure Block Deployments & DevOps deployment , kubernetes , infrastructure , infrastructure-blocks	0	309	September 2, 2022
CI/CD for Prefect 2.0 DataflowOps for Prefect 2.0 prefect-2-0 , deployment , ci-cd , docker , infrastructure , environments , list-of-resources , todo	0	938	May 14, 2022
How can I parallelize execution across 8 CPU cores? Workflow Orchestration migration-guide , prefect-1-0 , prefect-2-0 , dask , executor , task-runner , parallel-processing , dask-task-runner , local-dask-executor , infrastructure , multithreading , multiprocessing	0	380	January 31, 2022
How can I run tasks in parallel? Workflow Orchestration migration-guide , prefect-1-0 , prefect-2-0 , dask , executor , task-runner , parallel-processing , dask-task-runner , infrastructure , multithreading , multiprocessing , basics	0	456	January 31, 2022
Docker compose to run Prefect 2.0 locally with Postgres Tutorials & Resources prefect-2-0 , local-development , docker , user-contribution , infrastructure , postgres , docker-compose , orion-api-server	0	405	May 14, 2022
Do Prefect base images include extras from the task library e.g. postgres for the Postgres tasks? FAQ prefect-1-0 , task-library , dockerfile , infrastructure , code-dependencies , docker-image , best-practices , base-image , extra_pip_packages	1	283	June 27, 2022
Can Prefect run each task in a different Docker container? Deployments & DevOps prefect-1-0 , flow-of-flows , docker , create_flow_run , orchestrator-pattern , wait_for_flow_run , infrastructure , code-dependencies , docker-run , docker-image , base-image	1	633	June 14, 2022
How to deploy self-hosted Server and register first flows to the Server backend? Deployments & DevOps backend , blog-post , server , helm-chart , config-toml , infrastructure	0	353	January 27, 2022
How to assign IAM permissions to a Kubernetes cluster so that my flows can access other AWS services such as S3 or DynamoDB? Deployments & DevOps prefect-1-0 , auth-and-permissions , kubernetes , aws , infrastructure , iam , service-accounts , iam-roles-for-service-accounts	0	302	February 20, 2022
How scalable is Prefect Server for scheduling concurrent runs of tens of thousands of flows? Deployments & DevOps prefect-1-0 , server , infrastructure , postgres , scaling-up-and-out , performance	0	275	March 21, 2022
How to handle ECS errors during flow run deployment? Is it possible to implement a flow-level retry for errors such as this one: "Failed to start [ECS] task for flow run SOME_UUID"? Deployments & DevOps prefect-1-0 , aws , ecs , infrastructure , cpu-memory-resource-allocation , ecs-agent	0	147	March 28, 2022
How to disable the SSL verification when setting up a Prefect agent? I'm getting SSLCertVerificationError: "CERTIFICATE_VERIFY_FAILED certificate verify failed: unable to get local issuer certificate" Deployments & DevOps prefect-1-0 , infrastructure , troubleshooting , kubernetes-agent , network-and-firewall , ssl	1	405	March 28, 2022
Can I run a flow-of-flows that triggers each child flow on a different machine? Deployments & DevOps prefect-1-0 , flow-of-flows , agent , labels , create_flow_run , orchestrator-pattern , wait_for_flow_run , infrastructure	0	450	February 7, 2022
Docker agent cannot communicate with a Prefect Server Postgres database instance running locally on a host VM Deployments & DevOps prefect-1-0 , docker , agent , server , infrastructure , docker-agent , postgres , network-and-firewall	6	710	March 18, 2022
Launch Week Day 4 is not over yet! We've just released Customer Success Recipes and Prefect 2.0b2! Announcements prefect-2-0 , release-notes , dbt , kubernetes , aws , ci-cd , s3 , docker , secrets , event-driven , airbyte , ecs , infrastructure , kubernetes-agent , docker-agent , docker-flow-runner , network-and-firewall , aws-ec2 , launch-week , agent-setup , linux , terraform , infrastructure-as-code , customer-success-recipes , ecs-agent	0	320	March 17, 2022
How to handle custom resource (CPU vs. GPU) and code dependencies across various tasks and flows? Workflow Orchestration prefect-1-0 , prefect-2-0 , flow-of-flows , kubernetes , aws , orchestrator-pattern , subflows , infrastructure , cpu_request , kubernetes-agent , aws-eks , radar-view , gpu-intensive-workflows , cpu-memory-resource-allocation	2	889	March 15, 2022
I'm getting an error when pulling a Prefect Docker image - how to troubleshoot? Deployments & DevOps docker , server , infrastructure , troubleshooting , docker-image	0	215	March 14, 2022
Are there any infrastructure-specific limits when using Prefect Cloud? FAQ organizational , prefect-1-0 , prefect-2-0 , cloud-1-0 , backend , infrastructure , getting-started , hybrid-execution-model	0	149	March 11, 2022
Multi-cloud setup - data on GCP, agents on AWS EKS, or is it better to deploy one agent in each cloud provider? Deployments & DevOps prefect-1-0 , cloud-1-0 , aws , gcp , agent , infrastructure	0	140	March 3, 2022
How to deal with the long startup time (latency) on flow runs deployed to ECS Fargate? Deployments & DevOps kubernetes , aws , fargate , docker , agent , ecs , infrastructure , kubernetes-run , kubernetes-agent , aws-eks , docker-image , aws-ec2 , image-pull-policy	0	432	March 3, 2022
When using a KubernetesAgent on Azure AKS with Prefect Cloud, I got an error: "Connection aborted.', OSError(0, 'Error')" Deployments & DevOps prefect-1-0 , kubernetes , azure , infrastructure , kubernetes-run , kubernetes-agent , aks , network-and-firewall	0	221	February 25, 2022
Why is my flow stuck in a Submitted State? Workflow Orchestration prefect-1-0 , states , infrastructure , stuck , submitted-state , lazarus	1	869	February 24, 2022
What's the log retention in Prefect Cloud (Starter and Standard)? FAQ prefect-1-0 , logging , infrastructure , log-retention	0	153	February 20, 2022

Main topic: Flow Deployment Patterns & Getting Started Recipes

prefect-2-0 , deployment , ci-cd , event-driven , infrastructure , real-time-streaming , infrastructure-as-code , execution-layer-deployment-tutorial , list-of-resources , tbd

This list of resources will point you to any topic of your interest regarding how to build a reliable flow deployment process suitable to your use cases, your data stack, and your team. 1. Getting started: install P…