storage

Topic Replies Views Activity
How to deploy a Prefect 2.0 agent to an EC2 instance as your execution layer?
Tutorials & Resources
prefect-2-0 ,  storage ,  aws ,  s3 ,  agent ,  getting-started ,  aws-ec2 ,  ubuntu ,  work-queue ,  linux ,  execution-layer-deployment-tutorial
0 206 April 8, 2022
Why it's beneficial to separate flow storage from code dependencies required by your flow?
FAQ
prefect-2-0 ,  storage ,  code-dependencies
0 25 June 11, 2022
When should I use FileStorageBlock vs. specific cloud-platform storage block such as S3StorageBlock, GoogleCloudStorageBlock or AzureBlobStorageBlock?
Workflow Orchestration
prefect-2-0 ,  storage ,  s3 ,  gcs ,  best-practices ,  azure-blob-storage ,  s3-storage ,  blocks
1 105 June 6, 2022
New Prefect 2.0 release: 2.0b4 including flow and task runner improvements, performance optimizations, numerous fixes and new Prefect Collections!
Announcements
prefect-2-0 ,  release-notes ,  cli ,  storage ,  deployment ,  flow-runner ,  task-runner ,  dask-task-runner ,  async ,  twitter ,  kubernetes-flow-runner ,  great-expectations ,  ray-task-runner ,  cube-js ,  prefect-collections ,  subprocess-flow-runner ,  github
2 191 May 13, 2022
When I run my flow, I see an error: Failed to load and execute Flow's environment: ModuleNotFoundError("No module named '/Users/username'"). What is happening?
FAQ
prefect-1-0 ,  cli ,  storage ,  defaults ,  flow-registration ,  pickle ,  local-agent ,  script-storage ,  pickle-storage ,  local-storage
3 549 May 10, 2022
Does Prefect 2.0 support Redis for flow storage?
Deployments & DevOps
prefect-2-0 ,  storage ,  kv-store ,  redis
0 56 April 26, 2022
How to fix the error: "Failed to load and execute flow run: FlowStorageError(‘An error occurred while unpickling the flow"
Deployments & DevOps
prefect-1-0 ,  storage ,  serialization ,  pickle ,  pickle-storage
1 49 April 21, 2022
How to deserialize flow run results generated by default S3 storage?
Workflow Orchestration
prefect-2-0 ,  storage ,  aws ,  s3 ,  pickle ,  results ,  s3-storage
0 46 April 20, 2022
One Flow with Multiple Parameter Sets and Schedules
Workflow Orchestration
prefect-1-0 ,  storage ,  parameters ,  scheduling
0 112 April 5, 2022
Additional upload options with S3 Storage and S3 Results
Deployments & DevOps
prefect-1-0 ,  storage ,  aws ,  s3 ,  s3-storage ,  server-side-encryption
7 113 March 24, 2022
How can I remove already created storage options?
Getting Started
prefect-2-0 ,  cli ,  storage ,  delete-flow
1 55 March 24, 2022
When getting started with Prefect 2.0, I'm getting an error "MissingContextError: No profile context found"
Getting Started
prefect-2-0 ,  storage ,  troubleshooting ,  getting-started
7 115 March 22, 2022
When do I need to reregister my flow? Which changes in flow metadata result in Prefect bumping up the flow version?
Workflow Orchestration
prefect-1-0 ,  dag-flow-structure ,  cli ,  storage ,  executor ,  flow-registration ,  versioning ,  script-storage ,  pickle-storage ,  flow-metadata ,  serialized_hash
0 167 March 2, 2022
How to set Secrets (e.g. GITHUB_ACCESS_TOKEN) on Server?
Deployments & DevOps
prefect-1-0 ,  storage ,  kubernetes ,  server ,  secrets ,  helm-chart ,  config-toml ,  environment-variable ,  job_template ,  git
0 281 January 26, 2022
New Orion release 2.0a13 featuring work queues, agents, profiles, remote object storage and distributed parallel processing with Ray!
Announcements
prefect-2-0 ,  release-notes ,  storage ,  aws ,  s3 ,  gcp ,  task-runner ,  parallel-processing ,  agent ,  azure ,  concurrency-limits ,  kubernetes-flow-runner ,  filters ,  gcs ,  docker-flow-runner ,  profiles-prefect-config ,  work-queue ,  ray ,  ray-task-runner ,  azure-blob-storage
4 241 March 7, 2022
How can I run my flow in a Docker container with a custom image?
Deployments & DevOps
migration-guide ,  prefect-1-0 ,  prefect-2-0 ,  storage ,  deployment ,  flow-runner ,  run_config ,  docker ,  getting-started ,  docker-flow-runner ,  docker-run ,  docker-image
0 125 January 31, 2022
How can I run my flow in a Docker container?
Deployments & DevOps
migration-guide ,  prefect-1-0 ,  prefect-2-0 ,  storage ,  deployment ,  flow-runner ,  run_config ,  docker ,  getting-started ,  docker-flow-runner ,  docker-run
0 123 January 25, 2022
How can I deploy my flow to the API backend?
Deployments & DevOps
migration-guide ,  prefect-1-0 ,  prefect-2-0 ,  backend ,  storage ,  deployment ,  flow-runner ,  run_config ,  getting-started
0 94 January 21, 2022
Using LocalStorage with DockerRun
Deployments & DevOps
prefect-1-0 ,  storage ,  run_config ,  docker ,  local-storage ,  docker-run
0 95 February 24, 2022
What exactly does a prefect agent pull from storage?
FAQ
prefect-1-0 ,  storage ,  agent ,  code-dependencies
0 121 February 20, 2022
Why do I get the error TypeError: cannot pickle object?
FAQ
prefect-1-0 ,  storage ,  serialization ,  pickle ,  results ,  checkpointing
1 131 February 18, 2022
When building a docker image using M1 chip, I'm getting an error: "standard_init_linux.go:228: exec user process caused: exec format error"
Deployments & DevOps
prefect-1-0 ,  storage ,  docker ,  arm64 ,  code-dependencies ,  amd64
0 431 February 2, 2022
How to allocate more memory or more worker nodes on a per flow run basis?
Distributed Computing
prefect-1-0 ,  storage ,  run_config ,  kubernetes ,  dask ,  executor ,  kubernetes-run ,  coiled ,  dask-executor ,  memory_request ,  cpu_request
0 83 January 31, 2022
I got an error: Failed to load and execute Flow's environment: GitlabGetError('404 Project Not Found') - how to fix it?
Deployments & DevOps
prefect-1-0 ,  storage ,  secrets ,  gitlab
1 115 January 31, 2022
When registering my flow, I get an error: ValueError: Local Secret "AZURE_STORAGE_CONNECTION_STRING" was not found
Deployments & DevOps
prefect-1-0 ,  cloud-1-0 ,  storage ,  deployment ,  defaults ,  secrets ,  environment-variable ,  flow-registration ,  azure
0 107 January 26, 2022
A modern ETL pattern using Prefect’s S3 Storage and Kubernetes runtime
Tutorials & Resources
storage ,  kubernetes ,  blog-post ,  s3
0 84 January 21, 2022