Topics tagged storage

Topic	Replies	Views	Activity
One Flow with Multiple Parameter Sets and Schedules Archive prefect-1-0 , storage , parameters , scheduling	1	797	February 28, 2024
A modern ETL pattern using Prefect’s S3 Storage and Kubernetes runtime Archive storage , kubernetes , blog-post , s3	1	432	February 12, 2024
GitHub storage block and dynamically setting branches Help prefect-2-0 , storage , github-storage	3	441	July 28, 2023
Cloud run - Write output files directly in local path Help prefect-2-0 , storage , agent , server	0	266	July 25, 2023
S3 storage and Flow code retrieval Help prefect-2-0 , storage , aws	0	195	July 17, 2023
How do I configure pandas serialization for a flow involving lots of DataFrames? Archive prefect-2-0 , storage , getting-started	7	1680	June 16, 2023
S3 bucket specifies that the Bucket Folder will still download all the buckets Help prefect-2-0 , storage , aws , blocks	0	300	May 15, 2023
Scheduled Flow (every 30min) crashed after running on local server for 2 days Help prefect-2-0 , storage , server , marvin	1	842	April 24, 2023
How can I run my flow in a Docker container? Archive migration-guide , prefect-1-0 , prefect-2-0 , storage , deployment , flow-runner , run_config , docker , getting-started , docker-flow-runner , docker-run	10	3974	March 30, 2023
Where/how is code pulled from storage Help prefect-2-0 , storage , deployment	1	306	January 13, 2023
How can I stop the persistence of flow meta data in local file system Archive storage	2	386	October 6, 2022
How to set Secrets (e.g. GITHUB_ACCESS_TOKEN) on Server? Archive prefect-1-0 , storage , kubernetes , server , secrets , helm-chart , config-toml , environment-variable , job_template , git	2	2858	October 5, 2022
Agent Debugging & Agent Execution error with FTP Storage in Windows Archive storage , agent	3	576	September 15, 2022
Subflow error - PrefectHTTPStatusError - Client error '403 Forbidden' Archive prefect-2-0 , storage , subflows	1	731	September 8, 2022
How to deploy a Prefect 2.0 agent to an EC2 instance as your execution layer? Archive prefect-2-0 , storage , aws , s3 , agent , getting-started , aws-ec2 , ubuntu , work-queue , linux , execution-layer-deployment-tutorial	0	4495	April 8, 2022
Prefect 2.0 Docker Container using GitLab storage Archive prefect-2-0 , storage , docker , git , gitlab , github-storage	2	1281	July 29, 2022
New Orion release 2.0a13 featuring work queues, agents, profiles, remote object storage and distributed parallel processing with Ray! Announcements prefect-2-0 , release-notes , storage , aws , s3 , gcp , task-runner , parallel-processing , agent , azure , concurrency-limits , kubernetes-flow-runner , filters , gcs , docker-flow-runner , profiles-prefect-config , work-queue , ray , ray-task-runner , azure-blob-storage	7	1091	July 29, 2022
Are there some packages that are not compatible with prefect2? Archive prefect-2-0 , storage , task-runner , dask-task-runner	12	987	July 23, 2022
How can I deploy a flow? Archive migration-guide , prefect-1-0 , prefect-2-0 , backend , storage , deployment , flow-runner , run_config , getting-started	0	641	January 21, 2022
New Prefect 2.0 release: 2.0b4 including flow and task runner improvements, performance optimizations, numerous fixes and new Prefect Collections! Announcements prefect-2-0 , release-notes , cli , storage , deployment , flow-runner , task-runner , dask-task-runner , async , twitter , kubernetes-flow-runner , great-expectations , ray-task-runner , cube-js , prefect-collections , subprocess-flow-runner , github	3	1151	July 5, 2022
Why it's beneficial to separate flow storage from code dependencies required by your flow? Archive prefect-2-0 , storage , code-dependencies	0	776	June 11, 2022
When I run my flow, I see an error: Failed to load and execute Flow's environment: ModuleNotFoundError("No module named '/Users/username'"). What is happening? Archive prefect-1-0 , cli , storage , defaults , flow-registration , pickle , local-agent , script-storage , pickle-storage , local-storage	3	5407	May 10, 2022
Does Prefect 2.0 support Redis for flow storage? Archive prefect-2-0 , storage , kv-store , redis	0	907	April 26, 2022
How to fix the error: "Failed to load and execute flow run: FlowStorageError(‘An error occurred while unpickling the flow" Archive prefect-1-0 , storage , serialization , pickle , pickle-storage	1	1151	April 21, 2022
Additional upload options with S3 Storage and S3 Results Archive prefect-1-0 , storage , aws , s3 , s3-storage , server-side-encryption	7	1278	March 24, 2022
How can I remove already created storage options? Archive prefect-2-0 , cli , storage , delete-flow	1	775	March 24, 2022
When getting started with Prefect 2.0, I'm getting an error "MissingContextError: No profile context found" Archive prefect-2-0 , storage , troubleshooting , getting-started	7	3415	March 22, 2022
When do I need to reregister my flow? Which changes in flow metadata result in Prefect bumping up the flow version? Archive prefect-1-0 , dag-flow-structure , cli , storage , executor , flow-registration , versioning , script-storage , pickle-storage , flow-metadata , serialized_hash	0	1141	March 2, 2022
How can I run my flow in a Docker container with a custom image? Archive migration-guide , prefect-1-0 , prefect-2-0 , storage , deployment , flow-runner , run_config , docker , getting-started , docker-flow-runner , docker-run , docker-image	0	838	January 31, 2022
Using LocalStorage with DockerRun Archive prefect-1-0 , storage , run_config , docker , local-storage , docker-run	0	809	February 24, 2022

One Flow with Multiple Parameter Sets and Schedules

Archive

1

797

February 28, 2024

A modern ETL pattern using Prefect’s S3 Storage and Kubernetes runtime

Archive

storage , kubernetes , blog-post , s3

1

432

February 12, 2024

GitHub storage block and dynamically setting branches

Help

prefect-2-0 , storage , github-storage

3

441

July 28, 2023

Cloud run - Write output files directly in local path

Help

prefect-2-0 , storage , agent , server

0

266

July 25, 2023

S3 storage and Flow code retrieval

Help

prefect-2-0 , storage , aws

0

195

July 17, 2023

How do I configure pandas serialization for a flow involving lots of DataFrames?

Archive

prefect-2-0 , storage , getting-started

7

1680

June 16, 2023

S3 bucket specifies that the Bucket Folder will still download all the buckets

Help

prefect-2-0 , storage , aws , blocks

0

300

May 15, 2023

Scheduled Flow (every 30min) crashed after running on local server for 2 days

Help

prefect-2-0 , storage , server , marvin

1

842

April 24, 2023

How can I run my flow in a Docker container?

Archive

migration-guide , prefect-1-0 , prefect-2-0 , storage , deployment , flow-runner , run_config , docker , getting-started , docker-flow-runner , docker-run

10

3974

March 30, 2023

Where/how is code pulled from storage

Help

prefect-2-0 , storage , deployment

1

306

January 13, 2023

How can I stop the persistence of flow meta data in local file system

Archive

storage

2

386

October 6, 2022

How to set Secrets (e.g. GITHUB_ACCESS_TOKEN) on Server?

Archive

prefect-1-0 , storage , kubernetes , server , secrets , helm-chart , config-toml , environment-variable , job_template , git

2

2858

October 5, 2022

Agent Debugging & Agent Execution error with FTP Storage in Windows

Archive

storage , agent

3

576

September 15, 2022

Subflow error - PrefectHTTPStatusError - Client error '403 Forbidden'

Archive

prefect-2-0 , storage , subflows

1

731

September 8, 2022

How to deploy a Prefect 2.0 agent to an EC2 instance as your execution layer?

Archive

prefect-2-0 , storage , aws , s3 , agent , getting-started , aws-ec2 , ubuntu , work-queue , linux , execution-layer-deployment-tutorial

0

4495

April 8, 2022

Prefect 2.0 Docker Container using GitLab storage

Archive

prefect-2-0 , storage , docker , git , gitlab , github-storage

2

1281

July 29, 2022

New Orion release 2.0a13 featuring work queues, agents, profiles, remote object storage and distributed parallel processing with Ray!

Announcements

prefect-2-0 , release-notes , storage , aws , s3 , gcp , task-runner , parallel-processing , agent , azure , concurrency-limits , kubernetes-flow-runner , filters , gcs , docker-flow-runner , profiles-prefect-config , work-queue , ray , ray-task-runner , azure-blob-storage

7

1091

July 29, 2022

Are there some packages that are not compatible with prefect2?

Archive

prefect-2-0 , storage , task-runner , dask-task-runner

12

987

July 23, 2022

How can I deploy a flow?

Archive

migration-guide , prefect-1-0 , prefect-2-0 , backend , storage , deployment , flow-runner , run_config , getting-started

0

641

January 21, 2022

New Prefect 2.0 release: 2.0b4 including flow and task runner improvements, performance optimizations, numerous fixes and new Prefect Collections!

Announcements

prefect-2-0 , release-notes , cli , storage , deployment , flow-runner , task-runner , dask-task-runner , async , twitter , kubernetes-flow-runner , great-expectations , ray-task-runner , cube-js , prefect-collections , subprocess-flow-runner , github

3

1151

July 5, 2022

Why it's beneficial to separate flow storage from code dependencies required by your flow?

Archive

prefect-2-0 , storage , code-dependencies

0

776

June 11, 2022

When I run my flow, I see an error: Failed to load and execute Flow's environment: ModuleNotFoundError("No module named '/Users/username'"). What is happening?

Archive

prefect-1-0 , cli , storage , defaults , flow-registration , pickle , local-agent , script-storage , pickle-storage , local-storage

3

5407

May 10, 2022

Does Prefect 2.0 support Redis for flow storage?

Archive

prefect-2-0 , storage , kv-store , redis

0

907

April 26, 2022

How to fix the error: "Failed to load and execute flow run: FlowStorageError(‘An error occurred while unpickling the flow"

Archive

prefect-1-0 , storage , serialization , pickle , pickle-storage

1

1151

April 21, 2022

Additional upload options with S3 Storage and S3 Results

Archive

prefect-1-0 , storage , aws , s3 , s3-storage , server-side-encryption

7

1278

March 24, 2022

How can I remove already created storage options?

Archive

prefect-2-0 , cli , storage , delete-flow

1

775

March 24, 2022

When getting started with Prefect 2.0, I'm getting an error "MissingContextError: No profile context found"

Archive

prefect-2-0 , storage , troubleshooting , getting-started

7

3415

March 22, 2022

When do I need to reregister my flow? Which changes in flow metadata result in Prefect bumping up the flow version?

Archive

prefect-1-0 , dag-flow-structure , cli , storage , executor , flow-registration , versioning , script-storage , pickle-storage , flow-metadata , serialized_hash

0

1141

March 2, 2022

How can I run my flow in a Docker container with a custom image?

Archive

migration-guide , prefect-1-0 , prefect-2-0 , storage , deployment , flow-runner , run_config , docker , getting-started , docker-flow-runner , docker-run , docker-image

0

838

January 31, 2022

Using LocalStorage with DockerRun

Archive

prefect-1-0 , storage , run_config , docker , local-storage , docker-run

0

809

February 24, 2022