Hello everyone,
Currently, I’m managing some flows which uses Postgres Tasks functions of Prefect1.0. I was very pleased that Prefect2.0 added concurrency options as part of the open-source release, and I wanted to migrate some of our workflows to the 2.0 version as this would solve many of our issues.
I soon realized that this won’t be possible without quite heavily refactoring tasks, which uses the Postgres Task library from prefect.tasks.postgres.postgres
.
Is there any plan to add these functions or its alternative to the Prefect 2.0 Collections?
Thanks.
1 Like
Hi @sasinka_radim , welcome to Prefect Discourse! Great to have you here!
A Postgres Prefect Collection is 100% on the roadmap! I can’t say when, but it will definitely be added.
If you would like to contribute in order to speed up the process, you may check those:
What are Prefect Collections?
Prefect Collections are groupings of pre-built tasks and flows used to quickly build data flows with Prefect.
Collections are grouped around the services with which they interact. For example, to download data from an S3 bucket, you could use the s3_download task from the prefect-aws collection, or if you want to send a Slack message as part of your flow, you could use the send_message task from the prefect-slack collection.
By using Prefect Collections, you ca…
View in #prefect-community on Slack
[Chu_Lục_Ninh] @Chu_Lục_Ninh: Hi @Kevin_Kho, Orion is getting more attractive now. But the collection is only maintained by prefect, can I contribute to Orion task collection? Or any guide to create my own task collection and share that to other developers?
[Kevin_Kho] @Kevin_Kho: I will tell the people behind collections to respond to you tom. I don’t know the guidelines yet myself since they’re a bit new
@Anna_Geller : @Chu_Lục_Ninh you can absolutely con…
Otherwise, it will be added for sure sometime in the coming weeks or months.
Will be creating the postgres collection this week!
I wanted to ask which tasks you’re specifically using; is it just PostgresExecute
or are you using PostgresExecuteMany
and PostgresFetch
too?
1 Like
Hi @ahuang11 ,
that’s great to hear. We were about to do that on our side as well, but this will help a ton.
We are currently using mostly PostgresExecute
and PostgresFetch
.
Thanks!
2 Likes
We decided to first release sqlalchemy, which supports postgres, here
https://prefecthq.github.io/prefect-sqlalchemy/
1 Like