How to migrate Prefect 1.0 task library tasks to a Prefect Collection repository in Prefect 2.0?

View in #prefect-contributors on Slack

ale @ale: Hey folks :wave:
Are there any guidelines to “convert” existing Prefect 1.0 tasks to make them work with Prefect 2.0?

Anna_Geller @Anna_Geller: Hi Ale, great question! I asked Chris about it - we will have more concrete guidelines for that later. For now, we have a guide on how you can contribute to Prefect Collections discussed here How to contribute to Prefect Collections?

ale @ale: Thanks Anna! :raised_hands:
After digging into the code and other collections I was able to setup a prefect-cubejs collection on my local environment :blush:

Here you can find the code!

What are the next steps to submit the code into Prefect repo?

Anna_Geller @Anna_Geller: That’s awesome! I’m sure @alex can help with the review.

alex @alex: Hey @ale! Thanks so much for creating a Prefect Collection!

Here’s are the next steps I recommend for prefect-cubejs:

  • Ensure that your code has good unit test coverage. You can check this by running coverage run --branch -m pytest tests. We recommend at least 80% coverage.
  • Make sure that all the documentation is updated. This includes the docstrings in your Python code and README.md since they are used to generate your documentation. You can run mkdocs serve to view the generated documentation.
  • Publish a version of prefect-cubejs on PyPI. You can do this automatically by creating a tag in GitHub with a v prefix (e.g. v0.1.0). This will create a release on PyPI and also publish the autogenerated docs to GitHub Pages. You’ll need a PyPI to to publish to PyPI and you can refer to MAINTAINERS.md in your repo for more information.

Once you’ve completed the initial release, we can work together to get prefect-cubejs listed in the Prefect Collections Catalog.

If you run into any issues, let me know and I’ll be happy to help!

ale @ale: Hey @alex :wave:
Thanks for the detailed instructions!
I was able to release the very first version of prefect-cubejs on PyPi :tada:

Is there anything else I need to do?

alex @alex: Can you send me a link to the deployed documentation?

ale @ale: Sure, let me find it and will send it to you :sweat_smile:
Where can I find the URL of the generated Github Pages?

alex @alex: I think this quickstart will give you the info you need: Quickstart for GitHub Pages - GitHub Docs

ale @ale: I discovered Github Pages was disabled on my repo.
I’ve enabled it now, but I think I need to run the action again
There you go @alex https://alessandrolollo.github.io/prefect-cubejs/

alex @alex: Nice! Looks like the docstring on run_query need to be filled in. Can you you add a docstring that documents the arguments, return type, and examples for the task similar to these tasks here: prefect-slack/messages.py at main · PrefectHQ/prefect-slack · GitHub?

ale @ale: Working on it
Done!

alex @alex: Nice! In order to deploy the updated docs you’ll need to push the changes to the docs branch. Here’s a workflow that you can include in your project to deploy a new version of the docs: prefect-slack/publish-docs.yml at main · PrefectHQ/prefect-slack · GitHub. That workflow can be triggered manually from the GitHub Actions UI.

Also, I noticed references to prefect.engine.signals.FAIL in the new docstring, but I think that should be CubeJSAPIFailureException.

ale @ale: Will check it, thanks @alex :raised_hands:
Docstrings have been fixed and published to Github Pages :raised_hands:
@alex please let me know if there’s anything else I need to fix/improve

alex @alex: It looks like mkdocs is having trouble parsing the docstring on your task. I think that you can remove the typing from the docstring (e.g. (str, optional)) since mkdocs can pull typing info from the type hints on your function. If you want to experiment with changes to the documentation on your local you can run mkdocs serve which will rebuild the docs each time you make a change.

ale @ale: Should be fixed now

1 Like