Use local secrets in prefect 2

Help
, ,
1

Hello,

After moving from Prefect 1 to Prefect 2, one of the feature that I miss is the possibility to set local secrets via environment variables.

The only way I have found to manage secrets in a dev environment in prefect 2 is:

  • to create an entirely new workspace dedicated to dev
  • create secrets blocks.
  • Have everyone that needs these secrets to connect to the dev workspace.

However, it also means that I have to manually set every single new secret that my team may need to use; or let them manage existing secrets (which is not ideal security wise)

I was just wondering if there was a way to manage secrets locally with Prefect 2 that I may have missed ?

Thanks in advance

2 Likes
2

You’re totally right, that’s still missing but it’s on the roadmap, you can track it here:

1 Like
3

You can work around this and avoid creating another workspace by reading in a dev/prod prefix/suffix from a local file during deployment and set it as a parameter for your flow. Then you can append the same prefix/suffix to your secret blocks that need to swap based on your current environment.

deployment

from prefect.deployments import Deployment
from pythonscript import flowname
from os import getenv
from mods.secrets import secrets

def deploy():
    ## some module to load local environment vars
    secrets.load_env_vars()

    deployment = Deployment.build_from_flow(
        flow=flowname,
        name="common_flow_name",
### set flow parameter to 'dev' or 'prod' from local .env 
        parameters={"testProd_ind": getenv("Environment")},
        work_pool_name=getenv("work_pool"),
        work_queue_name=getenv("work_queue")
    )
    deployment.apply()

if __name__ == "__main__":
    deploy()

code in flow

from prefect import task, flow, get_run_logger
from prefect.blocks.system import Secret

@flow(name='common_flow_name')
def flowname(testProd_ind: str = 'dev'):
    
    secret_block_name = 'db-password-' + testProd_ind
    secret_block = Secret.load(secret_block_name )
    secret_block .get()