View in #prefect-server on Slack
@Scarlett_King: Hi guys, I’ve just redeployed prefect server to AKS via helm chart and having trouble connecting to graphql from ui. How can I troubleshoot this?
@Anna_Geller: You need to ensure that the right ports are open. Networking is a bit challenging on Azure and I’m no networking expert, but you need to ensure that:
• API endpoint is reachable - the default port is 4200 and it would need to be open to allow the API to be reachable from Server
• if you implement a Kubernetes agent, the port 8080 should be open by default to allow health checks - you can disable that in your agent when needed
@Scarlett_King: So from AKS, it looks like graphQL wasn’t started when we deployed the helm chart
Is there a way I can troubleshoot why graphql wasn’t started?
@Anna_Geller: The first thing I would look are the logs from the respective pods. If you are using the latest Server version, it has been quite recently moved to Hasura 2.0 which could be perhaps related to your issue?
Also, have you ran the database migration? I think this is the most common issue users come across when migrating server. There is a command that can help with DB migration:
prefect-server database upgrade -y
@Michael_Adkins: (The helm chart should run db migrations for you) If you can connect to the cluster you can use kubectl to list the pods e.g. kubctl get pods then use kubectl describe pod <NAME> and kubectl logs <NAME> to inspect the pod.
@Scarlett_King: Ah ok, that explains it. We revert back to an older image and it works. Where do I run the command for db migration? We’re deploying to AKS via the helm chart
The helm chart was stuck at upgrade step. I wonder if it’s an issue because we’re running persistent db?
@Anna_Geller: As long as you configured your persistent database on the helm chart, GraphQL should automatically run the migration for you in this line
GitHub: server/deployment.yaml at master · PrefectHQ/server
@Scarlett_King: Hi Anna, it seems we have configured the persistence db on the helm chart but we still get issue at the db upgrade command. On the helm chart, useSubChart is set to false and persistence is kept at enable: false as well. Do we need to set that to true? The notes on the helm chart seems to recommend that it stays false.
@Anna_Geller: Perhaps you can trigger it manually? The command should be:
kubectl exec graphqlPodName prefect-server database upgrade -y
@Scarlett_King: Ok, let me try it
Note: the same command in docker-compose Server set up would be:
docker exec graphqlContainerName prefect-server database upgrade -y
Note that after running DB migrations manually, you need to restart your Apollo container so that it can refetch the updated database schemas:
docker restart apolloContainerName
