Prefect 2 Cloud REST API: Task Run Request Returns 503

I’m currently using the python requests library to obtain task_run details from the Prefect 2 REST API. With that said, I keep getting a 503 status for an “unknown” reason.

Details:

POST JSON DATA PASSED--> {'offset': 600, 'limit': 200, 'task_runs': {'operator': 'and_', 'start_time': {'after_': '2024-06-21T00:00:00'}}}
https://api.prefect.cloud/api/accounts/our-prod-account-id/workspaces/our-prod-workspace-id/task_runs/filter
POST request: using the 'task_runs' route...
{'error': {'exception_message': 'Service Unavailable'}, 'status_code': 503}

More or less, I am running a while loop that chunks through the task_run requests until no more data is available. The basic code gist is below. From the results above, I expect the number of task_runs to be much greater than the offset where this fails (i.e. 600). The same code works for “flow_runs” until the while returns 0 recs (or len(records) < 200 limit). The documentation says I would get a 429 if I had hit a limit.

headers = {
    "Authorization": f"Bearer {self.prefect_api_key}",
    "Content-Type": "application/json",
}
try:
    req = (
        Request(method=method, url=url, headers=headers, json=json)
        if json
        else Request(method=method, url=url, headers=headers)
    )

    with session as s:
        # 5 second timeout for connection, 20 second timeout for read
        # retry 5 times with backoff factor of 0.1
        session_timeout = session_timeout
        prepped_req = req.prepare()
        resp = s.send(prepped_req, timeout=session_timeout)

    # if resp.status_code < 400, evaluates to true
    if resp:
        # print(resp.status_code, resp.json())
        # only expecting either a dict or a list
        return {"data": resp.json(), "status_code": resp.status_code}
    else:
        bad_resp = {"error": resp.json(), "status_code": resp.status_code}
        print(bad_resp)
        return bad_resp

except Exception as e:
    return {"error": e, "status_code": 500}

Thanks in advance for any help.

1 Like