Latest httpcore (0.17.3) and httpx (0.24.1) lead to concurrency crashes in the Prefect engine with httpx.ReadTimeout

Hi there,

we recently updated our Prefect and its dependencies and suddenly experienced a new error: an httpx.ReadTimeout (see stack trace further down) on a mapped task, occurring non-deterministically.

At first we believed the Prefect version to be the culprit (sorry : ), but we kept reproducing the error with Prefect 2.10, 2.11, and 2.12. Digging into the package dependencies, we found that pinning the httpcore package to version 0.16.3 and httpx to 0.23.3 solved the issue.

Since it is originating from the Prefect engine (see stack trace), could you please look into this issue?

Stack trace:

Sorry, as a new user I cannot upload more than one image, so I’m afraid no execution graph from me : (
We have several mapped tasks that run in parallel, then a mapped collection task that has a wait_for parameter on all the previous tasks, and then another mapped collection task that waits on the previous collection task.
It are instances of that last task that sometimes fail with the httpx.ReadTimeout error. This is new, and has not happened in the past months, and did not happen when I pinned the httpcore package to version 0.16.3 and httpx to 0.23.3.