Can I override CPU and memory value of the ECSTask infrastructure block for a specific container?

Yes. Here is an example of how you can override that configuration for a specific command.

from prefect_aws.ecs import ECSTask

ecs = ECSTask.load("default")
ecs.cpu = 4096
ecs.memory = 8192
ecs.command = ["python", "/opt/prefect/"]

Where can I run that from?

You can trigger that from any terminal or process authenticated with Prefect Cloud - e.g. from a Prefect flow, a Prefect task, or from a custom script running outside of Prefect.