The purpose behind writing this at all is just very selfish. I want to understand it better which is the only reason I'm writing it. I struggled a lot when I first started to wrap my head around concurrency and parallelism (they both are different). Had some task at work which explicitly needed in depth knowledge of both of these and especially concurrency. Went through fastapi docs, python official docs, but didn't get the intuition. But slowly read some articles and it started to click.

What really is concurrency?

  • It is nothing but a fake parallelism for poor people with no real computing power.
  • Really, it basically is saying "you continue with the main thing, I'll let you know when I'm done and you can include me."

What is the problem anyways?

We'll start with a basic example. Let's say you have 2 things to cook — pizza and burger.

Now we'll introduce a term called synchronously (means we go line by line in layman terms).

Normally what we do is cook pizza and then cook burger. But what we should do is make pizza and burger concurrently. Let's say you arranged the pizza and now you just have to bake it — you can put it in oven and start cooking burger. Now pizza is being baked and you are making burger, no time is being wasted, you are not waiting for the pizza to be ready. Now you can put burger somewhere to be cooked and take out pizza and make it ready to serve, meanwhile in that time, when burger gets cooked, you can also take it out and make it ready.

Some technicalities

Now async functions are identified by writing async in front of them and in those functions you can await some operations which you think would be I/O heavy operations or something like that.

await literally means suspend here, go run the next thing and I'll let you know when I'm ready.

Now this all is managed by an event loop and what you define with async are not normal functions but they are coroutines, and they won't run until they are awaited.

def a():
    return 22

print(a())

async def aa():
    return 11

print(aa())

Output:

22
<coroutine object aa at 0x10a0f5900>
async_pr.py:158: RuntimeWarning: coroutine 'aa' was never awaited
print(aa())
RuntimeWarning: Enable tracemalloc to get the object allocation traceback

See, it's a coroutine object — you would have to await it to run it.

Now the Event Loop has all the control. It starts when you run asyncio.run(main()) and ends when you end it. You can kind of say this is how async context manager works — not exactly, but how FastAPI would run.

async def ab():
    print('1')
    await asyncio.sleep(1)
    print('2')

async def aa():
    print('3')
    await asyncio.sleep(2)
    print('4')

async def main():
    await asyncio.gather(ab(), aa())

asyncio.run(main())

Output:

1
3
2
4

Basically it suspends when it hits await.

Flow — starts with calling ab, prints 1, then as it hits await it gets suspended. Next in line is aa, hits aa, prints 3, suspends on await. Now it has a queue internally — kind of a ready queue.

Initially — [ab, aa]

then — [aa]

then — []

then — [ab's await is done, it returns to ready queue and it runs from where it was left]

similarly for aa

This is the basic flow of how the general execution would happen.

Note — Not every await would suspend. It depends on the operation; if it is very small, it may return instantly, and if it is big enough to be awaited, it will be suspended. This is the reason you do not keep small operations like OS ops or json.dumps etc. in asynchronous manner — there would be overhead of registering them, then checking if they are complete, the transfer of control from event loop back and forth. Rather you could have kept it in synchronous manner and let it run; it won't affect the performance as much.

Tasks

One thing you would have noticed is nothing after await would run in a coroutine object — after the awaited thing is returned, only then does it run forward, so you would have to wait until you get its result. Tasks are just a wrapper around coroutines, so they do not block your further operations and they return immediately (now they won't start until the next await is called).