The Practical Guide Every Backend Developer Needs
When building scalable backend systems in Python, one question always appears:
Should I use async, threading, or multiprocessing?
I’ve seen many developers blindly use one of them without understanding the real difference — and that often leads to performance issues.
Let’s break it down clearly and practically.
No theory overload.
Just what actually matters.
Before choosing between async, threading, or multiprocessing, you must ask:
👉 Is your task I/O-bound or CPU-bound?
Database calls
API requests
File reading/writing
Network operations
Data processing
Image manipulation
Machine learning
Heavy computations
Your choice depends entirely on this.
Async is built around:
async def
await
Event loop
asyncio
It works using cooperative multitasking.
When a task hits await, it pauses and allows another task to run.
It does NOT:
Create new threads
Use multiple CPU cores
It simply uses waiting time efficiently.
Example:
import asyncioasync def fetch_data():
await asyncio.sleep(2)
return "Data"async def main():
result = await fetch_data()
print(result)asyncio.run(main())Because of the Global Interpreter Lock (GIL), async will not speed up computation-heavy code.
Threading allows multiple threads inside one process.
But here’s the important part:
Python has something called the GIL (Global Interpreter Lock).
The GIL ensures that only one thread executes Python bytecode at a time.
That means:
👉 Threads are NOT truly parallel for CPU tasks.
Example:
import threading
import timedef task():
time.sleep(2)
print("Done")t = threading.Thread(target=task)
t.start()
t.join()Threads work well when tasks spend time waiting (network, DB, etc.).
Because the GIL prevents true parallel execution.
Multiprocessing creates separate processes.
Each process:
Has its own memory
Has its own Python interpreter
Has its own GIL
This means:
👉 True parallelism
👉 Multiple CPU cores utilized
Example:
from multiprocessing import Process
import timedef task():
time.sleep(2)
print("Done")p = Process(target=task)
p.start()
p.join()Higher memory usage
Slower startup
More complex inter-process communication
FeatureAsyncThreadingMultiprocessingUses multiple cores Good for I/O Good for CPU tasks Memory usageLowMediumHighComplexityMediumMediumHigher
Imagine you’re building a FastAPI service:
Use Async.
FastAPI is built on async. It scales beautifully for I/O.
Use Multiprocessing or background workers.
Async won’t help here.
Threading is acceptable.
Think of it like this:
Handles many tables while waiting for food.
But only one can cook at a time.
Now you’re cooking in parallel.
They try to fix slow CPU code using async.
Async improves scalability for waiting tasks.
It does NOT increase computational power.
If your task:
Waits a lot → Use async
Computes a lot → Use multiprocessing
Simple I/O concurrency → Threading is fine
Python gives you three powerful tools:
Async for scalable I/O
Threading for lightweight concurrency
Multiprocessing for true parallelism
Choosing the wrong one won’t just reduce performance —
it can completely break scalability under load.
Understanding the difference makes you a better backend engineer.
0
5
0