Skip to content

Commit 5a06a48

Browse files
committed
docs: add detailed README for Section 30 - Async IO with asyncio
⚡ Learn how to write asynchronous programs in Python using `async/await` and the `asyncio` module: - 🧠 Understand event loop and concurrency model - 📦 Define async functions with `async def` and use `await` to run them - 🔄 Schedule multiple tasks concurrently using `create_task()` and `gather()` - 🧰 Real-world example – concurrent web scraping using `aiohttp` and async file reading with `aiofiles` - ⏳ Use `asyncio.sleep()` to simulate I/O delays - 🧬 Integrate async with threads when needed - 💡 Hidden notes: - Always guard async code with `if __name__ == '__main__'` on Windows - Avoid CPU-bound work inside coroutines — use multiprocessing if needed - Prefer async libraries like `aiohttp`, `aiofiles`, `asyncpg` - Use timeouts with `wait_for()` to avoid hanging tasks - Don’t mix sync and async calls without proper tools
1 parent 28772ff commit 5a06a48

File tree

1 file changed

+330
-0
lines changed

1 file changed

+330
-0
lines changed

Section_30_Async_IO/README.md

Lines changed: 330 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,330 @@
1+
# 🧵 Section 30: Async IO (asyncio)
2+
## Writing Asynchronous Python Programs
3+
4+
**Learn how to write asynchronous programs using the `asyncio` module**, a powerful library for managing **I/O-bound tasks** like network calls, file handling, or system-level operations — all without blocking the main thread.
5+
6+
This section explains:
7+
- 🔍 What is asynchronous programming and when to use it
8+
- 📦 Define `async` functions and `await` results
9+
- 🔄 Run multiple coroutines concurrently
10+
- ⏳ Use `asyncio.sleep()` to simulate async I/O
11+
- 💡 Hidden notes and best practices for writing clean async code
12+
- 🧪 Real-world example – concurrent web requests with `aiohttp`
13+
14+
15+
16+
## 🧠 What You'll Learn
17+
18+
| Concept | Description |
19+
|--------|-------------|
20+
| **Asynchronous IO (`asyncio`)** | Handle many I/O-bound tasks in parallel |
21+
| **`async def`** | Define a coroutine that can be run asynchronously |
22+
| **`await`** | Wait for a coroutine to complete |
23+
| **`asyncio.run()`** | Start and manage an async event loop |
24+
| **`asyncio.create_task()`** | Schedule a coroutine to run in the event loop |
25+
| **`asyncio.gather()`** | Run multiple tasks and wait for all to finish |
26+
| **Best Practices** | When to use async over threads or processes |
27+
28+
29+
30+
## ⚙️ Introduction to Async IO in Python
31+
32+
Python’s `asyncio` module allows you to write **non-blocking** programs that perform **multiple I/O-bound tasks** at once — ideal for:
33+
- Web scraping
34+
- API clients
35+
- Network servers
36+
- Concurrent file reading/writing
37+
- Long-running background jobs
38+
39+
🔹 **Example – Basic Async Function**
40+
```python
41+
import asyncio
42+
43+
async def greet(name):
44+
print(f"Hello, {name}")
45+
await asyncio.sleep(1)
46+
print(f"Goodbye, {name}")
47+
48+
asyncio.run(greet("Alice"))
49+
```
50+
51+
🔸 The `await` keyword tells Python to pause execution until the awaited task completes.
52+
53+
54+
55+
## 📌 Define and Run Coroutines
56+
57+
### ✅ Defining Coroutines
58+
59+
Use `async def` to define a function that returns a **coroutine object** — not executed immediately.
60+
61+
🔹 **Example – Multiple Coroutines**
62+
```python
63+
async def task_one():
64+
print("Task One Started")
65+
await asyncio.sleep(2)
66+
print("Task One Completed")
67+
68+
async def task_two():
69+
print("Task Two Started")
70+
await asyncio.sleep(1)
71+
print("Task Two Completed")
72+
```
73+
74+
### ✅ Running Coroutines
75+
76+
You must run coroutines inside an event loop. Use `asyncio.run()` in Python 3.7+:
77+
78+
```python
79+
asyncio.run(task_one())
80+
```
81+
82+
Or run them concurrently using `create_task()`:
83+
84+
```python
85+
async def main():
86+
t1 = asyncio.create_task(task_one())
87+
t2 = asyncio.create_task(task_two())
88+
await t1
89+
await t2
90+
91+
asyncio.run(main())
92+
```
93+
94+
🔸 Output will show overlapping execution due to concurrency.
95+
96+
97+
## 🔄 Awaitable Objects
98+
99+
Any object that can be used in an `await` expression is called an **awaitable**.
100+
101+
They include:
102+
- Coroutines
103+
- Tasks (created via `create_task()`)
104+
- Futures (used internally by asyncio)
105+
106+
🔹 **Example – Using `await` on a Task**
107+
```python
108+
async def fetch_data():
109+
await asyncio.sleep(1)
110+
return "Data"
111+
112+
async def main():
113+
task = asyncio.create_task(fetch_data())
114+
result = await task
115+
print("Result:", result)
116+
117+
asyncio.run(main()) # Result: Data
118+
```
119+
120+
121+
122+
## 🧰 Real-World Example – Fetching from Multiple URLs
123+
124+
Let’s build a program that fetches data from multiple URLs concurrently using `aiohttp`.
125+
126+
### 🧱 Step 1: Install aiohttp
127+
```bash
128+
pip install aiohttp
129+
```
130+
131+
### 🛠️ Step 2: Define Coroutine for Fetching
132+
```python
133+
import asyncio
134+
import aiohttp
135+
136+
async def fetch_url(session, url):
137+
async with session.get(url) as response:
138+
content = await response.text()
139+
print(f"Fetched {url}{len(content)} bytes")
140+
return content
141+
142+
async def main(urls):
143+
async with aiohttp.ClientSession() as session:
144+
tasks = [fetch_url(session, url) for url in urls]
145+
results = await asyncio.gather(*tasks)
146+
print(f"Fetched {len(results)} pages.")
147+
148+
urls = [
149+
'https://example.com',
150+
'https://httpbin.org/get',
151+
'https://jsonplaceholder.typicode.com'
152+
]
153+
154+
asyncio.run(main(urls))
155+
```
156+
157+
🔸 This shows how to make **multiple HTTP requests concurrently** — faster than sequential or even threaded approaches.
158+
159+
160+
161+
## ⏳ Working with Delays and Timeouts
162+
163+
Use `await asyncio.sleep(n)` to simulate I/O delays.
164+
165+
🔹 **Example – Delayed Greeting**
166+
```python
167+
async def delayed_greet(name, delay):
168+
print(f"{name} is waiting {delay} seconds...")
169+
await asyncio.sleep(delay)
170+
print(f"Hello, {name}!")
171+
172+
async def main():
173+
asyncio.create_task(delayed_greet("Alice", 1))
174+
asyncio.create_task(delayed_greet("Bob", 2))
175+
asyncio.create_task(delayed_greet("Charlie", 0.5))
176+
await asyncio.sleep(3)
177+
178+
asyncio.run(main())
179+
```
180+
181+
🔸 Tasks run concurrently — Charlie finishes first, then Alice, then Bob.
182+
183+
184+
185+
## 🧩 Advanced Example – Concurrent File Reader
186+
187+
Read multiple files asynchronously using `aiofiles`.
188+
189+
### 📁 Step 1: Install aiofiles
190+
```bash
191+
pip install aiofiles
192+
```
193+
194+
### 📖 Step 2: Read Files Without Blocking
195+
```python
196+
import asyncio
197+
import aiofiles
198+
199+
async def read_file(filename):
200+
async with aiofiles.open(filename, mode='r') as f:
201+
content = await f.read()
202+
print(f"Read {filename}: {len(content)} characters")
203+
return content
204+
205+
async def main():
206+
files = ["file1.txt", "file2.txt", "file3.txt"]
207+
tasks = [read_file(f) for f in files]
208+
await asyncio.gather(*tasks)
209+
210+
asyncio.run(main())
211+
```
212+
213+
🔸 Each file is read without blocking others — great for batch processing large logs or config files.
214+
215+
216+
217+
## 🧯 Best Practices & Notes
218+
219+
| Practice | Description |
220+
|---------|-------------|
221+
| 🧠 Use `async/await` instead of callbacks | Cleaner and easier to maintain |
222+
| 📦 Prefer `asyncio.run()` in Python 3.7+ | It manages the event loop automatically |
223+
| 🔄 Use `create_task()` to schedule work | For true concurrency |
224+
| 🧾 Don’t mix sync and async calls | Can cause performance bottlenecks |
225+
| 🕒 Always use `await` with coroutines | Otherwise they won't execute |
226+
| 🧲 Use `gather()` to collect results from multiple tasks | Clean way to await all results |
227+
| 🧽 Avoid long-running CPU-bound logic inside coroutines | Use multiprocessing if needed |
228+
| 🧩 Use async libraries like `aiohttp`, `aiofiles`, `asyncpg` | Maximize async benefits |
229+
| 🧾 Add timeouts to prevent hanging | `await asyncio.wait_for(task, timeout=5)` |
230+
| 🧠 Use `async for` or `async with` when working with async iterators and context managers |
231+
232+
233+
234+
## 🧬 Using `asyncio` with Threads
235+
236+
Sometimes you need to integrate async with threading — for example, calling async code from a GUI app.
237+
238+
🔹 **Example – Run async code inside a thread**
239+
```python
240+
import asyncio
241+
import threading
242+
243+
def start_async_loop(loop):
244+
asyncio.set_event_loop(loop)
245+
loop.run_forever()
246+
247+
loop = asyncio.new_event_loop()
248+
thread = threading.Thread(target=start_async_loop, args=(loop,), daemon=True)
249+
thread.start()
250+
251+
async def add_to_queue(item):
252+
await asyncio.sleep(1)
253+
print(f"Processed {item}")
254+
255+
future = asyncio.run_coroutine_threadsafe(add_to_queue("test"), loop)
256+
future.result() # Wait for async result
257+
```
258+
259+
🔸 This pattern is useful in apps where the main thread runs a different loop (e.g., Tkinter or Flask).
260+
261+
262+
263+
## 🧪 Real-World Use Case – Async Database Query Pool
264+
265+
Let’s say you want to query multiple database endpoints in parallel.
266+
267+
### 🧱 Sample Async DB Query Function
268+
```python
269+
import asyncio
270+
import random
271+
272+
async def query_db(db_name):
273+
delay = random.uniform(0.5, 2)
274+
print(f"Querying {db_name} ({delay:.2f}s)")
275+
await asyncio.sleep(delay)
276+
print(f"Finished querying {db_name}")
277+
return f"Results from {db_name}"
278+
279+
async def main():
280+
db_list = ["users", "orders", "inventory", "logs"]
281+
tasks = [query_db(db) for db in db_list]
282+
results = await asyncio.gather(*tasks)
283+
for result in results:
284+
print(result)
285+
286+
asyncio.run(main())
287+
```
288+
289+
🔸 Simulates querying multiple databases concurrently — real use cases might involve actual async drivers like `asyncpg` or `motor`.
290+
291+
292+
293+
## 💡 Hidden Tips & Notes
294+
295+
- 🧠 On Windows, you may need to set the event loop policy:
296+
```python
297+
asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy())
298+
```
299+
- 🧱 Always wrap your top-level async logic inside `main()` and call `asyncio.run(main())`
300+
- 📦 Prefer `create_task()` over `ensure_future()` or manual scheduling
301+
- 🧾 Use `await asyncio.sleep(0)` to yield control to other coroutines (for cooperative multitasking)
302+
- 🧐 Use `asyncio.all_tasks()` and `asyncio.current_task()` for debugging running tasks
303+
- 🧵 Use `run_coroutine_threadsafe()` to interact with async code from threads
304+
- 🧩 Use `asyncio.Queue` for producer/consumer patterns within async
305+
- 🧾 Use `asyncio.TimeoutError` and `wait_for()` to avoid infinite waits
306+
307+
308+
309+
## 📌 Summary
310+
311+
| Feature | Purpose |
312+
|--------|---------|
313+
| `async def` | Define a coroutine |
314+
| `await` | Execute and wait for a coroutine |
315+
| `asyncio.run()` | Start and manage the event loop |
316+
| `create_task()` | Schedule a coroutine for execution |
317+
| `gather()` | Collect results from multiple tasks |
318+
| `sleep()` | Simulate I/O delays |
319+
| `aiohttp`, `aiofiles` | External async libraries for real-world use |
320+
| Best Practice | Don’t block inside coroutines — keep I/O non-blocking |
321+
322+
323+
324+
🎉 Congratulations! You now understand how to write **asynchronous programs in Python** using `asyncio`, including:
325+
- How to define and run coroutines
326+
- How to schedule and await multiple tasks
327+
- Real-world applications like web scraping and file reading
328+
- Best practices for avoiding blocking calls and managing timeouts
329+
330+
This completes our full roadmap from **Python fundamentals to advanced topics like OOP, metaprogramming, and concurrency models**!

0 commit comments

Comments
 (0)