Skip to content
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.

Commit 8d0ef46

Browse files
committedFeb 20, 2021
rename; add coroutine
1 parent d6f6bfc commit 8d0ef46

21 files changed

+651
-993
lines changed
 

‎ch01.md

Lines changed: 0 additions & 205 deletions
This file was deleted.

‎ch06.md

Lines changed: 0 additions & 4 deletions
This file was deleted.

‎ch07.md

Lines changed: 0 additions & 155 deletions
This file was deleted.

‎ch08.md

Lines changed: 0 additions & 127 deletions
This file was deleted.

‎ch09.md

Lines changed: 0 additions & 11 deletions
This file was deleted.

‎ch10.md

Lines changed: 0 additions & 183 deletions
This file was deleted.

‎ch11.md

Lines changed: 0 additions & 302 deletions
This file was deleted.

‎ch14.md renamed to ‎import.md

File renamed without changes.

‎opcache.md

Whitespace-only changes.

‎threadlocal.md

Lines changed: 129 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,129 @@
1+
# Thread Local 的实现原理
2+
3+
Python 标准库中有一个模块[_threading_local.py](Lib/_threading_local.py), 使用纯 Pyhton 实现了 thread local. 代码比较少, 复制了一份:
4+
5+
```py
6+
from weakref import ref
7+
from contextlib import contextmanager
8+
9+
__all__ = ["local"]
10+
11+
# We need to use objects from the threading module, but the threading
12+
# module may also want to use our `local` class, if support for locals
13+
# isn't compiled in to the `thread` module. This creates potential problems
14+
# with circular imports. For that reason, we don't import `threading`
15+
# until the bottom of this file (a hack sufficient to worm around the
16+
# potential problems). Note that all platforms on CPython do have support
17+
# for locals in the `thread` module, and there is no circular import problem
18+
# then, so problems introduced by fiddling the order of imports here won't
19+
# manifest.
20+
21+
class _localimpl:
22+
"""A class managing thread-local dicts"""
23+
__slots__ = 'key', 'dicts', 'localargs', 'locallock', '__weakref__'
24+
25+
def __init__(self):
26+
# The key used in the Thread objects' attribute dicts.
27+
# We keep it a string for speed but make it unlikely to clash with
28+
# a "real" attribute.
29+
self.key = '_threading_local._localimpl.' + str(id(self))
30+
# { id(Thread) -> (ref(Thread), thread-local dict) }
31+
self.dicts = {}
32+
33+
def get_dict(self):
34+
"""Return the dict for the current thread. Raises KeyError if none
35+
defined."""
36+
thread = current_thread()
37+
return self.dicts[id(thread)][1]
38+
39+
def create_dict(self):
40+
"""Create a new dict for the current thread, and return it."""
41+
localdict = {}
42+
key = self.key
43+
thread = current_thread()
44+
idt = id(thread)
45+
def local_deleted(_, key=key):
46+
# When the localimpl is deleted, remove the thread attribute.
47+
thread = wrthread()
48+
if thread is not None:
49+
del thread.__dict__[key]
50+
def thread_deleted(_, idt=idt):
51+
# When the thread is deleted, remove the local dict.
52+
# Note that this is suboptimal if the thread object gets
53+
# caught in a reference loop. We would like to be called
54+
# as soon as the OS-level thread ends instead.
55+
local = wrlocal()
56+
if local is not None:
57+
dct = local.dicts.pop(idt)
58+
# 这里使用弱引用是为了避免循环引用
59+
# 在 dicts 中记录每个线程是为了在 thread local 被销毁时, 方便删除线程中引用的 thread local
60+
# 在线程中记录 thread local 是为了在线程被销毁时, 方便删除 thread local 中引用的线程.
61+
# 而这些又都是通过弱引用的 callback 来实现的.
62+
# 这个实现思路非常值得学习.
63+
wrlocal = ref(self, local_deleted)
64+
wrthread = ref(thread, thread_deleted)
65+
thread.__dict__[key] = wrlocal
66+
self.dicts[idt] = wrthread, localdict
67+
return localdict
68+
69+
70+
@contextmanager
71+
def _patch(self):
72+
impl = object.__getattribute__(self, '_local__impl')
73+
try:
74+
dct = impl.get_dict()
75+
except KeyError:
76+
dct = impl.create_dict()
77+
args, kw = impl.localargs
78+
self.__init__(*args, **kw)
79+
with impl.locallock:
80+
object.__setattr__(self, '__dict__', dct)
81+
yield
82+
83+
84+
class local:
85+
__slots__ = '_local__impl', '__dict__'
86+
87+
def __new__(cls, /, *args, **kw):
88+
if (args or kw) and (cls.__init__ is object.__init__):
89+
raise TypeError("Initialization arguments are not supported")
90+
self = object.__new__(cls)
91+
impl = _localimpl()
92+
impl.localargs = (args, kw)
93+
impl.locallock = RLock()
94+
object.__setattr__(self, '_local__impl', impl)
95+
# We need to create the thread dict in anticipation of
96+
# __init__ being called, to make sure we don't call it
97+
# again ourselves.
98+
impl.create_dict()
99+
return self
100+
101+
def __getattribute__(self, name):
102+
with _patch(self):
103+
return object.__getattribute__(self, name)
104+
105+
def __setattr__(self, name, value):
106+
if name == '__dict__':
107+
raise AttributeError(
108+
"%r object attribute '__dict__' is read-only"
109+
% self.__class__.__name__)
110+
with _patch(self):
111+
return object.__setattr__(self, name, value)
112+
113+
def __delattr__(self, name):
114+
if name == '__dict__':
115+
raise AttributeError(
116+
"%r object attribute '__dict__' is read-only"
117+
% self.__class__.__name__)
118+
with _patch(self):
119+
return object.__delattr__(self, name)
120+
121+
122+
from threading import current_thread, RLock
123+
```
124+
125+
上面这段代码的有些细节有点不太好理解, 不过基本实现原理还是比较容易理解的. [最早的实现](https://github.com/python/cpython/commit/d15dc06df062fdf0fe8badec2982c6c5e0e28eb0)非常简单, 有助于帮助理解.
126+
127+
当然, thread local 也有 C 语言的实现, 具体的代码在[_threadmodule.c](https://github.com/ausaki/cpython/tree/v3.9.notes/Modules/_threadmodule.c), 里面有我写的一些注释.
128+
129+
和使用纯 Python 实现的 thread local 不同的是, 使用 C 实现不需要使用锁, 因为这些操作已经被 GIL 保护了.

‎ch16.md renamed to ‎内存管理.md

File renamed without changes.

‎ch04.md renamed to ‎列表.md

File renamed without changes.
File renamed without changes.

‎协程.md

Lines changed: 516 additions & 6 deletions
Large diffs are not rendered by default.

‎ch15.md renamed to ‎多线程.md

File renamed without changes.

‎ch05.md renamed to ‎字典.md

File renamed without changes.

‎ch03.md renamed to ‎字符串.md

File renamed without changes.

‎ch02.md renamed to ‎整数.md

File renamed without changes.

‎ch12.md renamed to ‎类型系统.md

File renamed without changes.

‎虚拟机.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -542,3 +542,9 @@ sys 模块增加了很多设置 hook 的方法, 具体可以查看 [sys 文档](
542542
```
543543

544544

545+
## 总结
546+
547+
548+
一个用 Python 写的 Python 解释器:
549+
550+
- [a-python-interpreter-written-in-python](http://aosabook.org/en/500L/a-python-interpreter-written-in-python.html)

0 commit comments

Comments
 (0)
Please sign in to comment.