Python线程管理

星哥玩云

发布于 2022-09-08 13:49:57

3810

发布于 2022-09-08 13:49:57

文章被收录于专栏：开源部署

一、线程

1、概念

线程是操作系统能够进行运算调度的最小单位。它被包含在进程之中，是进程中的实际运作单位。一条线程指的是进程中一个单一顺序的控制流，一个进程中可以并发多个线程，每条线程并行执行不同的任务。在Unix System V及SunOS中也被称为轻量进程（lightweight processes），但轻量进程更多指内核线程（kernel thread），而把用户线程（user thread）称为线程
多线程是指从软件或者硬件上实现多个线程并发执行的技术。具有多线程能力的计算机因有硬件支持而能够在同一时间执行多于一个线程，进而提升整体处理性能。具有这种能力的系统包括对称多处理机、多核心处理器以及芯片级多处理（Chip-level multithreading）或同时多线程（Simultaneous multithreading）处理器。
模块 _thread模块：低级模块 threading模块：高级模块，对_thread进行了封装

2、启动线程实现多任务

<span class="hljs-keyword">import</span> time
<span class="hljs-keyword">import</span> threading

<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">run1</span><span class="hljs-params">()</span>:</span>
                                      <span class="hljs-comment"># 线程名字</span>
    print(<span class="hljs-string">"启动%s子线程……"</span>%(threading.current_thread().name))
    <span class="hljs-keyword">for</span> i <span class="hljs-keyword">in</span> range(<span class="hljs-number">5</span>):
        print(<span class="hljs-string">"zutuanxue_com is a good man"</span>)
        time.sleep(<span class="hljs-number">1</span>)

<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">run2</span><span class="hljs-params">(name, word)</span>:</span>
    print(<span class="hljs-string">"启动%s子线程……"</span> % (threading.current_thread().name))
    <span class="hljs-keyword">for</span> i <span class="hljs-keyword">in</span> range(<span class="hljs-number">5</span>):
        print(<span class="hljs-string">"%s is a %s man"</span>%(name, word))
        time.sleep(<span class="hljs-number">1</span>)

<span class="hljs-keyword">if</span> __name__ == <span class="hljs-string">"__main__"</span>:
    t1 = time.clock()
    <span class="hljs-comment"># 主进程中默认有一个线程，称为主线程(父线程)</span>
    <span class="hljs-comment"># 主线程一般作为调度而存在，不具体实现业务逻辑</span>

    <span class="hljs-comment"># 创建子线程</span>
    <span class="hljs-comment"># name参数可以设置线程的名称，如果不设置按顺序设置为Thread-n</span>
    th1 = threading.Thread(target=run1, name=<span class="hljs-string">"th1"</span>)
    th2 = threading.Thread(target=run2, args=(<span class="hljs-string">"kaige"</span>, <span class="hljs-string">"nice"</span>))

    <span class="hljs-comment">#启动</span>
    th1.start()
    th2.start()

    <span class="hljs-comment">#等待子线程结束</span>
    th1.join()
    th2.join()

    t2 = time.clock()
    print(<span class="hljs-string">"耗时：%.2f"</span>%(t2-t1))

3、线程间共享数据

<span class="hljs-keyword">import</span> time
<span class="hljs-keyword">import</span> threading

money = <span class="hljs-number">0</span>

<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">run1</span><span class="hljs-params">()</span>:</span>
    <span class="hljs-keyword">global</span> money
    money = <span class="hljs-number">1</span>
    print(<span class="hljs-string">"run1-----------"</span>, money)
    print(<span class="hljs-string">"启动%s子线程……"</span>%(threading.current_thread().name))
    <span class="hljs-keyword">for</span> i <span class="hljs-keyword">in</span> range(<span class="hljs-number">5</span>):
        print(<span class="hljs-string">"zutuanxue_com is a good man"</span>)
        time.sleep(<span class="hljs-number">1</span>)

<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">run2</span><span class="hljs-params">(name, word)</span>:</span>
    print(<span class="hljs-string">"run2-----------"</span>, money)
    print(<span class="hljs-string">"启动%s子线程……"</span> % (threading.current_thread().name))
    <span class="hljs-keyword">for</span> i <span class="hljs-keyword">in</span> range(<span class="hljs-number">5</span>):
        print(<span class="hljs-string">"%s is a %s man"</span>%(name, word))
        time.sleep(<span class="hljs-number">1</span>)

<span class="hljs-keyword">if</span> __name__ == <span class="hljs-string">"__main__"</span>:
    t1 = time.clock()

    th1 = threading.Thread(target=run1, name=<span class="hljs-string">"th1"</span>)
    th2 = threading.Thread(target=run2, args=(<span class="hljs-string">"kaige"</span>, <span class="hljs-string">"nice"</span>))

    th1.start()
    th2.start()
    th1.join()
    th2.join()

    t2 = time.clock()
    print(<span class="hljs-string">"耗时：%.2f"</span>%(t2-t1))
    print(<span class="hljs-string">"main-----------"</span>, money)

说明：多线程和多进程的最大的不同在于多进程中同一个全局变量每个子进程各自有一份拷贝，相互不影响。而在多线程中，所有变量都由线程共享

注意：多个线程同时修改一个变量容易内存错乱，下节课我们将进行演示

4、多线程内存错乱问题

<span class="hljs-keyword">import</span> time
<span class="hljs-keyword">import</span> threading

money = <span class="hljs-number">0</span>

<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">change_money</span><span class="hljs-params">(n)</span>:</span>
    <span class="hljs-keyword">global</span> money
    money += n
    money -= n

<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">run1</span><span class="hljs-params">()</span>:</span>
    <span class="hljs-keyword">for</span> i <span class="hljs-keyword">in</span> range(<span class="hljs-number">1000000</span>):
        change_money(<span class="hljs-number">5</span>)

<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">run2</span><span class="hljs-params">()</span>:</span>
    <span class="hljs-keyword">for</span> i <span class="hljs-keyword">in</span> range(<span class="hljs-number">1000000</span>):
        change_money(<span class="hljs-number">9</span>)

<span class="hljs-keyword">if</span> __name__ == <span class="hljs-string">"__main__"</span>:
    t1 = time.clock()

    th1 = threading.Thread(target=run1)
    th2 = threading.Thread(target=run2)

    th1.start()
    th2.start()
    th1.join()
    th2.join()

    t2 = time.clock()
    print(<span class="hljs-string">"耗时：%.2f"</span>%(t2-t1))
    print(<span class="hljs-string">"main-----------"</span>, money)

问题：两个线程对同一数据同时进行读写，可能造成数据值的不对，我们必须保证一个线程在修改money时其他的线程一定不能修改，线程锁解决数据混乱问题

<span class="hljs-keyword">import</span> time
<span class="hljs-keyword">import</span> threading

<span class="hljs-comment">#锁对象</span>
lock = threading.Lock()

money = <span class="hljs-number">0</span>

<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">change_money</span><span class="hljs-params">(n)</span>:</span>
    <span class="hljs-keyword">global</span> money
    money += n
    money -= n

<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">run1</span><span class="hljs-params">()</span>:</span>
    <span class="hljs-keyword">for</span> i <span class="hljs-keyword">in</span> range(<span class="hljs-number">1000000</span>):
        <span class="hljs-comment">#获取线程锁，如果已上锁，则阻塞</span>
        lock.acquire()
        <span class="hljs-keyword">try</span>:
            change_money(<span class="hljs-number">5</span>)
        <span class="hljs-keyword">finally</span>:
            <span class="hljs-comment">#释放锁：千万要注意释放锁(自己的锁自己放)，否则会造成死锁</span>
            lock.release()

<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">run2</span><span class="hljs-params">()</span>:</span>
    <span class="hljs-keyword">for</span> i <span class="hljs-keyword">in</span> range(<span class="hljs-number">1000000</span>):
        <span class="hljs-comment">#简写，功能与上面一致</span>
        <span class="hljs-keyword">with</span> lock:
            change_money(<span class="hljs-number">9</span>)

<span class="hljs-keyword">if</span> __name__ == <span class="hljs-string">"__main__"</span>:
    t1 = time.clock()

    th1 = threading.Thread(target=run1)
    th2 = threading.Thread(target=run2)

    th1.start()
    th2.start()
    th1.join()
    th2.join()

    t2 = time.clock()
    print(<span class="hljs-string">"耗时：%.2f"</span>%(t2-t1))
    print(<span class="hljs-string">"main-----------"</span>, money)

5、Thread Local

得到一个ThreadLocal对象，可以为每个线程提供独立的存储空间，每个线程都可以对它进行读写操作，但是相互不影响

<span class="hljs-keyword">import</span> time
<span class="hljs-keyword">import</span> threading

local = threading.local()

<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">run1</span><span class="hljs-params">()</span>:</span>
    local.money = <span class="hljs-number">1</span>
    print(<span class="hljs-string">"run1-----------"</span>, local.money)
    time.sleep(<span class="hljs-number">1</span>)

<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">run2</span><span class="hljs-params">()</span>:</span>
    time.sleep(<span class="hljs-number">2</span>)
    local.money = <span class="hljs-number">2</span>
    print(<span class="hljs-string">"run2-------------"</span>, local.money)

<span class="hljs-keyword">if</span> __name__ == <span class="hljs-string">"__main__"</span>:
    t1 = time.clock()

    th1 = threading.Thread(target=run1)
    th2 = threading.Thread(target=run2)
    th1.start()
    th2.start()
    th1.join()
    th2.join()

    t2 = time.clock()
    print(<span class="hljs-string">"耗时：%.2f"</span>%(t2-t1))

6、封装线程

zutuanxue_com_thread.py

<span class="hljs-keyword">import</span> threading

<span class="hljs-class"><span class="hljs-keyword">class</span> <span class="hljs-title">Zutuanxue_comThread</span><span class="hljs-params">(threading.Thread)</span>:</span>
    <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">__init__</span><span class="hljs-params">(self, num)</span>:</span>
        super().__init__()
        self.num = num
    <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">run</span><span class="hljs-params">(self)</span>:</span>
        print(<span class="hljs-string">"-----------"</span>, self.num)

main.py

<span class="hljs-keyword">from</span> zutuanxue_com_thread <span class="hljs-keyword">import</span> Zutuanxue_comThread

<span class="hljs-keyword">if</span> __name__ == <span class="hljs-string">"__main__"</span>:
    th = Zutuanxue_comThread(<span class="hljs-number">1</span>)
    th.start()
    th.join()

7、线程通信

通信原理

<span class="hljs-keyword">import</span> threading

<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">run</span><span class="hljs-params">(event)</span>:</span>
    <span class="hljs-keyword">for</span> i <span class="hljs-keyword">in</span> range(<span class="hljs-number">10</span>):
        event.wait()
        print(<span class="hljs-string">"----------"</span>, i)
        event.clear()

<span class="hljs-keyword">if</span> __name__ == <span class="hljs-string">"__main__"</span>:
    event = threading.Event()

    th = threading.Thread(target=run, args=(event,))
    th.start()

    <span class="hljs-keyword">for</span> i <span class="hljs-keyword">in</span> range(<span class="hljs-number">10</span>):
        value = input()
        <span class="hljs-keyword">if</span> value == <span class="hljs-string">"y"</span>:
            <span class="hljs-comment"># 让线程运行一下</span>
            event.set()

    th.join()

生产者与消费者

<span class="hljs-keyword">import</span> threading
<span class="hljs-keyword">import</span> time

<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">product</span><span class="hljs-params">(q,e)</span>:</span>
    print(<span class="hljs-string">"启动生产子线程……"</span>)
    <span class="hljs-keyword">for</span> data <span class="hljs-keyword">in</span> [<span class="hljs-string">"good"</span>, <span class="hljs-string">"nice"</span>, <span class="hljs-string">"cool"</span>, <span class="hljs-string">"handsome"</span>]:
        time.sleep(<span class="hljs-number">2</span>)
        print(<span class="hljs-string">"生产出：%s"</span>%data)
        <span class="hljs-comment"># 将生产的数据写入队列</span>
        q.append(data)
        e.set()
    print(<span class="hljs-string">"发送结束信号"</span>)
    time.sleep(<span class="hljs-number">2</span>)
    e.set()
    print(<span class="hljs-string">"结束生产子线程……"</span>)

<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">customer</span><span class="hljs-params">(q,e)</span>:</span>
    print(<span class="hljs-string">"启动消费子线程……"</span>)
    <span class="hljs-keyword">while</span> <span class="hljs-number">1</span>:
        print(<span class="hljs-string">"等待生产者生产数据"</span>)
        e.wait()
        <span class="hljs-keyword">if</span> len(q) == <span class="hljs-number">0</span>:
            <span class="hljs-keyword">break</span>
        <span class="hljs-keyword">else</span>:
            value = q.pop()
        print(<span class="hljs-string">"消费者消费了%s数据"</span>%(value))
        e.clear()

    print(<span class="hljs-string">"结束消费子线程……"</span>)

<span class="hljs-keyword">if</span> __name__ == <span class="hljs-string">"__main__"</span>:
    q = []
    e = threading.Event()

    p1 = threading.Thread(target=product, args=(q,e))
    p2 = threading.Thread(target=customer, args=(q,e))
    p1.start()
    p2.start()
    p1.join()
    p2.join()

    print(<span class="hljs-string">"END"</span>)

二、进程VS线程

多任务的实现原理通常我们会设计Master-Worker模式，Master负责分配任务，Worker负责执行任务，因此，多任务环境下，通常是一个Master，多个Worker
多进程主进程就是Master，其他进程就是Worker
- 优点稳定性高：一个子进程崩溃了，不会影响主进程和其他子进程，当然主进程挂了所有进程就全挂了，但是Master进程只负责分配任务，挂掉的概率低
- 缺点创建进程的代价大：在Unix/Linux系统下，用fork调用还行，在Windows下创建进程开销巨大操作系统能同时运行的进程数也是有限的：在内存和CPU的限制下，如果有几千个进程同时运行，操作系统连调度都会成问题
多线程主线程就是Master，其他线程就是Worker
- 优点多线程模式通常比多进程快一点，但是也快不到哪去在Windows下，多线程的效率比多进程要高
- 缺点任何一个线程挂掉都可能直接造成整个进程崩溃：所有线程共享进程的内存。在Windows上，如果一个线程执行的代码出了问题，你经常可以看到这样的提示：“该程序执行了非法操作，即将关闭”，其实往往是某个线程出了问题，但是操作系统会强制结束整个进程
计算密集型 vs IO密集型
- 计算密集型要进行大量的计算，消耗CPU资源，比如计算圆周率、对视频进行高清解码等等，全靠CPU的运算能力。这种计算密集型任务虽然也可以用多任务完成，但是任务越多，花在任务切换的时间就越多，CPU执行任务的效率就越低，所以，要最高效地利用CPU，计算密集型任务同时进行的数量应当等于CPU的核心数
- IO密集型涉及到网络、磁盘IO的任务都是IO密集型任务，这类任务的特点是CPU消耗很少，任务的大部分时间都在等待IO操作完成（因为IO的速度远远低于CPU和内存的速度）。对于IO密集型任务，任务越多，CPU效率越高，但也有一个限度。常见的大部分任务都是IO密集型任务，比如Web应用

本文参与腾讯云自媒体同步曝光计划，分享自作者个人站点/博客。

如有侵权请联系 cloudcommunity@tencent.com 删除

windows