threading 模块实现多线程。Thread 类创建和管理。import threadingimport timedef worker(num): print(f"线程 {num} 开始工作") time.sleep(2) print(f"线程 {num} 结束工作")# 创建线程thread1 = threading.Thread(target=worker, args=(1,))thread2 = threading.Thread(target=worker, args=(2,))# 启动线程thread1.start()thread2.start()# 等待线程结束thread1.join()thread2.join()print("所有线程完成")php331 Bytes© 菜鸟-创作你的创作多线程同时访问共享资源可能导致数据竞争,需用同步机制。
import threadinglock = threading.Lock()counter = 0def task(): global counter for _ in range(1000): lock.acquire() counter += 1 lock.release()threads = [threading.Thread(target=task) for _ in range(10)]for t in threads: t.start()for t in threads: t.join()print(f"最终计数值: {counter}")php331 Bytes© 菜鸟-创作你的创作with 语句更优雅def task(): global counter for _ in range(1000): with lock: counter += 1php100 Bytes© 菜鸟-创作你的创作方法 | 作用 |
|---|---|
start() | 启动线程 |
join() | 等待线程结束 |
is_alive() | 判断线程是否存活 |
setDaemon() | 设置守护线程,主线程结束自动结束子线程 |
Python 3.2+ 提供 concurrent.futures 模块,简化线程池管理:
from concurrent.futures import ThreadPoolExecutorimport timedef worker(num): print(f"线程 {num} 开始") time.sleep(1) return f"线程 {num} 结束"with ThreadPoolExecutor(max_workers=3) as executor: futures = [executor.submit(worker, i) for i in range(5)] for future in futures: print(future.result())php321 Bytes© 菜鸟-创作你的创作multiprocessing模块)。好的!下面给你一个Python多线程爬虫的完整示例,演示如何用多线程并发请求多个网页,并收集结果。示例包含线程创建、同步、线程池使用等内容,适合实战参考。
import threadingimport requestsfrom queue import Queuefrom concurrent.futures import ThreadPoolExecutor# 模拟的目标网址列表(示例用,可替换为真实网址)urls = [ "https://httpbin.org/get?query=1", "https://httpbin.org/get?query=2", "https://httpbin.org/get?query=3", "https://httpbin.org/get?query=4", "https://httpbin.org/get?query=5",]# 用于保存抓取结果的字典,线程安全用锁保护results = {}lock = threading.Lock()def fetch_url(url): """请求指定URL,返回状态码和内容长度""" try: response = requests.get(url, timeout=5) content_length = len(response.content) with lock: results[url] = (response.status_code, content_length) print(f"[完成] {url} 状态码: {response.status_code}, 内容长度: {content_length}") except Exception as e: with lock: results[url] = (None, None) print(f"[错误] {url} 请求失败,原因:{e}")def thread_pool_crawler(url_list): """使用线程池爬取URL列表""" max_workers = 3 # 最大线程数 with ThreadPoolExecutor(max_workers=max_workers) as executor: executor.map(fetch_url, url_list)if __name__ == "__main__": print("开始多线程爬虫任务...") thread_pool_crawler(urls) print("\n所有任务完成,结果汇总:") for url, (status, length) in results.items(): print(f"{url} -> 状态码: {status}, 内容长度: {length}")php1.23 KB© 菜鸟-创作你的创作fetch_url:具体爬取任务,发送请求并将结果写入共享字典,使用锁保证线程安全。thread_pool_crawler:使用 ThreadPoolExecutor 创建线程池,自动管理线程创建与销毁,调用 fetch_url 并发爬取。urls:待爬取的URL列表,可以替换为你实际目标。pip install requestsphp20 Bytes© 菜鸟-创作你的创作https://www.52runoob.com/archives/5704
原创声明:本文系作者授权腾讯云开发者社区发表,未经许可,不得转载。
如有侵权,请联系 cloudcommunity@tencent.com 删除。
原创声明:本文系作者授权腾讯云开发者社区发表,未经许可,不得转载。
如有侵权,请联系 cloudcommunity@tencent.com 删除。