还在为动态网页反爬彻夜调试?还在为UI自动化反复编写XPath定位器?
当传统爬虫与测试工具遭遇瓶颈时,Browser Use + 大模型已悄然开启新范式——通过类人化决策链实现智能数据抓取与自适应UI测试,让代码拥有“人”的智慧:理解页面语义、模拟视觉判断、动态修正操作路径,真正突破规则化脚本的天花板!
今天给大家介绍一下Browser Use!
Browser Use 是一款开源Python库,专为大语言模型设计的智能浏览器工具,目的是让 AI 能够像人类一样自然地浏览和操作网页。它支持多标签页管理、视觉识别、内容提取,并能记录和重复执行特定动作。
Browser Use 还支持开发者自定义动作,如保存数据到数据库,文件等。支持多种主流的大型语言模型,如 DeepSeek,GPT-4 和 Claude等,并支持同时运行多个任务,具备自我修正功能,从而提高任务执行的准确性和效率。
1、安装python环境,python版本要求3.11及以上
2、安装browser use库
pip install browser-use
3、安装playwright
playwright install
一、爬取基金数据
llm = ChatOpenAI(
model='deepseek-chat',
api_key='*************',
base_url='https://api.deepseek.com',
temperature=0
)
asyncdefmain():
agent = Agent(
task="""
1、导航到网址:https://fund.eastmoney.com/
2、点击基金排行
3、返回排行前10的基金数据,以json格式返回
""",
llm=llm,
use_vision=False,
)
result = await agent.run()
print(result.final_result())
asyncio.run(main())
效果展示
1、导航到指定网址
2、点击基金排行
3、提取top 10的基金数据
DEBUG [browser_use] --act Execution time: 0.00 seconds
INFO [controller] 📄 Extracted from page
: ```json
{
"top_10_funds": [
{
"rank": 1,
"fund_code": "018124",
"fund_name": "永赢先进制造智选混合发起A",
"date": "03-07",
"unit_net_value": "2.1654",
"cumulative_net_value": "2.1654",
"daily_growth_rate": "2.21%",
"1_week": "9.15%",
"1_month": "17.58%",
"3_months": "64.06%",
"6_months": "191.13%",
"1_year": "115.94%",
"since_inception": "116.54%",
"handling_fee": "0.15%"
},
{
"rank": 2,
"fund_code": "018125",
"fund_name": "永赢先进制造智选混合发起C",
"date": "03-07",
"unit_net_value": "2.1501",
"cumulative_net_value": "2.1501",
"daily_growth_rate": "2.21%",
"1_week": "9.15%",
"1_month": "17.54%",
"3_months": "63.89%",
"6_months": "190.55%",
"1_year": "115.07%",
"since_inception": "115.01%",
"handling_fee": "0.00%"
},
{
"rank": 3,
"fund_code": "016530",
"fund_name": "鹏华碳中和主题混合A",
"date": "03-07",
"unit_net_value": "1.7881",
"cumulative_net_value": "1.7881",
"daily_growth_rate": "3.00%",
"1_week": "10.23%",
"1_month": "22.92%",
"3_months": "68.07%",
"6_months": "178.39%",
"1_year": "104.00%",
"since_inception": "78.81%",
"handling_fee": "0.15%"
},
{
"rank": 4,
"fund_code": "016531",
"fund_name": "鹏华碳中和主题混合C",
"date": "03-07",
"unit_net_value": "1.7685",
"cumulative_net_value": "1.7685",
"daily_growth_rate": "3.00%",
"1_week": "10.21%",
"1_month": "22.86%",
"3_months": "67.81%",
"6_months": "177.59%",
"1_year": "102.79%",
"since_inception": "76.85%",
"handling_fee": "0.00%"
},
{
"rank": 5,
"fund_code": "001970",
"fund_name": "泰信鑫选灵活配置混合A",
"date": "03-07",
"unit_net_value": "1.3310",
"cumulative_net_value": "1.3310",
"daily_growth_rate": "-1.04%",
"1_week": "7.25%",
"1_month": "4.80%",
"3_months": "31.00%",
"6_months": "125.59%",
"1_year": "95.45%",
"since_inception": "33.10%",
"handling_fee": "0.15%"
},
{
"rank": 6,
"fund_code": "002580",
"fund_name": "泰信鑫选灵活配置混合C",
"date": "03-07",
"unit_net_value": "1.3220",
"cumulative_net_value": "1.3220",
"daily_growth_rate": "-0.97%",
"1_week": "7.31%",
"1_month": "4.84%",
"3_months": "31.15%",
"6_months": "125.60%",
"1_year": "95.27%",
"since_inception": "31.67%",
"handling_fee": "0.00%"
},
{
"rank": 7,
"fund_code": "016295",
"fund_name": "新华利率债债券E",
"date": "03-07",
"unit_net_value": "1.7977",
"cumulative_net_value": "1.9906",
"daily_growth_rate": "-0.13%",
"1_week": "-0.06%",
"1_month": "-0.66%",
"3_months": "0.85%",
"6_months": "94.13%",
"1_year": "92.89%",
"since_inception": "99.00%",
"handling_fee": "0.00%"
},
{
"rank": 8,
"fund_code": "019457",
"fund_name": "平安先进制造主题股票发起A",
"date": "03-07",
"unit_net_value": "1.7593",
"cumulative_net_value": "1.7593",
"daily_growth_rate": "1.78%",
"1_week": "10.41%",
"1_month": "23.92%",
"3_months": "57.40%",
"6_months": "134.29%",
"1_year": "90.71%",
"since_inception": "75.93%",
"handling_fee": "0.15%"
},
{
"rank": 9,
"fund_code": "019458",
"fund_name": "平安先进制造主题股票发起C",
"date": "03-07",
"unit_net_value": "1.7452",
"cumulative_net_value": "1.7452",
"daily_growth_rate": "1.78%",
"1_week": "10.40%",
"1_month": "23.87%",
"3_months": "57.17%",
"6_months": "133.60%",
"1_year": "89.59%",
"since_inception": "74.52%",
"handling_fee": "0.00%"
},
{
"rank": 10,
"fund_code": "007713",
"fund_name": "华富科技动能混合A",
"date": "03-07",
"unit_net_value": "1.4600",
"cumulative_net_value": "1.5100",
"daily_growth_rate": "1.47%",
"1_week": "8.19%",
"1_month": "18.94%",
"3_months": "47.07%",
"6_months": "135.41%",
"1_year": "89.14%",
"since_inception": "51.93%",
"handling_fee": "0.15%"
}
]
}
二、结合pytest实现页面自动化测试
异步执行需要安装插件pytest-asyncio
pip install pytest-asyncio
@pytest.mark.asyncio
@pytest.mark.parametrize("username,password,expected", [("kevin@xxxx.com", "a123456", "kevin"),("kevin@xxxx.com", "123456", "账号密码输入错误")])
asyncdeftest_login(username, password, expected):
agent = Agent(
task=f"""
1、导航到网址:https://xxxxxxx.com
2、输入用户名:{username}, 密码:{password}
3、点击登录按钮
4、验证是否登录成功,登录成功返回{expected}
""",
llm=llm,
use_vision=False,
)
result = await agent.run()
assert expected in str(result.final_result())
效果展示
Browser Use 深度集成大语言模型(LLM),通过语义理解与视觉决策链实现零硬编码自动化,彻底颠覆传统脚本开发模式。
AI 自动解析页面结构、动态生成操作路径,无需人工编写 XPath/CSS 定位器,开发效率提升 5 倍以上,尤其擅长处理动态验证、反爬策略及多步骤交互场景,成为金融数据抓取、跨平台测试的新一代智能引擎。