问在CSS类中提取文本
EN

Stack Overflow用户

提问于 2018-04-01 05:38:46

回答 2查看 347关注 0票数 1

试图从网页提取数据到表。例如：

Block Number XXX
Building Name YYY
Street Name zzz
Pin Code 123456789

我试着用这个代码以表格的形式得到公司的所有细节.

html_doc='https://s3.amazonaws.com/todel162/test.html'

from urllib.request import urlopen
from bs4 import BeautifulSoup

soup = BeautifulSoup(urlopen(html_doc), 'html.parser')

mydivs = soup.findAll("div", {"class": "row"})

for i in mydivs:
    for x in i.findAll("div", {"class": "col-md-3 col-sm-3"}):
        print (x)
        print ('new x')

但我不知道如何提取特定CSS类的HTML标记中的文本。

python

pandas

beautifulsoup

即时通信IM，高可靠强安全，1折起

多平台跨终端互通，低代码 UI 组件助您30分钟集成单聊、群聊、推送等能力，适用于直播互动、客服咨询等多种场景

回答 2

Stack Overflow用户

回答已采纳

发布于 2018-04-01 22:40:10

您需要的所有项目都在<div class="col-md-3 col-sm-3">标记中。并且，除了General以外的所有项目都遵循特定的格式:第一个div标记是标签，第二个是相应的文本。因此，您可以简单地找到它们，然后交替打印它们作为标签和它的文本。

因此，除了通用信息之外，其他所有内容都可以在<div id="fldFirm">标记中获得。由于通用信息不遵循上述格式，因此您必须先逐个对其进行刮除。您可以使用limit=2获取前两个标记。

import requests
from bs4 import BeautifulSoup

r = requests.get('https://s3.amazonaws.com/todel162/test.html')
soup = BeautifulSoup(r.text, 'lxml')

gen_info = [x.text.strip() for x in soup.find('div', id='divInfoType').find_all('div', class_='col-md-3 col-sm-3', limit=2)]
print('{}: {}'.format(gen_info[0], gen_info[1]))

all_info = [x.text.strip() for x in soup.find('div', id='fldFirm').find_all('div', class_='col-md-3 col-sm-3')]

for i in range(0, len(all_info), 2):
    print('{}: {}'.format(all_info[i], all_info[i+1]))

输出：

Information Type: Other Than Individual
Name: Bellissimo Developers Thane Private Limited (Lodha Developers Pvt Ltd)
Organization Type: Company
Description For Other Type Organization: NA
Do you have any Past Experience ?: No
Block Number: 412, Floor 4 17G
Building Name: Vardhaman Chamber
Street Name: Cawasji Patel Road
Locality: Fort
Land mark: Horniman Circle
State/UT: MAHARASHTRA
Division: Konkan
District: Mumbai City
Taluka: Mumbai City
Village: Mumbai City
Pin Code: 400001
Office Number: 02261334263
Website URL: www.lodhagroup.com

编辑：

要创建一个dataframe，可以使用以下内容：

gen_info = [x.text.strip() for x in soup.find('div', id='divInfoType').find_all('div', class_='col-md-3 col-sm-3', limit=2)]
all_info = gen_info + [x.text.strip() for x in soup.find('div', id='fldFirm').find_all('div', class_='col-md-3 col-sm-3')]
df = pd.DataFrame({'A': all_info[0::2], 'B': all_info[1::2]})

票数 1

Stack Overflow用户

发布于 2018-04-01 06:30:15

你只是想要地址信息还是整个列表？这是整个清单的代码。

html_doc='https://s3.amazonaws.com/todel162/test.html'

from urllib.request import urlopen
from bs4 import BeautifulSoup
import re

soup = BeautifulSoup(urlopen(html_doc), 'html.parser')

mydivs = soup.findAll("div", {"class": "form-group"})

for x in mydivs:
    print (re.sub( '\s+', ' ', x.text ).strip())
    print ('#############')

产出：

Information Type Other Than Individual
#############
Name Bellissimo Developers Thane Private Limited (Lodha Developers Pvt Ltd)
#############
Organization Type Company
#############
Do you have any Past Experience ? No
#############
Block Number 412, Floor 4 17G Building Name Vardhaman Chamber
#############
Street Name Cawasji Patel Road Locality Fort
#############
Land mark Horniman Circle State/UT MAHARASHTRA
#############
Division Konkan District Mumbai City
#############
Taluka Mumbai City Village Mumbai City
#############
Pin Code 400001
#############
Office Number 02261334263
#############
Website URL www.lodhagroup.com
#############

如果您想在表中使用它(非常简单的例子)：

...
mydivs = soup.findAll("div", {"class": "form-group"})
print ("<table>")
for x in mydivs:
    label = x.find('label')
    if label is not None:
        print ("<tr><td>" + label.text + "</td><td> ")
        label.extract()
    else:
        print ("<tr><td></td><td> ")
    print (re.sub( '\s+', ' ', x.text ).strip())
    print ('</td></tr>')
print ("</table>")

产出：

<table>
<tr><td>Information Type</td><td> 
Other Than Individual
</td></tr>
<tr><td>Name</td><td> 
Bellissimo Developers Thane Private Limited (Lodha Developers Pvt Ltd)
</td></tr>
<tr><td>Organization Type</td><td> 
Company
</td></tr>
<tr><td></td><td> 
Do you have any Past Experience ? No
</td></tr>
<tr><td>Block Number</td><td> 
412, Floor 4 17G Building Name Vardhaman Chamber
</td></tr>
<tr><td>Street Name</td><td> 
Cawasji Patel Road Locality Fort
</td></tr>
<tr><td>Land mark</td><td> 
Horniman Circle State/UT MAHARASHTRA
</td></tr>
<tr><td>Division</td><td> 
Konkan District Mumbai City
</td></tr>
<tr><td>Taluka</td><td> 
Mumbai City Village Mumbai City
</td></tr>
<tr><td>Pin Code</td><td> 
400001
</td></tr>
<tr><td>Office Number</td><td> 
02261334263
</td></tr>
<tr><td>Website URL</td><td> 
www.lodhagroup.com
</td></tr>
</table>