《Pandas 1.x Cookbook · 第二版》第06章选取数据子集

SeanCheney

发布于 2021-03-02 15:27:56

3270

发布于 2021-03-02 15:27:56

6.1 选取Series数据

读取大学数据集，使用校名作为索行引：

>>> import pandas as pd
>>> import numpy as np
>>> college = pd.read_csv(
...     "data/college.csv", index_col="INSTNM"
... )
>>> city = college["CITY"]
>>> city
INSTNM
Alabama A & M University                                       Normal
University of Alabama at Birmingham                        Birmingham
Amridge University                                         Montgomery
University of Alabama in Huntsville                        Huntsville
Alabama State University                                   Montgomery
...
SAE Institute of Technology  San Francisco                 Emeryville
Rasmussen College - Overland Park                         Overland...
National Personal Training Institute of Cleveland         Highland...
Bay Area Medical Academy - San Jose Satellite Location       San Jose
Excel Learning Center-San Antonio South                   San Antonio
Name: CITY, Length: 7535, dtype: object

从Series选取标量值：

>>> city["Alabama A & M University"]
'Normal'

使用.loc提取标量值：

>>> city.loc["Alabama A & M University"]
'Normal'

使用.iloc提取标量值：

>>> city.iloc[0]
'Normal'

提取出多个值，如果传入的是列表，返回的是Series：

>>> city[
...     [
...         "Alabama A & M University",
...         "Alabama State University",
...     ]
... ]
INSTNM
Alabama A & M University        Normal
Alabama State University    Montgomery
Name: CITY, dtype: object

使用.loc重复上面的步骤：

>>> city.loc[
...     [
...         "Alabama A & M University",
...         "Alabama State University",
...     ]
... ]
INSTNM
Alabama A & M University        Normal
Alabama State University    Montgomery
Name: CITY, dtype: object

使用.iloc重复上面的步骤：

>>> city.iloc[[0, 4]]
INSTNM
Alabama A & M University        Normal
Alabama State University    Montgomery
Name: CITY, dtype: object

使用切片提取多值：

>>> city[
...     "Alabama A & M University":"Alabama State University"
... ]
INSTNM
Alabama A & M University                   Normal
University of Alabama at Birmingham    Birmingham
Amridge University                     Montgomery
University of Alabama in Huntsville    Huntsville
Alabama State University               Montgomery
Name: CITY, dtype: object

使用位置切片提取多值：

>>> city[0:5]
INSTNM
Alabama A & M University                   Normal
University of Alabama at Birmingham    Birmingham
Amridge University                     Montgomery
University of Alabama in Huntsville    Huntsville
Alabama State University               Montgomery
Name: CITY, dtype: object

使用布尔数组提取多值：

>>> alabama_mask = city.isin(["Birmingham", "Montgomery"])
>>> city[alabama_mask]
INSTNM
University of Alabama at Birmingham    Birmingham
Amridge University                     Montgomery
Alabama State University               Montgomery
Auburn University at Montgomery        Montgomery
Birmingham Southern College            Birmingham
                                          ...     
Fortis Institute-Birmingham            Birmingham
Hair Academy                           Montgomery
Brown Mackie College-Birmingham        Birmingham
Nunation School of Cosmetology         Birmingham
Troy University-Montgomery Campus      Montgomery
Name: CITY, Length: 26, dtype: object

使用.loc和.iloc在原始DataFrame操作也可以实现同样的目的：

>>> college.loc["Alabama A & M University", "CITY"]
'Normal'
>>> college.iloc[0, 0]
'Normal'
>>> college.loc[
...     [
...         "Alabama A & M University",
...         "Alabama State University",
...     ],
...     "CITY",
... ]
INSTNM
Alabama A & M University        Normal
Alabama State University    Montgomery
Name: CITY, dtype: object
>>> college.iloc[[0, 4], 0]
INSTNM
Alabama A & M University        Normal
Alabama State University    Montgomery
Name: CITY, dtype: object
>>> college.loc[
...     "Alabama A & M University":"Alabama State University",
...     "CITY",
... ]
INSTNM
Alabama A & M University                   Normal
University of Alabama at Birmingham    Birmingham
Amridge University                     Montgomery
University of Alabama in Huntsville    Huntsville
Alabama State University               Montgomery
Name: CITY, dtype: object
>>> college.iloc[0:5, 0]
INSTNM
Alabama A & M University                   Normal
University of Alabama at Birmingham    Birmingham
Amridge University                     Montgomery
University of Alabama in Huntsville    Huntsville
Alabama State University               Montgomery
Name: CITY, dtype: object

使用.loc切片时要注意，索引如果越界，会返回空值：

>>> city.loc[
...     "Reid State Technical College":"Alabama State University"
... ]
Series([], Name: CITY, dtype: object)

6.2 选取DataFrame行

这一节和上节有点像，还是先读取数据：

>>> college = pd.read_csv(
...     "data/college.csv", index_col="INSTNM"
... )
>>> college.sample(5, random_state=42)
                     CITY STABBR  ...  MD_EARN_WNE_P10  GRAD_DEBT_MDN_SUPP
INSTNM                            ...
Career Po...  San Antonio     TX  ...        20700            14977
Ner Israe...    Baltimore     MD  ...  PrivacyS...      PrivacyS...
Reflectio...      Decatur     IL  ...          NaN      PrivacyS...
Capital A...  Baton Rouge     LA  ...        26400      PrivacyS...
West Virg...   Montgomery     WV  ...        43400            23969
<BLANKLINE>
[5 rows x 26 columns]

使用.iloc提取一整行：

>>> college.iloc[60]
CITY                  Anchorage
STABBR                       AK
HBCU                          0
MENONLY                       0
WOMENONLY                     0
                        ...
PCTPELL                  0.2385
PCTFLOAN                 0.2647
UG25ABV                  0.4386
MD_EARN_WNE_P10           42500
GRAD_DEBT_MDN_SUPP      19449.5
Name: University of Alaska Anchorage, Length: 26, dtype: object

使用.loc实现上一步：

>>> college.loc["University of Alaska Anchorage"]
CITY                  Anchorage
STABBR                       AK
HBCU                          0
MENONLY                       0
WOMENONLY                     0
                        ...
PCTPELL                  0.2385
PCTFLOAN                 0.2647
UG25ABV                  0.4386
MD_EARN_WNE_P10           42500
GRAD_DEBT_MDN_SUPP      19449.5
Name: University of Alaska Anchorage, Length: 26, dtype: object

使用.iloc提取一组不连续的行：

>>> college.iloc[[60, 99, 3]]
                    CITY STABBR  ...  MD_EARN_WNE_P10  GRAD_DEBT_MDN_SUPP
INSTNM                           ...
Universit...   Anchorage     AK  ...        42500          19449.5
Internati...       Tempe     AZ  ...        22200            10556
Universit...  Huntsville     AL  ...        45500            24097
<BLANKLINE>
[3 rows x 26 columns]

使用.loc提取一组不连续的行：

>>> labels = [
...     "University of Alaska Anchorage",
...     "International Academy of Hair Design",
...     "University of Alabama in Huntsville",
... ]
>>> college.loc[labels]
                    CITY STABBR  ...  MD_EARN_WNE_P10  GRAD_DEBT_MDN_SUPP
INSTNM                           ...
Universit...   Anchorage     AK  ...        42500          19449.5
Internati...       Tempe     AZ  ...        22200            10556
Universit...  Huntsville     AL  ...        45500            24097
<BLANKLINE>
[3 rows x 26 columns]

使用.iloc提取一组连续的行：

>>> college.iloc[99:102]
                 CITY STABBR  ...  MD_EARN_WNE_P10  GRAD_DEBT_MDN_SUPP
INSTNM                        ...
Internati...    Tempe     AZ  ...        22200            10556
GateWay C...  Phoenix     AZ  ...        29800             7283
Mesa Comm...     Mesa     AZ  ...        35200             8000
<BLANKLINE>
[3 rows x 26 columns]

.loc的切片是包含起始和结束的索引的：

>>> start = "International Academy of Hair Design"
>>> stop = "Mesa Community College"
>>> college.loc[start:stop]
                 CITY STABBR  ...  MD_EARN_WNE_P10  GRAD_DEBT_MDN_SUPP
INSTNM                        ...
Internati...    Tempe     AZ  ...        22200            10556
GateWay C...  Phoenix     AZ  ...        29800             7283
Mesa Comm...     Mesa     AZ  ...        35200             8000
<BLANKLINE>
[3 rows x 26 columns]

将行索引的序号转变为字符串：

>>> college.iloc[[60, 99, 3]].index.tolist()
['University of Alaska Anchorage', 'International Academy of Hair Design', 'University of Alabama in Huntsville']

6.3 同时选取DataFrame的行和列

.iloc和.loc可以使用双切片，同时提取行和列：

>>> college = pd.read_csv(
...     "data/college.csv", index_col="INSTNM"
... )
>>> college.iloc[:3, :4]
                    CITY STABBR  HBCU  MENONLY
INSTNM                                        
Alabama A...      Normal     AL   1.0      0.0
Universit...  Birmingham     AL   0.0      0.0
Amridge U...  Montgomery     AL   0.0      0.0
>>> college.loc[:"Amridge University", :"MENONLY"]
                    CITY STABBR  HBCU  MENONLY
INSTNM                                        
Alabama A...      Normal     AL   1.0      0.0
Universit...  Birmingham     AL   0.0      0.0
Amridge U...  Montgomery     AL   0.0      0.0

选取不同两列的所有行：

>>> college.iloc[:, [4, 6]].head()
                                     WOMENONLY  SATVRMID
INSTNM
Alabama A & M University                   0.0     424.0
University of Alabama at Birmingham        0.0     570.0
Amridge University                         0.0       NaN
University of Alabama in Huntsville        0.0     595.0
Alabama State University                   0.0     425.0
>>> college.loc[:, ["WOMENONLY", "SATVRMID"]].head()
                                     WOMENONLY  SATVRMID
INSTNM
Alabama A & M University                   0.0     424.0
University of Alabama at Birmingham        0.0     570.0
Amridge University                         0.0       NaN
University of Alabama in Huntsville        0.0     595.0
Alabama State University                   0.0     425.0

选取不连续的行和列：

>>> college.iloc[[100, 200], [7, 15]]
                                       SATMTMID  UGDS_NHPI
INSTNM
GateWay Community College                   NaN     0.0029
American Baptist Seminary of the West       NaN        NaN
>>> rows = [
...     "GateWay Community College",
...     "American Baptist Seminary of the West",
... ]
>>> columns = ["SATMTMID", "UGDS_NHPI"]
>>> college.loc[rows, columns]
                                       SATMTMID  UGDS_NHPI
INSTNM
GateWay Community College                   NaN     0.0029
American Baptist Seminary of the West       NaN        NaN

选取一个标量值：

>>> college.iloc[5, -4]
0.401
>>> college.loc["The University of Alabama", "PCTFLOAN"]
0.401

选取单列，对行做切分：

>>> college.iloc[90:80:-2, 5]
INSTNM                              
Empire Beauty School-Flagstaff     0
Charles of Italy Beauty College    0
Central Arizona College            0
University of Arizona              0
Arizona State University-Tempe     0
Name: RELAFFIL, dtype: int64
>>> start = "Empire Beauty School-Flagstaff"
>>> stop = "Arizona State University-Tempe"
>>> college.loc[start:stop:-2, "RELAFFIL"]
INSTNM                              
Empire Beauty School-Flagstaff     0
Charles of Italy Beauty College    0
Central Arizona College            0
University of Arizona              0
Arizona State University-Tempe     0
Name: RELAFFIL, dtype: int64

下面两种操作等价：

college.iloc[:10]
college.iloc[:10, :]

6.4 用整数和标签选取数据

先读取数据：

>>> college = pd.read_csv(
...     "data/college.csv", index_col="INSTNM"
... )

使用.get_loc找到某一列的序号：

>>> col_start = college.columns.get_loc("UGDS_WHITE")
>>> col_end = college.columns.get_loc("UGDS_UNKN") + 1
>>> col_start, col_end
(10, 19)

用col_start和col_end选取列：

>>> college.iloc[:5, col_start:col_end]
              UGDS_WHITE  UGDS_BLACK  ...  UGDS_NRA  UGDS_UNKN
INSTNM                                ...                     
Alabama A...      0.0333      0.9353  ...    0.0059     0.0138
Universit...      0.5922      0.2600  ...    0.0179     0.0100
Amridge U...      0.2990      0.4192  ...    0.0000     0.2715
Universit...      0.6988      0.1255  ...    0.0332     0.0350
Alabama S...      0.0158      0.9208  ...    0.0243     0.0137
<BLANKLINE>
[5 rows x 9 columns]

行索引切片提取多行多列：

>>> row_start = college.index[10]
>>> row_end = college.index[15]
>>> college.loc[row_start:row_end, "UGDS_WHITE":"UGDS_UNKN"]
              UGDS_WHITE  UGDS_BLACK  ...  UGDS_NRA  UGDS_UNKN
INSTNM                                ...                     
Birmingha...      0.7983      0.1102  ...    0.0000     0.0051
Chattahoo...      0.4661      0.4372  ...    0.0000     0.0139
Concordia...      0.0280      0.8758  ...    0.0466     0.0000
South Uni...      0.3046      0.6054  ...    0.0019     0.0326
Enterpris...      0.6408      0.2435  ...    0.0012     0.0069
James H F...      0.6979      0.2259  ...    0.0007     0.0009
<BLANKLINE>
[6 rows x 9 columns]

6.5 按字母顺序切分

先读取数据：

>>> college = pd.read_csv(
...     "data/college.csv", index_col="INSTNM"
... )

尝试选取Sp和Su之间的学校：

>>> college.loc["Sp":"Su"]
Traceback (most recent call last):
  ...
ValueError: index must be monotonic increasing or decreasing
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  ...
KeyError: 'Sp'

报错是因为索引没有排序，对索引做排序：

>>> college = college.sort_index()

重复一开始的操作：

>>> college.loc["Sp":"Su"]
                    CITY STABBR  ...  MD_EARN_WNE_P10  GRAD_DEBT_MDN_SUPP
INSTNM                           ...
Spa Tech ...     Ipswich     MA  ...        21500             6333
Spa Tech ...    Plymouth     MA  ...        21500             6333
Spa Tech ...    Westboro     MA  ...        21500             6333
Spa Tech ...   Westbrook     ME  ...        21500             6333
Spalding ...  Louisville     KY  ...        41700            25000
...                  ...    ...  ...          ...              ...
Studio Ac...    Chandler     AZ  ...          NaN             6333
Studio Je...    New York     NY  ...  PrivacyS...      PrivacyS...
Stylemast...    Longview     WA  ...        17000            13320
Styles an...      Selmer     TN  ...  PrivacyS...      PrivacyS...
Styletren...   Rock Hill     SC  ...  PrivacyS...           9495.5
<BLANKLINE>
[201 rows x 26 columns]

用.is_monotonic_increasing或is_monotonic_decreasing判断索引是否是单调排序的：

>>> college = college.sort_index(ascending=False)
>>> college.index.is_monotonic_decreasing
True
>>> college.loc["E":"B"]
                                                  CITY  ...
INSTNM                                                  ...
Dyersburg State Community College            Dyersburg  ...
Dutchess Community College                Poughkeepsie  ...
Dutchess BOCES-Practical Nursing Program  Poughkeepsie  ...
Durham Technical Community College              Durham  ...
Durham Beauty Academy                           Durham  ...
...                                                ...  ...
Bacone College                                Muskogee  ...
Babson College                               Wellesley  ...
BJ's Beauty & Barber College                    Auburn  ...
BIR Training Center                            Chicago  ...
B M Spurr School of Practical Nursing        Glen Dale  ...

本文参与腾讯云自媒体同步曝光计划，分享自作者个人站点/博客。

如有侵权请联系 cloudcommunity@tencent.com 删除

python

本文分享自作者个人站点/博客前往查看

如有侵权，请联系 cloudcommunity@tencent.com 删除。

本文参与腾讯云自媒体同步曝光计划，欢迎热爱写作的你一起参与！

python

登录后参与评论

0 条评论

热度