学习分享｜初入Anaconda3以及实践
原创
简介： Anaconda，中文大蟒蛇，是一个开源的Python发行版本，其包含了conda、Python等180多个科学包及其依赖项。
Anaconda3
Anaconda，中文大蟒蛇，是一个开源的Python发行版本，其包含了conda、Python等180多个科学包及其依赖项。我当初安装这个是为了学习人工智能和算法。
1、安装
1、命令操作
安装好之后，可以使用命令conda，可以使用一系列命令参数，conda --help 或 conda -h 、 conda --version 或conda -V
例如，使用conda info命令如下所示信息：
$xxx:~ xxx$ conda info
active environment : None
shell level : 0
user config file : /Users/xxx/.condarc
populated config files : /Users/xxx/.condarc
conda version : 4.8.2
conda-build version : 3.18.11
python version : 3.7.6.final.0
virtual packages : __osx=10.15.7
base environment : /Users/xxx/opt/anaconda3 (writable)
channel URLs : https://repo.anaconda.com/pkgs/main/osx-64
https://repo.anaconda.com/pkgs/main/noarch
https://repo.anaconda.com/pkgs/r/osx-64
https://repo.anaconda.com/pkgs/r/noarch
package cache : /Users/xxx/opt/anaconda3/pkgs
/Users/xxx/.conda/pkgs
envs directories : /Users/xxx/opt/anaconda3/envs
/Users/xxx/.conda/envs
platform : osx-64
user-agent : conda/4.8.2 requests/2.22.0 CPython/3.7.6 Darwin/19.6.0 OSX/10.15.7
UID:GID : 501:20
netrc file : None
offline mode : False2、导出所有存在依赖
1）导出所有依赖包到记事本
conda list -e > [fileName(.txt)]
例如，conda list -e > /usr/local/data/conda-lib-98.txt
1、Kafka 集群防火墙
2、运行情况，或者执行命令，查看status
3、手动执行消费者或生产者，订阅消息，查看消费情况
问题记录FQA
Q1、ModuleNotFoundError: No module named 'pymysql'
Q1：执行python程序，出现 ``ModuleNotFoundError: No module named 'pymysql'
A1：
1）查找确认依赖库
初步判定是安装阿naconda3缺少pymysql依赖包，执行命令`` conda list `` 分别查看85服务器和96服务器anaconda3的已经安装插件库，依赖包如下所示，85服务器已安装插件库295个插件，96服务器已安装插件库288个插件
2）查找、下载依赖库
85插件所在路径， /data/_software/_install_pack/_python/，anaconda3的home路径，/data/_software/python/anaconda3/
86插件所在路径，， /data/_software/_pack/_python/，anaconda3的home路径，/data/_software/python/anaconda3/
96插件所在路径，/data/_software/_package/，anaconda3的home路径，/data/_software/python/anaconda3/
经过插件列表比对并排查，发现96服务器缺失插件列表如下所示
85服务器有whl插件源件：
- PyNaCl-1.3.0-cp34-abi3-manylinux1_x86_64.whl
- bcrypt-3.1.7-cp34-abi3-manylinux1_x86_64.whl
- confluent_kafka-1.1.0-cp37-cp37m-manylinux1_x86_64.whl
- paramiko-2.6.0-py2.py3-none-any.whl
- PyMySQL-0.9.3-py2.py3-none-any.whl
- cx_Oracle-8.0.0-cp37-cp37m-manylinux1_x86_64.whl
85服务器未找到whl插件源件（其中avro-python3使用python安装，cx-oracle使用pip安装）：
- avro-python3=1.8.2=pypi_0
- cx-oracle=8.0.0=pypi_0
先下载所需要的插件包，whl格式本质上是一个压缩包，里面包含了py文件，以及经过编译的pyd文件，使得可以在不具备编译环境的情况下，选择合适自己的python环境进行安装，简单来说，适合内网环境。
其次有些时候在内网环境中并不可以使用pip命令来下载依赖包，这就出现了需要手动安装依赖包的情况，由于网络不通，采取手动安装插件库，查找85服务器是否存在插件库或网络下载插件库再传给96服务器。经过检查，发现原来85服务器已经下载插件。在85服务器找到所依赖的插件，打包上传到96服务器。
3）安装依赖库
安装插件的命令使用pip，此命令在已经内置在anaconda3的依赖插件中，具体位置为home/lib/python3.7/site-packages。
- 安装已有whl文件
安装bcrypt-3.1.7-cp34-abi3-manylinux1_x86_64.whl
$ pip install bcrypt-3.1.7-cp34-abi3-manylinux1_x86_64.whl
Processing ./bcrypt-3.1.7-cp34-abi3-manylinux1_x86_64.whl
Requirement already satisfied: cffi>=1.1 in /data/_software/python/anaconda3/lib/python3.7/site-packages (from bcrypt==3.1.7) (1.12.3)
Requirement already satisfied: six>=1.4.1 in /data/_software/python/anaconda3/lib/python3.7/site-packages (from bcrypt==3.1.7) (1.12.0)
Requirement already satisfied: pycparser in /data/_software/python/anaconda3/lib/python3.7/site-packages (from cffi>=1.1->bcrypt==3.1.7) (2.19)
Installing collected packages: bcrypt
Successfully installed bcrypt-3.1.7
$ pip install confluent_kafka-1.1.0-cp37-cp37m-manylinux1_x86_64.whl
Processing ./confluent_kafka-1.1.0-cp37-cp37m-manylinux1_x86_64.whl
Installing collected packages: confluent-kafka
Successfully installed confluent-kafka-1.1.0安装PyNaCl-1.3.0-cp34-abi3-manylinux1_x86_64.whl
先安装PyNaCl，再安装paramiko
$ pip install PyNaCl-1.3.0-cp34-abi3-manylinux1_x86_64.whl
Processing ./PyNaCl-1.3.0-cp34-abi3-manylinux1_x86_64.whl
Requirement already satisfied: cffi>=1.4.1 in /data/_software/python/anaconda3/lib/python3.7/site-packages (from PyNaCl==1.3.0) (1.12.3)
Requirement already satisfied: six in /data/_software/python/anaconda3/lib/python3.7/site-packages (from PyNaCl==1.3.0) (1.12.0)
Requirement already satisfied: pycparser in /data/_software/python/anaconda3/lib/python3.7/site-packages (from cffi>=1.4.1->PyNaCl==1.3.0) (2.19)
Installing collected packages: PyNaCl
Successfully installed PyNaCl-1.3.0安装paramiko-2.6.0-py2.py3-none-any.whl
$ pip install paramiko-2.6.0-py2.py3-none-any.whl
Processing ./paramiko-2.6.0-py2.py3-none-any.whl
Requirement already satisfied: bcrypt>=3.1.3 in /data/_software/python/anaconda3/lib/python3.7/site-packages (from paramiko==2.6.0) (3.1.7)
Requirement already satisfied: pynacl>=1.0.1 in /data/_software/python/anaconda3/lib/python3.7/site-packages (from paramiko==2.6.0) (1.3.0)
Requirement already satisfied: cryptography>=2.5 in /data/_software/python/anaconda3/lib/python3.7/site-packages (from paramiko==2.6.0) (2.7)
Requirement already satisfied: six>=1.4.1 in /data/_software/python/anaconda3/lib/python3.7/site-packages (from bcrypt>=3.1.3->paramiko==2.6.0) (1.12.0)
Requirement already satisfied: cffi>=1.1 in /data/_software/python/anaconda3/lib/python3.7/site-packages (from bcrypt>=3.1.3->paramiko==2.6.0) (1.12.3)
Requirement already satisfied: asn1crypto>=0.21.0 in /data/_software/python/anaconda3/lib/python3.7/site-packages (from cryptography>=2.5->paramiko==2.6.0) (0.24.0)
Requirement already satisfied: pycparser in /data/_software/python/anaconda3/lib/python3.7/site-packages (from cffi>=1.1->bcrypt>=3.1.3->paramiko==2.6.0) (2.19)
Installing collected packages: paramiko
Successfully installed paramiko-2.6.0安装PyMySQL-0.9.3-py2.py3-none-any.whl
$ pip install PyMySQL-0.9.3-py2.py3-none-any.whl
Processing ./PyMySQL-0.9.3-py2.py3-none-any.whl
Installing collected packages: PyMySQL
Successfully installed PyMySQL-0.9.3安装cx_Oracle-8.0.0-cp37-cp37m-manylinux1_x86_64.whl
$ pip install cx_Oracle-8.0.0-cp37-cp37m-manylinux1_x86_64.whl
Processing ./cx_Oracle-8.0.0-cp37-cp37m-manylinux1_x86_64.whl
Installing collected packages: cx-Oracle
Successfully installed cx-Oracle-8.0.0安装未有whl文件的tar.gz文件，进入到arvo_python3所在目录
cd /data/_software/_package解压arvo_python3-1.8.2.tar.gz，会自动创建并解压到arvo_python3-1.8.2
tar -zxvf arvo_python3-1.8.2.tar.gz进入arvo_python3-1.8.2包
cd arvo_python3-1.8.2执行命令python setup.py install
$ python setup.py install
running install
running bdist_egg
running egg_info
writing avro_python3.egg-info/PKG-INFO
writing dependency_links to avro_python3.egg-info/dependency_links.txt
writing top-level names to avro_python3.egg-info/top_level.txt
reading manifest file 'avro_python3.egg-info/SOURCES.txt'
writing manifest file 'avro_python3.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
creating build
creating build/lib
creating build/lib/avro
copying avro/__init__.py -> build/lib/avro
copying avro/datafile.py -> build/lib/avro
copying avro/io.py -> build/lib/avro
copying avro/ipc.py -> build/lib/avro
copying avro/protocol.py -> build/lib/avro
copying avro/schema.py -> build/lib/avro
copying avro/tool.py -> build/lib/avro
copying avro/txipc.py -> build/lib/avro
copying avro/HandshakeRequest.avsc -> build/lib/avro
copying avro/HandshakeResponse.avsc -> build/lib/avro
copying avro/VERSION.txt -> build/lib/avro
copying avro/LICENSE -> build/lib/avro
copying avro/NOTICE -> build/lib/avro
creating build/bdist.linux-x86_64
creating build/bdist.linux-x86_64/egg
creating build/bdist.linux-x86_64/egg/avro
copying build/lib/avro/__init__.py -> build/bdist.linux-x86_64/egg/avro
copying build/lib/avro/datafile.py -> build/bdist.linux-x86_64/egg/avro
copying build/lib/avro/io.py -> build/bdist.linux-x86_64/egg/avro
copying build/lib/avro/ipc.py -> build/bdist.linux-x86_64/egg/avro
copying build/lib/avro/protocol.py -> build/bdist.linux-x86_64/egg/avro
copying build/lib/avro/schema.py -> build/bdist.linux-x86_64/egg/avro
copying build/lib/avro/tool.py -> build/bdist.linux-x86_64/egg/avro
copying build/lib/avro/txipc.py -> build/bdist.linux-x86_64/egg/avro
copying build/lib/avro/HandshakeRequest.avsc -> build/bdist.linux-x86_64/egg/avro
copying build/lib/avro/HandshakeResponse.avsc -> build/bdist.linux-x86_64/egg/avro
copying build/lib/avro/VERSION.txt -> build/bdist.linux-x86_64/egg/avro
copying build/lib/avro/LICENSE -> build/bdist.linux-x86_64/egg/avro
copying build/lib/avro/NOTICE -> build/bdist.linux-x86_64/egg/avro
byte-compiling build/bdist.linux-x86_64/egg/avro/__init__.py to __init__.cpython-37.pyc
byte-compiling build/bdist.linux-x86_64/egg/avro/datafile.py to datafile.cpython-37.pyc
byte-compiling build/bdist.linux-x86_64/egg/avro/io.py to io.cpython-37.pyc
byte-compiling build/bdist.linux-x86_64/egg/avro/ipc.py to ipc.cpython-37.pyc
byte-compiling build/bdist.linux-x86_64/egg/avro/protocol.py to protocol.cpython-37.pyc
byte-compiling build/bdist.linux-x86_64/egg/avro/schema.py to schema.cpython-37.pyc
byte-compiling build/bdist.linux-x86_64/egg/avro/tool.py to tool.cpython-37.pyc
byte-compiling build/bdist.linux-x86_64/egg/avro/txipc.py to txipc.cpython-37.pyc
creating build/bdist.linux-x86_64/egg/EGG-INFO
installing scripts to build/bdist.linux-x86_64/egg/EGG-INFO/scripts
running install_scripts
running build_scripts
creating build/scripts-3.7
copying and adjusting scripts/avro -> build/scripts-3.7
changing mode of build/scripts-3.7/avro from 600 to 755
creating build/bdist.linux-x86_64/egg/EGG-INFO/scripts
copying build/scripts-3.7/avro -> build/bdist.linux-x86_64/egg/EGG-INFO/scripts
changing mode of build/bdist.linux-x86_64/egg/EGG-INFO/scripts/avro to 755
copying avro_python3.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO
copying avro_python3.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying avro_python3.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying avro_python3.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
zip_safe flag not set; analyzing archive contents...
avro.__pycache__.__init__.cpython-37: module references __file__
avro.__pycache__.ipc.cpython-37: module references __file__
creating dist
creating 'dist/avro_python3-1.8.2-py3.7.egg' and adding 'build/bdist.linux-x86_64/egg' to it
removing 'build/bdist.linux-x86_64/egg' (and everything under it)
Processing avro_python3-1.8.2-py3.7.egg
creating /data/_software/python/anaconda3/lib/python3.7/site-packages/avro_python3-1.8.2-py3.7.egg
Extracting avro_python3-1.8.2-py3.7.egg to /data/_software/python/anaconda3/lib/python3.7/site-packages
Adding avro-python3 1.8.2 to easy-install.pth file
Installing avro script to /data/_software/python/anaconda3/bin
Installed /data/_software/python/anaconda3/lib/python3.7/site-packages/avro_python3-1.8.2-py3.7.egg
Processing dependencies for avro-python3==1.8.2
Finished processing dependencies for avro-python3==1.8.24）检查确认依赖库
执行安装新插件后，96服务器插件列表为295个插件
执行命令 conda list -e > [输出插件列表到文件.txt]
Q2、sshpass命令没有找到
Q2：执行python程序，提示sshpass命令没有找到
A2：
1）下载 sshpass-1.06.tar.gz
有网络情况下，执行命令
wget http://sourceforge.net/projects/sshpass/files/latest/download -O sshpass.tar.gz2）解压
解压sshpass-1.06.tar.gz到当前目录（自动创建目录并写入解压文件）
tar -zxvf sshpass-1.06.tar.gz3）编译
进入sshpass-1.06目录，进行编译
./configure4）安装
执行安装命令
sudo make install5）其他参考
https://linux.cn/article-8086-1.html
Q3、No such file or directory
Q3、执行python程序，控制台打印出来以下错误信息
Traceback (most recent call last):
File "test_oracle_lyc.py", line 4, in <module>
db = cx_Oracle.connect("xxx.xxx.xxx.xxx", "username", "password", "orclName", charset='utf8' )
cx_Oracle.DatabaseError: DPI-1047: Cannot locate a 64-bit Oracle Client library: "libnsl.so.1: cannot open shared object file: No such file or directory". See https://cx-oracle.readthedocs.io/en/latest/user_guide/installation.html for helpA3：
1）python、oracle依赖包
经过排查，发现缺少oracle_client，安装oracle-instantclient19.3-basic-19.3.0.0.0-1.x86_64.rpm
2）安装
使用root用户执行安装命令，安装到默认路径
rpm -Uvh oracle-instantclient19.3-basic-19.3.0.0.0-1.x86_64.rpm3）环境变量
使用root用户登录进去配置相关环境变量（这个好像不用配置也可以，我在官方网站看到18+以下数据库客户端才需要配置环境变量，以及手动配置lib库地址和bin目录，现在我们使用的版本为19，已经自动创建libaio以及其他依赖包）。vim /etc/profile，输入环境变量信息
export ORACLE_HOME=/usr/lib/oracle/19.3/client64
export ORABIN=/usr/lib/oracle/19.3/client64/bin
export LD_LIBRARY_PATH=/usr/lib/oracle/19.3/client64/lib
export PATH=$PATH:$ORACLE_HOME/bin让环境变量立即生效
source /etc/profile4）安装系统依赖包
在root用户下，安装libnsl.so.1，进入到目录查看系统依赖包，由于我们都是使用的64bit操作系统，所有的软件工具也都是64bit
cd /usr/lib64查看（ls）是否有libnsl依赖包，在85服务器上，我查询到libnsl* 的依赖包为
libnsl.so.2
libnsl.so.2.0.0查看（ll）是否有libnsl依赖包，在85服务器上，我查询到libnsl* 的依赖包为
lrwxrwxrwx 1 root root 15 Mar 15 2020 libnsl.so.2 -> libnsl.so.2.0.0
-rwxr-xr-x 1 root root 96480 Mar 15 2020 libnsl.so.2.0.0然后进行拷贝，软连接设置
cp -r libnsl.so.2 libnsl.so.1再次执行查看操作
lrwxrwxrwx 1 root root 15 Jan 23 16:40 libnsl.so.1 -> libnsl.so.2.0.0
lrwxrwxrwx 1 root root 15 Mar 15 2020 libnsl.so.2 -> libnsl.so.2.0.0
-rwxr-xr-x 1 root root 96480 Mar 15 2020 libnsl.so.2.0.05）切换测试
新打开一个窗口，使用appuser登录，或者重新登录到服务器，然后编写一个连接oracle的测试脚本，脚本如下
import cx_Oracle
db = connect("userName", "password", "IP:Port/serviceName", "UTF-8")
print(db.version)保存代码为 test_oracle.py
执行python test_oracle.py，如果打印出来数据库服务器的版本号，则说明已经连接并成功读取数据。
总结
笔记属于个人的喜好，但是形成了自我的风格就是博文了。希望在记录自己博文道路越走越远。
原创声明：本文系作者授权腾讯云开发者社区发表，未经许可，不得转载。
如有侵权，请联系 cloudcommunity@tencent.com 删除。
原创声明：本文系作者授权腾讯云开发者社区发表，未经许可，不得转载。
如有侵权，请联系 cloudcommunity@tencent.com 删除。
腾讯云开发者
