Ubuntu系统下部署大语言模型：Ollama和OpenWebUI实现各大模型的人工智能自由

$ sudo apt  install curl
​
$ curl -fsSL https://ollama.com/install.sh | sh

$ sudo mkdir ollama
cd ollama
$ sudo wget https://ollama.com/install.sh

$ sudo vim install.sh
​
修改文件：
status "Downloading ollama..."
#curl --fail --show-error --location --progress-bar -o $TEMP_DIR/ollama "https://ollama.com/download/ollama-linux-${ARCH}${VER_PARAM}"

$ sudo vim install.sh
​
修改文件：
status "Installing ollama to $BINDIR..."
$SUDO install -o0 -g0 -m755 -d $BINDIR
#$SUDO install -o0 -g0 -m755 $TEMP_DIR/ollama $BINDIR/ollama
$SUDO install -o0 -g0 -m755 ./ollama-linux-amd64  $BINDIR/ollama

sh  ./install.sh

cd 
sudo vim .bashrc
​
sudo mkdir -p /home/star/ollama/ollama_cache

### ollama model dir 改为自己的路径
# export OLLAMA_MODELS=/path/ollama_cache
export OLLAMA_MODELS=/home/star/ollama/ollama_cache
​

# ollama --help
Large language model runner
​
Usage:
  ollama [flags]
  ollama [command]
​
Available Commands:
  serve       Start ollama
  create      Create a model from a Modelfile
  show        Show information for a model
  run         Run a model
  pull        Pull a model from a registry
  push        Push a model to a registry
  list        List models
  ps          List running models
  cp          Copy a model
  rm          Remove a model
  help        Help about any command
​
Flags:
  -h, --help      help for ollama
  -v, --version   Show version information
​
Use "ollama [command] --help" for more information about a command.

star@star-ai:~$ ollama serve
Couldn't find '/home/star/.ollama/id_ed25519'. Generating new private key.
Your new public key is: 
​
ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPmYsSi6aIsyhC4EHEsCdBtSOqnfKmNVSf0Ofz9sVzyB
​
Error: listen tcp 127.0.0.1:11434: bind: address already in use
​

vim /etc/systemd/system/ollama.service
在 [Service] 下添加  Environment="OLLAMA_HOST=0.0.0.0"
​
cat /etc/systemd/system/ollama.service
[Unit]
Description=Ollama Service
After=network-online.target
​
[Service]
ExecStart=/usr/local/bin/ollama serve
User=ollama
Group=ollama
Restart=always
RestartSec=3
Environment="PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin"
Environment="OLLAMA_HOST=0.0.0.0"
​
[Install]
WantedBy=default.target

systemctl daemon-reload
​
systemctl restart ollama
​
关闭服务
systemctl stop ollama
启动服务
systemctl start ollama

ollama run  qwen

sudo curl -sSL https://get.docker.com/ | sh
​
安装完成之后
star@star-ai:~$ sudo docker --version
Docker version 26.1.3, build b72abbb
​

$ sudo docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main

$ sudo docker run -d -p 3000:8080 --gpus all --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:cuda

$ sudo docker run -d -p 3000:8080 --gpus all --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always  registry.cn-shenzhen.aliyuncs.com/funet8/open-webui:cuda

sudo docker run -d -p 3000:8080 --gpus all --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always  registry.cn-shenzhen.aliyuncs.com/funet8/open-webui:cuda
254b47e7994b2f0087ce0058918621523b39cf9b0e89018777c0cf98943ba2d1
docker: Error response from daemon: could not select device driver "" with capabilities: [[gpu]].

$ sudo nvidia-smi
Fri May 17 18:37:15 2024       
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.54.15              Driver Version: 550.54.15      CUDA Version: 12.4     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce GTX 1060 6GB    Off |   00000000:01:00.0 Off |                  N/A |
| 40%   33C    P8              6W /  120W |      65MiB /   6144MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+
                                                                                         
+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|    0   N/A  N/A      1030      G   /usr/lib/xorg/Xorg                             56MiB |
|    0   N/A  N/A      1109      G   /usr/bin/gnome-shell                            4MiB |
+-----------------------------------------------------------------------------------------+

sudo apt-get update
sudo apt-get install -y nvidia-container-toolkit
sudo systemctl restart docker

sudo vim /etc/docker/daemon.json
添加：
​
{
  "default-runtime": "nvidia",
  "runtimes": {
    "nvidia": {
      "path": "nvidia-container-runtime",
      "runtimeArgs": []
    }
  }
}
编辑完文件后，重启Docker服务：
sudo systemctl restart docker
​

sudo docker version
​
nvidia-container-runtime --version
​

sudo docker start open-webui

阿里巴巴的大模型：
ollama run  qwen
ollama run qwen:14b
ollama run qwen:32b
ollama run qwen:72b
ollama run qwen:110b   # 110b 表示该模型包含了 1100 亿（110 billion）个参数
​
​
脸书大模型：
ollama run llama2
ollama run llama3
ollama run llama3:8b
​
谷歌的大模型：
ollama run gemma
​
微软的大模型
ollama run phi3
​

显示所有模型
# ollama list
​
删除模型
# ollama rm llama3:latest

nvidia-smi