本文简单介绍并实践基于k8s平台运行虚机。
本文比较适合自建集群,阿里云,腾讯云,华为云等云厂商的普通云服务器不支持二次虚拟化,无法实现vm on vm。如果只是业务需要虚机,可以直接在云厂商购买虚机使用。
本文测试使用腾讯云的容器服务,节点使用裸金属云服务器。
开源的基于k8s的虚机解决方案有kubevirt和virtlet,其中kubevirt使用CRD扩展k8s API,virtlet则实现了CRI接口。
kubevirt使用CRD扩展k8s api,控制面以add-on的形式部署到k8s集群。
项目:https://github.com/kubevirt/kubevirt
架构:
virtlet是较早开源的基于k8s平台的虚机解决方案,实现了CRI接口,组件部署需要先给节点部署cri-proxy,cri-proxy会根据配置选择dockershim或者virtlet,然后再部署daemonset
项目:https://github.com/Mirantis/virtlet
文档:https://docs.virtlet.cloud/
架构:
kubevirt | virtlet | |
---|---|---|
公司 | redhat | Mirantis |
开源项目 | ||
现状 | 持续更新 | 3年未更新 |
版本 | v0.56.0 | v1.5.1 |
star | 3.5K | 686 |
同时运行虚机/容器 | 支持 | 支持 |
实现方式 | CRD add-on | CRI |
组件部署 | 简单(add-on) | 复杂(worker节点配置+add-on) |
使用方法 | CR,扩展性强 | pod annotation(原生workload) |
虚机方案 | libvirt API | libvirt API |
cloudInit | 支持 | 支持 |
多网卡 | 支持(依赖网络插件) | 支持(依赖网络插件) |
...... |
注意腾讯云只有裸金属云服务器的标准集群可以用于kubevirt/virtlet使用测试。
参考文档:https://cloud.tencent.com/document/product/457/32189
参考kubevirt文档:https://kubevirt.io/quickstart_cloud/
export VERSION=$(curl -s https://api.github.com/repos/kubevirt/kubevirt/releases | grep tag_name | grep -v -- '-rc' | sort -r | head -1 | awk -F': ' '{print $2}' | sed 's/,//' | xargs)
echo $VERSION
kubectl create -f https://github.com/kubevirt/kubevirt/releases/download/${VERSION}/kubevirt-operator.yaml
export VERSION=$(curl -s https://api.github.com/repos/kubevirt/kubevirt/releases | grep tag_name | grep -v -- '-rc' | sort -r | head -1 | awk -F': ' '{print $2}' | sed 's/,//' | xargs)
echo $VERSION
kubectl create -f https://github.com/kubevirt/kubevirt/releases/download/${VERSION}/kubevirt-cr.yaml
VERSION=$(kubectl get kubevirt.kubevirt.io/kubevirt -n kubevirt -o=jsonpath="{.status.observedKubeVirtVersion}")
ARCH=$(uname -s | tr A-Z a-z)-$(uname -m | sed 's/x86_64/amd64/') || windows-amd64.exe
echo ${ARCH}
curl -L -o virtctl https://github.com/kubevirt/kubevirt/releases/download/${VERSION}/virtctl-${VERSION}-${ARCH}
chmod +x virtctl
sudo install virtctl /usr/local/bin
kubevirt提供了测试的container-disk镜像可以直接用于vm创建。参考https://github.com/kubevirt/kubevirt/blob/main/containerimages/container-disk-images.md
cat > Dockerfile <<EOF
FROM kubevirt/container-disk-v1alpha
ADD fedora32.qcow2 /disk/
EOF
docker build -t kubevirt/fedora-sriov-testing:latest .
CDI使用CRD扩展api,用于将虚拟机映像或其他数据填充PVC来挂载虚机。数据可以来自不同的来源:URL、容器镜像仓库、或来自客户端的上传。
export VERSION=$(curl -s https://github.com/kubevirt/containerized-data-importer/releases/latest | grep -o "v[0-9]\.[0-9]*\.[0-9]*")
kubectl create -f https://github.com/kubevirt/containerized-data-importer/releases/download/$VERSION/cdi-operator.yaml
kubectl create -f https://github.com/kubevirt/containerized-data-importer/releases/download/$VERSION/cdi-cr.yaml
参考文档:https://kubevirt.io/labs/kubernetes/lab1.html
cat > cirros.yaml <<EOF
apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
name: testvm
spec:
running: true
template:
metadata:
labels:
kubevirt.io/size: small
kubevirt.io/domain: testvm
spec:
domain:
devices:
disks:
- name: containerdisk
disk:
bus: virtio
- name: cloudinitdisk
disk:
bus: virtio
interfaces:
- name: default
masquerade: {}
resources:
requests:
memory: 64M
networks:
- name: default
pod: {}
volumes:
- name: containerdisk
containerDisk:
image: quay.io/kubevirt/cirros-container-disk-demo
- name: cloudinitdisk
cloudInitNoCloud:
userDataBase64: SGkuXG4=
EOF
kubectl create -f cirros.yaml
virtctl start <vm-name>
virtctl console <vm-name>
virtctl ssh --local-ssh=true cirros@testvmi
kubevirt windows参考文档:https://kubevirt.io/2020/KubeVirt-installing_Microsoft_Windows_from_an_iso.html
注意:该测试受客户端环境影响,vm启动成功但是挂载的iso操作系统没有安装好。
ISO镜像下载网址:https://info.microsoft.com/ww-landing-windows-server-2012-R2.html
Note: 需要输入个人信息可以免费试用180天
cloudimage镜像下载网址:https://cloudbase.it/windows-cloud-images/
uploadproxy的集群服务
virtctl image-upload \
--image-path=./9600.17050.WINBLUE_REFRESH.140317-1640_X64FRE_SERVER_EVAL_EN-US-IR3_SSS_X64FREE_EN-US_DV9.ISO \
--pvc-name=iso-win2k12 \
--access-mode=ReadOnlyMany \
--pvc-size=5G \
--uploadproxy-url=https://10.96.164.35:443 \
--insecure \
--wait-secs=240
apiVersion: v1
kind: PersistentVolume
metadata:
name: pv-windows-iso
labels:
type: local
spec:
storageClassName: manual
capacity:
storage: 10Gi
accessModes:
- ReadWriteOnce
hostPath:
path: "/mnt/windows"
type: DirectoryOrCreate
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: "pvc-windows-iso"
labels:
app: containerized-data-importer
spec:
storageClassName: manual
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 10Gi
apiVersion: v1
kind: PersistentVolume
metadata:
name: pv-windows-hd
labels:
type: local
spec:
storageClassName: manual
capacity:
storage: 20Gi
accessModes:
- ReadWriteOnce
hostPath:
path: "/mnt/windows-hd"
type: DirectoryOrCreate
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: "pvc-windows-hd"
labels:
app: containerized-data-importer
spec:
storageClassName: manual
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 20Gi
apiVersion: kubevirt.io/v1alpha3
kind: VirtualMachine
metadata:
name: win2k12-iso
spec:
running: true
template:
metadata:
labels:
kubevirt.io/domain: win2k12-iso
spec:
domain:
cpu:
cores: 4
devices:
disks:
- bootOrder: 1
cdrom:
bus: sata
name: cdromiso
- disk:
bus: virtio
name: harddrive
- cdrom:
bus: sata
name: virtiocontainerdisk
machine:
type: q35
resources:
requests:
memory: 8G
volumes:
- name: cdromiso
persistentVolumeClaim:
claimName: pv-windows-iso
- name: harddrive
persistentVolumeClaim:
claimName: pv-windows-hd
- containerDisk:
image: kubevirt/virtio-container-disk
name: virtiocontainerdisk
参考文档:https://kubevirt.io/labs/kubernetes/lab2.html
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: "fedora"
labels:
app: containerized-data-importer
annotations:
cdi.kubevirt.io/storage.import.endpoint: "https://download.fedoraproject.org/pub/fedora/linux/releases/33/Cloud/x86_64/images/Fedora-Cloud-Base-33-1.2.x86_64.raw.xz"
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 5Gi
apiVersion: v1
kind: PersistentVolume
metadata:
name: task-pv-volume
labels:
type: local
spec:
storageClassName: manual
capacity:
storage: 10Gi
accessModes:
- ReadWriteOnce
hostPath:
path: "/mnt/data"
type: DirectoryOrCreate
apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
generation: 1
labels:
kubevirt.io/os: linux
name: vm1
spec:
running: true
template:
metadata:
creationTimestamp: null
labels:
kubevirt.io/domain: vm1
spec:
domain:
cpu:
cores: 2
devices:
disks:
- disk:
bus: virtio
name: disk0
- cdrom:
bus: sata
readonly: true
name: cloudinitdisk
machine:
type: q35
resources:
requests:
memory: 1024M
volumes:
- name: disk0
persistentVolumeClaim:
claimName: fedora
- cloudInitNoCloud:
userData: |
#cloud-config
hostname: vm1
ssh_pwauth: True
disable_root: false
ssh_authorized_keys:
- ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCzJrjWS+X39RGxRiyQkUFFzcHiw5zPg2ASHRHN+wF7418yd+XqZ4PWK8oM2z0Fy4pX+MRMgH36qUhcx0qO6N1UMTsYQr0vbEJ1+6NGrgR4E3Mj3852H3dncVRjcrQL0+JdSFAzpBi/xQnqLS2BpTE0q4rh6kOGPhJ/YhhBAUwxj633o37eiBNZybmkb2XkxqG+kN0513QzPyaIfMwQ1OMe2xTiTeDJIWk1TZ8EJtp6PGxDy+CHCZCtKsE9jzohM4oZF2bBGW9A+gkwxoKAsHAZ0xD3xQFU2xRAKEgl16VNfjgeBZnpbRVIPlhlpQXWyFlJLLqMiLbt0wHhBLB9HCNZ williamji@VM-46-151-centos
name: cloudinitdisk
virtctl console vm1
virtctl ssh fedora@vm1 -i <秘钥文件>
原创声明:本文系作者授权腾讯云开发者社区发表,未经许可,不得转载。
如有侵权,请联系 cloudcommunity@tencent.com 删除。
原创声明:本文系作者授权腾讯云开发者社区发表,未经许可,不得转载。
如有侵权,请联系 cloudcommunity@tencent.com 删除。