55_大模型部署：从云端到边缘的全场景实践

大模型部署技术演进
├── 早期阶段(2018-2020): 单节点部署，资源利用率低，扩展性差
├── 发展阶段(2021-2023): 容器化部署，分布式推理，初步实现弹性扩展
└── 成熟阶段(2024-2025): 云边协同，智能调度，服务网格，多模态融合部署

目录
├── 第一章：大模型部署的挑战与需求
├── 第二章：云原生架构与容器化部署
├── 第三章：分布式推理架构设计
├── 第四章：服务网格与流量治理
├── 第五章：边缘计算与云边协同
├── 第六章：部署工具与平台详解
├── 第七章：性能优化与监控
├── 第八章：安全与合规实践
├── 第九章：行业案例与最佳实践
└── 第十章：未来发展趋势与建议

部署架构演进时间线
2018-2020: 单机部署 → 2021-2023: 容器化部署 → 2023-2024: 分布式推理 → 2024-2025: 云边协同

# 大模型服务的Kubernetes部署示例
apiVersion: apps/v1
kind: Deployment
metadata:
  name: llm-inference-service
  namespace: ai-services
spec:
  replicas: 3
  selector:
    matchLabels:
      app: llm-inference
  template:
    metadata:
      labels:
        app: llm-inference
    spec:
      containers:
      - name: llm-container
        image: registry.example.com/llm-service:latest
        resources:
          requests:
            memory: "16Gi"
            cpu: "8"
            nvidia.com/gpu: 1
          limits:
            memory: "32Gi"
            cpu: "16"
            nvidia.com/gpu: 1
        ports:
        - containerPort: 8080
        volumeMounts:
        - name: model-cache
          mountPath: /models
        env:
        - name: MODEL_NAME
          value: "my-large-model"
        - name: BATCH_SIZE
          value: "8"
        - name: MAX_SEQUENCE_LENGTH
          value: "4096"
      volumes:
      - name: model-cache
        persistentVolumeClaim:
          claimName: model-cache-pvc

# HorizontalPodAutoscaler示例
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: llm-inference-hpa
  namespace: ai-services
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: llm-inference-service
  minReplicas: 2
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: nvidia.com/gpu
      target:
        type: Utilization
        averageUtilization: 70
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 80

# 大模型服务的优化Dockerfile示例
FROM nvidia/cuda:12.1.1-cudnn8-runtime-ubuntu22.04 AS base

# 安装系统依赖
RUN apt-get update && apt-get install -y --no-install-recommends \
    python3 python3-pip python3-dev \
    && rm -rf /var/lib/apt/lists/*

# 设置Python环境
RUN ln -s /usr/bin/python3 /usr/bin/python && \
    pip3 install --no-cache-dir --upgrade pip setuptools

# 安装模型依赖
COPY requirements.txt .
RUN pip3 install --no-cache-dir -r requirements.txt && \
    rm requirements.txt

# 复制应用代码
WORKDIR /app
COPY . /app

# 优化运行时配置
ENV NVIDIA_VISIBLE_DEVICES=all
ENV PYTHONUNBUFFERED=1
ENV MODEL_CACHE_DIR=/models

# 创建模型缓存目录
RUN mkdir -p $MODEL_CACHE_DIR

# 设置启动命令
CMD ["python", "inference_server.py"]

# Prometheus ServiceMonitor示例
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: llm-inference-monitor
  namespace: monitoring
spec:
  selector:
    matchLabels:
      app: llm-inference
  namespaceSelector:
    matchNames:
    - ai-services
  endpoints:
  - port: metrics
    interval: 15s
    path: /metrics

# vLLM服务器启动代码示例
from vllm import LLM, SamplingParams
from vllm.engine.arg_utils import AsyncEngineArgs
from vllm.engine.async_llm_engine import AsyncLLMEngine

async def create_llm_engine():
    # 配置引擎参数
    engine_args = AsyncEngineArgs(
        model="meta-llama/Llama-2-70b-hf",  # 模型名称或路径
        tensor_parallel_size=8,  # 张量并行度
        quantization="awq",  # 量化方法
        gpu_memory_utilization=0.9,  # GPU内存利用率
        max_model_len=4096,  # 最大序列长度
        trust_remote_code=True,  # 允许执行远程代码
    )
    
    # 创建异步引擎
    engine = await AsyncLLMEngine.from_engine_args(engine_args)
    return engine

async def generate_text(engine, prompts, sampling_params):
    # 生成文本
    results = []
    for prompt in prompts:
        request_id = str(uuid.uuid4())
        result_stream = engine.generate(prompt, sampling_params, request_id)
        # 处理流式输出
        full_text = ""
        async for output in result_stream:
            if output.outputs[0].text:
                full_text = output.outputs[0].text
                # 可以在这里实现流式返回给客户端
        results.append(full_text)
    return results

# vLLM在K8s上的部署示例
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: vllm-service
  namespace: ai-services
spec:
  serviceName: "vllm"
  replicas: 2
  selector:
    matchLabels:
      app: vllm
  template:
    metadata:
      labels:
        app: vllm
    spec:
      containers:
      - name: vllm-container
        image: vllm/vllm-openai:latest
        args:
        - --model
        - meta-llama/Llama-2-70b-hf
        - --tensor-parallel-size
        - "8"
        - --quantization
        - awq
        - --max-model-len
        - "4096"
        resources:
          requests:
            memory: "24Gi"
            cpu: "16"
            nvidia.com/gpu: 8
          limits:
            memory: "32Gi"
            cpu: "32"
            nvidia.com/gpu: 8
        ports:
        - containerPort: 8000
        volumeMounts:
        - name: model-cache
          mountPath: /model-cache
  volumeClaimTemplates:
  - metadata:
      name: model-cache
    spec:
      accessModes: ["ReadWriteOnce"]
      storageClassName: "fast-storage"
      resources:
        requests:
          storage: 100Gi

# 混合并行策略配置示例
from transformers import AutoModelForCausalLM, AutoTokenizer
from accelerate import init_empty_weights, load_checkpoint_and_dispatch

# 初始化空权重模型
with init_empty_weights():
    model = AutoModelForCausalLM.from_pretrained(
        "meta-llama/Llama-2-70b-hf",
        torch_dtype="auto",
        low_cpu_mem_usage=True
    )

# 配置模型并行策略
model = load_checkpoint_and_dispatch(
    model,
    "path/to/checkpoint",
    device_map="auto",  # 自动确定设备映射
    no_split_module_classes=["LlamaDecoderLayer"],  # 不分割的模块
    offload_folder="offload",  # CPU卸载路径
    offload_state_dict=True,  # 是否卸载状态字典
    max_memory={  # 每个设备的最大内存限制
        0: "24GiB",
        1: "24GiB",
        2: "24GiB",
        3: "24GiB",
        "cpu": "50GiB"
    }
)

# Istio VirtualService配置示例
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: llm-service-vs
  namespace: ai-services
spec:
  hosts:
  - llm-service
  http:
  - route:
    - destination:
        host: llm-service
        subset: v1
      weight: 90
    - destination:
        host: llm-service
        subset: v2
      weight: 10
    timeout: 30s
    retries:
      attempts: 3
      perTryTimeout: 10s
      retryOn: connect-failure,refused-stream,unavailable
    fault:
      delay:
        percentage:
          value: 5
        fixedDelay: 2s

# 云边协同框架示例
class EdgeCloudCoordinator:
    def __init__(self):
        self.edge_models = {}
        self.cloud_models = {}
        self.task_queue = []
        self.resource_monitor = ResourceMonitor()
    
    def register_model(self, model_id, model_config, deployment_location):
        """注册模型到边缘或云端"""
        if deployment_location == "edge":
            self.edge_models[model_id] = model_config
        else:
            self.cloud_models[model_id] = model_config
    
    def schedule_task(self, task):
        """智能调度任务到边缘或云端"""
        # 评估任务复杂度
        complexity = self.evaluate_task_complexity(task)
        
        # 检查边缘资源状况
        edge_resources = self.resource_monitor.get_edge_resources()
        
        # 根据规则决定处理位置
        if complexity <= edge_resources['max_complexity'] and \
           edge_resources['available_memory'] > self.estimate_memory_need(task):
            # 边缘处理条件满足
            return self.dispatch_to_edge(task)
        else:
            # 云端处理
            return self.dispatch_to_cloud(task)
    
    def handle_edge_offload(self, task):
        """处理边缘无法完成需卸载到云端的任务"""
        # 保留任务上下文信息
        task_context = self.extract_context(task)
        
        # 卸载到云端
        cloud_result = self.process_in_cloud(task, task_context)
        
        # 返回结果到边缘
        return self.return_to_edge(task, cloud_result)

# Triton在Kubernetes上的部署示例
apiVersion: apps/v1
kind: Deployment
metadata:
  name: triton-server
  namespace: ai-services
spec:
  replicas: 2
  selector:
    matchLabels:
      app: triton-server
  template:
    metadata:
      labels:
        app: triton-server
    spec:
      containers:
      - name: triton-server
        image: nvcr.io/nvidia/tritonserver:23.09-py3
        args:
        - "tritonserver"
        - "--model-repository=/models"
        - "--allow-grpc=true"
        - "--allow-http=true"
        - "--http-port=8000"
        - "--grpc-port=8001"
        - "--metrics-port=8002"
        resources:
          requests:
            memory: "16Gi"
            cpu: "8"
            nvidia.com/gpu: 1
          limits:
            memory: "32Gi"
            cpu: "16"
            nvidia.com/gpu: 1
        volumeMounts:
        - name: model-repository
          mountPath: /models
      volumes:
      - name: model-repository
        persistentVolumeClaim:
          claimName: triton-model-repository-pvc
---
apiVersion: v1
kind: Service
metadata:
  name: triton-service
  namespace: ai-services
spec:
  selector:
    app: triton-server
  ports:
  - name: http
    port: 8000
    targetPort: 8000
  - name: grpc
    port: 8001
    targetPort: 8001
  - name: metrics
    port: 8002
    targetPort: 8002
  type: LoadBalancer

# Triton模型配置示例 (config.pbtxt)
name: "llama_model"
platform: "tensorrtllm"
max_batch_size: 8
input [
  {
    name: "input_ids"
    data_type: TYPE_INT32
    dims: [ -1 ]
  },
  {
    name: "attention_mask"
    data_type: TYPE_INT32
    dims: [ -1 ]
  }
]
output [
  {
    name: "output_ids"
    data_type: TYPE_INT32
    dims: [ -1 ]
  }
]
dynamic_batching {
  preferred_batch_size: [ 1, 2, 4, 8 ]
  max_queue_delay_microseconds: 1000
}
instance_group [
  {
    count: 1
    kind: KIND_GPU
    gpus: [ 0 ]
  }
]

# GitLab CI/CD配置示例
stages:
  - build
  - test
  - deploy

variables:
  DOCKER_REGISTRY: "registry.example.com"
  IMAGE_NAME: "llm-service"

build_image:
  stage: build
  script:
    - docker build -t $DOCKER_REGISTRY/$IMAGE_NAME:$CI_COMMIT_SHA .
    - docker push $DOCKER_REGISTRY/$IMAGE_NAME:$CI_COMMIT_SHA

run_tests:
  stage: test
  script:
    - docker run --rm $DOCKER_REGISTRY/$IMAGE_NAME:$CI_COMMIT_SHA python -m pytest tests/
    - python scripts/benchmark.py --image $DOCKER_REGISTRY/$IMAGE_NAME:$CI_COMMIT_SHA

deploy_staging:
  stage: deploy
  environment:
    name: staging
  script:
    - kubectl config use-context staging
    - sed -i "s|image:.*|image: $DOCKER_REGISTRY/$IMAGE_NAME:$CI_COMMIT_SHA|g" k8s/staging/deployment.yaml
    - kubectl apply -f k8s/staging/
    - kubectl rollout status deployment/llm-service -n staging

deploy_production:
  stage: deploy
  environment:
    name: production
  when: manual
  script:
    - kubectl config use-context production
    - sed -i "s|image:.*|image: $DOCKER_REGISTRY/$IMAGE_NAME:$CI_COMMIT_SHA|g" k8s/production/deployment.yaml
    - kubectl apply -f k8s/production/
    - kubectl rollout status deployment/llm-service -n production

# 推理性能优化示例 - 使用FlashAttention和混合精度
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from transformers import BitsAndBytesConfig

# 配置量化参数
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_use_double_quant=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.float16
)

# 加载优化后的模型
model = AutoModelForCausalLM.from_pretrained(
    "meta-llama/Llama-2-70b-hf",
    quantization_config=bnb_config,
    device_map="auto",
    use_flash_attention_2=True,  # 启用FlashAttention-2
    torch_dtype=torch.float16,
    low_cpu_mem_usage=True
)

tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-2-70b-hf")

# 优化的推理函数
def optimized_generate(model, tokenizer, prompt, max_new_tokens=100, batch_size=4):
    # 预处理
    inputs = tokenizer(prompt, return_tensors="pt", padding=True).to(model.device)
    
    # 使用高效的生成参数
    with torch.no_grad():
        # 使用torch.inference_mode()进一步优化
        with torch.inference_mode():
            # 启用渐进式生成
            output = model.generate(
                **inputs,
                max_new_tokens=max_new_tokens,
                do_sample=True,
                temperature=0.7,
                top_p=0.95,
                use_cache=True,  # 启用KV缓存
                num_return_sequences=1,
                pad_token_id=tokenizer.eos_token_id,
                # 优化参数
                repetition_penalty=1.1,
                no_repeat_ngram_size=3,
                # 批处理优化
                batch_size=batch_size,
                # 启用编译优化
                torch_compile=True if hasattr(torch, "compile") else False
            )
    
    return tokenizer.decode(output[0], skip_special_tokens=True)

大模型部署成功要素
技术选型 → 架构设计 → 性能优化 → 安全合规 → 持续迭代 → 价值实现