limour-blog.github.io/source/_posts/2023-05-07-【迁移】Docker搭建vicuna.md

---

title: 【迁移】Docker搭建vicuna urlname: -qian-yi--Docker-da-jian-vicuna date: 2023-05-07 19:50:52 index_img: https://api.limour.top/randomImg?d=2023-05-07 19:50:52

tags: ['docker', 'llama']

• 项目地址：llama-cpp-python
• 镜像地址：hub-mirror
• 模型地址：ggml-vic13b-q5_1.bin

• 前端UI地址(目前仍不兼容)：BetterChatGPT

  部署Docker镜像

  mkdir -p ~/app/llama && cd ~/app/llama && nano docker-compose.yml
  mkdir models && cd models
  wget https://huggingface.co/eachadea/ggml-vicuna-13b-1.1/resolve/main/ggml-vic13b-q5_1.bin
  cd ~/app/llama
  sudo docker-compose up -d
  sudo docker-compose logs
  

  version: '3.3'
  services:
  llama:
      ports:
          - '1234:8000'
      restart: always
      environment:
          MODEL: /models/ggml-vic13b-q5_1.bin
      volumes:
          - './models:/models'
      image: togettoyou/ghcr.io.abetlen.llama-cpp-python:latest
      command: ["python3", "-m", "llama_cpp.server", "--model", "/models/ggml-vic13b-q5_1.bin"]
  

  测试是否成功

• 查看文档：http://localhost:1234/docs

  curl http://localhost:1234/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
  "model": "gpt-3.5-turbo",
  "messages": [{"role": "user", "content": "Hello!"}]
  }'