# 【迁移】Docker搭建vicuna - Limour's Blog
项目地址：llama-cpp-python 镜像地址：hub-mirror 模型地址：ggml-vic13b-q5_1.bin 前端UI地址(目前仍不兼容)：BetterChatGPT 部署Docker镜像 123456mkdir -p ~/app/llama &amp;&amp; cd ~/app/llama &amp;&amp; nano docker-compose.ymlmkdir m
Author: Limour

# 【迁移】Docker搭建vicuna

Last updated on March 19, 2024 pm

- 项目地址：llama-cpp-python
- 镜像地址：hub-mirror
- 模型地址：ggml-vic13b-q5_1.bin
- 前端UI地址(目前仍不兼容)：BetterChatGPT

## 部署Docker镜像

    1
    2
    3
    4
    5
    6

    mkdir -p ~/app/llama && cd ~/app/llama && nano docker-compose.yml
    mkdir models && cd models
    wget https://huggingface.co/eachadea/ggml-vicuna-13b-1.1/resolve/main/ggml-vic13b-q5_1.bin
    cd ~/app/llama
    sudo docker-compose up -d
    sudo docker-compose logs

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12

    version: '3.3'
    services:
        llama:
            ports:
                - '1234:8000'
            restart: always
            environment:
                MODEL: /models/ggml-vic13b-q5_1.bin
            volumes:
                - './models:/models'
            image: togettoyou/ghcr.io.abetlen.llama-cpp-python:latest
            command: ["python3", "-m", "llama_cpp.server", "--model", "/models/ggml-vic13b-q5_1.bin"]

## 测试是否成功

- 查看文档：`http://localhost:1234/docs`

    1
    2
    3
    4
    5
    6

    curl http://localhost:1234/v1/chat/completions \
      -H 'Content-Type: application/json' \
      -d '{
      "model": "gpt-3.5-turbo",
      "messages": [{"role": "user", "content": "Hello!"}]
    }'

#llama #docker
【迁移】Docker搭建vicuna
https://hexo.limour.top/-qian-yi--Docker-da-jian-vicuna
Author
Limour
Posted on
May 7, 2023
Updated on
March 19, 2024
Licensed under
【记录】DOCKER安装流程 Previous
【迁移】基于Chroma搭建个人知识库的全文索引 Next

#### Search

Keyword