Docker搭建vicuna

项目地址:llama-cpp-python
镜像地址:hub-mirror
模型地址:ggml-vic13b-q5_1.bin
前端UI地址(目前仍不兼容):BetterChatGPT

部署Docker镜像

1
2
3
4
5
6
7
8
9
10
11
12
version: '3.3'
services:
llama:
ports:
- '1234:8000'
restart: always
environment:
MODEL: /models/ggml-vic13b-q5_1.bin
volumes:
- './models:/models'
image: togettoyou/ghcr.io.abetlen.llama-cpp-python:latest
command: ["python3", "-m", "llama_cpp.server", "--model", "/models/ggml-vic13b-q5_1.bin"]

测试是否成功

1
2
3
4
5
6
curl http://localhost:1234/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{
"model": "gpt-3.5-turbo",
"messages": [{"role": "user", "content": "Hello!"}]
}'

Docker搭建vicuna
https://occdn.limour.top/2723.html
Author
Limour
Posted on
May 7, 2023
Licensed under