多厂商大模型统一接入网关,支持OpenAI兼容接口协议
和OpenAI SDK完全兼容,只需要把base_url改成本服务地址即可:
import openai
client = openai.OpenAI(
api_key="你的API_KEY",
base_url="http://localhost:8000/v1"
)
# 调用聊天补全
response = client.chat.completions.create(
model="gpt-3.5-turbo",
messages=[{"role": "user", "content": "你好"}]
)
print(response.choices[0].message.content)
curl http://localhost:8000/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer 你的API_KEY" \
-d '{
"model": "gpt-3.5-turbo",
"messages": [{"role": "user", "content": "你好"}]
}'
curl http://localhost:8000/health
返回:{"status":"ok"}
curl http://localhost:8000/v1/models \ -H "Authorization: Bearer 你的API_KEY"