文章目录
- 基础
- fastapi服务
- 请求fastapi接口
基础
https://huggingface.co/OpenGVLab/InternVL2-8B#%E7%AE%80%E4%BB%8B
InternVL2-26B应该更好,但显存没那么大,只能跑InternVL2-8B了。
下载:
cd /ssd/xiedong/InternVL2-26B
git clone https://huggingface.co/OpenGVLab/InternVL2-8B
运行docker:
docker run -it -v /ssd/xiedong/InternVL2-26B:/ssd/xiedong/InternVL2-26B --gpus device=3 -p 7895:7860 kevinchina/deeplearning:pytorch2.3.0-cuda12.1-cudnn8-devel-InternVL2 bash
进去路径:
cd /ssd/xiedong/InternVL2-26B
执行此python代码:
from lmdeploy import pipeline, TurbomindEngineConfig, ChatTemplateConfig
from lmdeploy.vl import load_imagemodel = '/ssd/xiedong/InternVL2-26B/InternVL2-8B'
system_prompt = 'Describe this image in English with no more than 50 words.'
image = load_image('https://raw.githubusercontent.com/open-mmlab/mmdeploy/main/tests/data/tiger.jpeg')
chat_template_config = ChatTemplateConfig('internvl-internlm2')
chat_template_config.meta_instruction = system_prompt
pipe = pipeline(model, chat_template_config=chat_template_config,backend_config=TurbomindEngineConfig(session_len=8192))
response = pipe(('Describe this image in English with no more than 50 words.', image))
print(response.text)
或者传一张图片后执行这种代码:
from lmdeploy import pipeline, TurbomindEngineConfig, ChatTemplateConfig
from lmdeploy.vl import load_imagemodel = '/ssd/xiedong/InternVL2-26B/InternVL2-8B'
system_prompt = 'Describe this image in English with no more than 50 words.'
image = load_image('/ssd/xiedong/InternVL2-26B/000030982.jpg')
chat_template_config = ChatTemplateConfig('internvl-internlm2')
chat_template_config.meta_instruction = system_prompt
pipe = pipeline(model, chat_template_config=chat_template_config,backend_config=TurbomindEngineConfig(session_len=8192))
response = pipe(('Describe this image in English with no more than 50 words.', image))
print(response.text)
fastapi服务
执行这个代码可以开启一个fastapi接口,7860端口会被docker映射到7895.
from fastapi import FastAPI, File, UploadFile
from fastapi.responses import JSONResponse
from lmdeploy import pipeline, TurbomindEngineConfig, ChatTemplateConfig
from lmdeploy.vl import load_image
from PIL import Image
import ioapp = FastAPI()model = '/ssd/xiedong/InternVL2-26B/InternVL2-8B'
system_prompt = 'Describe this image in English with no more than 50 words.'
chat_template_config = ChatTemplateConfig('internvl-internlm2')
chat_template_config.meta_instruction = system_prompt
pipe = pipeline(model, chat_template_config=chat_template_config,backend_config=TurbomindEngineConfig(session_len=8192))@app.post("/describe-image")
def describe_image(file: UploadFile = File(...)):try:# 将上传的文件转为Pillow图像对象image = Image.open(io.BytesIO(file.file.read()))# 使用load_image方法加载图像loaded_image = load_image(image)# 调用模型处理图片response = pipe(('Describe this image in English with no more than 50 words, just need to output a captioning of the image.', loaded_image))# 返回描述结果return JSONResponse(content={"description": response.text})except Exception as e:return JSONResponse(content={"error": str(e)}, status_code=500)finally:file.file.close()if __name__ == "__main__":import uvicornuvicorn.run(app, host="0.0.0.0", port=7860)
请求fastapi接口
import requestsurl = "http://10.136.19.26:7895/describe-image"# 要上传的图片文件路径
file_path = "output_image.png"# 打开文件并发送POST请求
with open(file_path, "rb") as file:files = {"file": file}response = requests.post(url, files=files)# 检查响应并打印结果
if response.status_code == 200:print("Description:", response.json().get("description"))
else:print("Error:", response.json().get("error"))