python-windows本地部署ChatGLM2-6B-int

先说结论

20分钟出来一句话，期间，内存使用50%左右，cpu使用率85%

电脑配置

电脑环境：4cpu16g

配置PYTHON环境：

修改python安装包路径：https://blog.csdn.net/qq_27466827/article/details/131163026
pip3 install torch torchvision torchaudio -i https://mirrors.aliyun.com/pypi/simple/
pip install transformers sentencepiece
pip install rouge_chinese cpm_kernels

配置GCC环境

TDM-GCC 10.3.0 下载https://jmeubank.github.io/tdm-gcc/download/

下载模型文件

云盘下载：https://cloud.tsinghua.edu.cn/d/674208019e314311ab5c/?p=%2Fchatglm2-6b-int4&mode=list
snapshot_download下载：
代码下载：如下

#执行不成功，多执行几次就可以了 
from huggingface_hub import snapshot_download
import sysrepo_id = "THUDM/chatglm2-6b"
local_dir = "/opt/models/chatglm2-6b/"
local_dir_use_symlinks = False
revision = "main"
snapshot_download(repo_id=repo_id, local_dir=local_dir,local_dir_use_symlinks=local_dir_use_symlinks,revision=revision)

运行

python
from transformers import AutoTokenizer, AutoModel
tokenizer = AutoTokenizer.from_pretrained("G://glm2-int4", trust_remote_code=True)
model = AutoModel.from_pretrained("G://glm2-int4",trust_remote_code=True).float()
model = model.eval()
response, history = model.chat(tokenizer, "晚上睡不着应该怎么办", history=[])
print(response)

参考链接