【ChatGLM】ChatGLM-6B模型Win+4GB显卡本地部署笔记

ChatGLM-6B是清华大学知识工程和数据挖掘小组发布的一个类似ChatGPT的开源对话机器人，由于该模型是经过约1T标识符的中英文训练，且大部分都是中文，因此十分适合国内使用。

预期环境

本机电脑备注： Win10专业版+ 32G内存+256固态系统盘+1T机械硬盘+4G英伟达显卡
python版本：3.10.7

在这里插入图片描述

下载代码GLM-6B

在这里插入图片描述

依赖Python库及版本

文件记录了ChatGLM-6B依赖的Python库及版本，如下：
在这里插入图片描述

命令执行

pip install -r requirements.txt

在这里插入图片描述

Lucky@Lucky MINGW64 /e/ikbp/ChatGLM-6B (main)
$ pip install -r requirements.txt
Collecting protobufDownloading protobuf-4.23.4-cp310-abi3-win_amd64.whl (422 kB)-------------------------------------- 422.5/422.5 kB 2.6 MB/s eta 0:00:00
Collecting transformers==4.27.1Downloading transformers-4.27.1-py3-none-any.whl (6.7 MB)---------------------------------------- 6.7/6.7 MB 10.0 MB/s eta 0:00:00
Collecting cpm_kernelsDownloading cpm_kernels-1.0.11-py3-none-any.whl (416 kB)------------------------------------- 416.6/416.6 kB 12.7 MB/s eta 0:00:00
Collecting torch>=1.10Downloading torch-2.0.1-cp310-cp310-win_amd64.whl (172.3 MB)-------------------------------------- 172.3/172.3 MB 3.3 MB/s eta 0:00:00
Collecting gradioDownloading gradio-3.36.1-py3-none-any.whl (19.8 MB)---------------------------------------- 19.8/19.8 MB 4.0 MB/s eta 0:00:00
Collecting mdtex2htmlDownloading mdtex2html-1.2.0-py3-none-any.whl (13 kB)
Collecting sentencepieceDownloading sentencepiece-0.1.99-cp310-cp310-win_amd64.whl (977 kB)-------------------------------------- 977.5/977.5 kB 8.9 MB/s eta 0:00:00
Collecting accelerateDownloading accelerate-0.20.3-py3-none-any.whl (227 kB)------------------------------------- 227.6/227.6 kB 14.5 MB/s eta 0:00:00
Collecting huggingface-hub<1.0,>=0.11.0Downloading huggingface_hub-0.16.4-py3-none-any.whl (268 kB)-------------------------------------- 268.8/268.8 kB 8.1 MB/s eta 0:00:00
Collecting tqdm>=4.27Downloading tqdm-4.65.0-py3-none-any.whl (77 kB)---------------------------------------- 77.1/77.1 kB 4.2 MB/s eta 0:00:00
Collecting filelockDownloading filelock-3.12.2-py3-none-any.whl (10 kB)
Collecting numpy>=1.17Downloading numpy-1.25.0-cp310-cp310-win_amd64.whl (15.0 MB)---------------------------------------- 15.0/15.0 MB 4.5 MB/s eta 0:00:00
Collecting requestsDownloading requests-2.31.0-py3-none-any.whl (62 kB)---------------------------------------- 62.6/62.6 kB ? eta 0:00:00
Collecting pyyaml>=5.1Downloading PyYAML-6.0-cp310-cp310-win_amd64.whl (151 kB)-------------------------------------- 151.7/151.7 kB 8.8 MB/s eta 0:00:00
Collecting packaging>=20.0Downloading packaging-23.1-py3-none-any.whl (48 kB)---------------------------------------- 48.9/48.9 kB ? eta 0:00:00
Collecting tokenizers!=0.11.3,<0.14,>=0.11.1Downloading tokenizers-0.13.3-cp310-cp310-win_amd64.whl (3.5 MB)---------------------------------------- 3.5/3.5 MB 11.1 MB/s eta 0:00:00
Collecting regex!=2019.12.17Downloading regex-2023.6.3-cp310-cp310-win_amd64.whl (268 kB)-------------------------------------- 268.0/268.0 kB 8.3 MB/s eta 0:00:00
Collecting typing-extensionsDownloading typing_extensions-4.7.1-py3-none-any.whl (33 kB)
Collecting sympyDownloading sympy-1.12-py3-none-any.whl (5.7 MB)---------------------------------------- 5.7/5.7 MB 7.6 MB/s eta 0:00:00
Collecting networkxDownloading networkx-3.1-py3-none-any.whl (2.1 MB)---------------------------------------- 2.1/2.1 MB 11.0 MB/s eta 0:00:00
Collecting jinja2Downloading Jinja2-3.1.2-py3-none-any.whl (133 kB)-------------------------------------- 133.1/133.1 kB 8.2 MB/s eta 0:00:00
Collecting altair>=4.2.0Downloading altair-5.0.1-py3-none-any.whl (471 kB)------------------------------------- 471.5/471.5 kB 14.4 MB/s eta 0:00:00
Collecting aiohttpDownloading aiohttp-3.8.4-cp310-cp310-win_amd64.whl (319 kB)-------------------------------------- 319.8/319.8 kB 9.7 MB/s eta 0:00:00
Collecting fastapiDownloading fastapi-0.100.0-py3-none-any.whl (65 kB)---------------------------------------- 65.7/65.7 kB ? eta 0:00:00
Collecting gradio-client>=0.2.7Downloading gradio_client-0.2.7-py3-none-any.whl (288 kB)-------------------------------------- 288.4/288.4 kB 8.7 MB/s eta 0:00:00
Collecting mdit-py-plugins<=0.3.3Downloading mdit_py_plugins-0.3.3-py3-none-any.whl (50 kB)---------------------------------------- 50.5/50.5 kB ? eta 0:00:00
Collecting orjsonDownloading orjson-3.9.2-cp310-none-win_amd64.whl (195 kB)------------------------------------- 195.7/195.7 kB 11.6 MB/s eta 0:00:00
Collecting httpxDownloading httpx-0.24.1-py3-none-any.whl (75 kB)---------------------------------------- 75.4/75.4 kB ? eta 0:00:00
Collecting markdown-it-py[linkify]>=2.0.0Downloading markdown_it_py-3.0.0-py3-none-any.whl (87 kB)---------------------------------------- 87.5/87.5 kB ? eta 0:00:00
Collecting markupsafeDownloading MarkupSafe-2.1.3-cp310-cp310-win_amd64.whl (17 kB)
Collecting aiofilesDownloading aiofiles-23.1.0-py3-none-any.whl (14 kB)
Collecting semantic-versionDownloading semantic_version-2.10.0-py2.py3-none-any.whl (15 kB)
Collecting uvicorn>=0.14.0Downloading uvicorn-0.22.0-py3-none-any.whl (58 kB)---------------------------------------- 58.3/58.3 kB 3.2 MB/s eta 0:00:00
Collecting matplotlibDownloading matplotlib-3.7.2-cp310-cp310-win_amd64.whl (7.5 MB)---------------------------------------- 7.5/7.5 MB 8.3 MB/s eta 0:00:00
Collecting pydubDownloading pydub-0.25.1-py2.py3-none-any.whl (32 kB)
Collecting pillowDownloading Pillow-10.0.0-cp310-cp310-win_amd64.whl (2.5 MB)---------------------------------------- 2.5/2.5 MB 11.4 MB/s eta 0:00:00
Collecting ffmpyDownloading ffmpy-0.3.0.tar.gz (4.8 kB)Preparing metadata (setup.py): startedPreparing metadata (setup.py): finished with status 'done'
Collecting pandasDownloading pandas-2.0.3-cp310-cp310-win_amd64.whl (10.7 MB)---------------------------------------- 10.7/10.7 MB 7.2 MB/s eta 0:00:00
Collecting pydanticDownloading pydantic-2.0.2-py3-none-any.whl (359 kB)------------------------------------- 359.1/359.1 kB 11.3 MB/s eta 0:00:00
Collecting websockets>=10.0Downloading websockets-11.0.3-cp310-cp310-win_amd64.whl (124 kB)---------------------------------------- 124.7/124.7 kB ? eta 0:00:00
Collecting pygments>=2.12.0Downloading Pygments-2.15.1-py3-none-any.whl (1.1 MB)---------------------------------------- 1.1/1.1 MB 10.3 MB/s eta 0:00:00
Collecting python-multipartDownloading python_multipart-0.0.6-py3-none-any.whl (45 kB)---------------------------------------- 45.7/45.7 kB 2.4 MB/s eta 0:00:00
Collecting markdownDownloading Markdown-3.4.3-py3-none-any.whl (93 kB)---------------------------------------- 93.9/93.9 kB ? eta 0:00:00
Collecting latex2mathmlDownloading latex2mathml-3.76.0-py3-none-any.whl (73 kB)---------------------------------------- 73.4/73.4 kB 3.9 MB/s eta 0:00:00
Collecting psutilDownloading psutil-5.9.5-cp36-abi3-win_amd64.whl (255 kB)------------------------------------- 255.1/255.1 kB 16.3 MB/s eta 0:00:00
Collecting jsonschema>=3.0Downloading jsonschema-4.18.0-py3-none-any.whl (81 kB)---------------------------------------- 81.5/81.5 kB ? eta 0:00:00
Collecting toolzDownloading toolz-0.12.0-py3-none-any.whl (55 kB)---------------------------------------- 55.8/55.8 kB 2.8 MB/s eta 0:00:00
Collecting fsspecDownloading fsspec-2023.6.0-py3-none-any.whl (163 kB)-------------------------------------- 163.8/163.8 kB 9.6 MB/s eta 0:00:00
Collecting mdurl~=0.1Downloading mdurl-0.1.2-py3-none-any.whl (10.0 kB)
Collecting linkify-it-py<3,>=1Downloading linkify_it_py-2.0.2-py3-none-any.whl (19 kB)
Collecting mdit-py-plugins<=0.3.3Downloading mdit_py_plugins-0.3.2-py3-none-any.whl (50 kB)---------------------------------------- 50.4/50.4 kB ? eta 0:00:00Downloading mdit_py_plugins-0.3.1-py3-none-any.whl (46 kB)---------------------------------------- 46.5/46.5 kB 2.3 MB/s eta 0:00:00Downloading mdit_py_plugins-0.3.0-py3-none-any.whl (43 kB)---------------------------------------- 43.7/43.7 kB ? eta 0:00:00Downloading mdit_py_plugins-0.2.8-py3-none-any.whl (41 kB)---------------------------------------- 41.0/41.0 kB ? eta 0:00:00Downloading mdit_py_plugins-0.2.7-py3-none-any.whl (41 kB)---------------------------------------- 41.0/41.0 kB ? eta 0:00:00Downloading mdit_py_plugins-0.2.6-py3-none-any.whl (39 kB)Downloading mdit_py_plugins-0.2.5-py3-none-any.whl (39 kB)Downloading mdit_py_plugins-0.2.4-py3-none-any.whl (39 kB)Downloading mdit_py_plugins-0.2.3-py3-none-any.whl (39 kB)Downloading mdit_py_plugins-0.2.2-py3-none-any.whl (39 kB)Downloading mdit_py_plugins-0.2.1-py3-none-any.whl (38 kB)Downloading mdit_py_plugins-0.2.0-py3-none-any.whl (38 kB)Downloading mdit_py_plugins-0.1.0-py3-none-any.whl (37 kB)
INFO: pip is looking at multiple versions of markdown-it-py[linkify] to determine which version is compatible with other requirements. This could take a while.
Collecting markdown-it-py[linkify]>=2.0.0Downloading markdown_it_py-2.2.0-py3-none-any.whl (84 kB)---------------------------------------- 84.5/84.5 kB ? eta 0:00:00
Collecting pytz>=2020.1Downloading pytz-2023.3-py2.py3-none-any.whl (502 kB)------------------------------------- 502.3/502.3 kB 10.5 MB/s eta 0:00:00
Collecting python-dateutil>=2.8.2Downloading python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)-------------------------------------- 247.7/247.7 kB 7.4 MB/s eta 0:00:00
Collecting tzdata>=2022.1Downloading tzdata-2023.3-py2.py3-none-any.whl (341 kB)------------------------------------- 341.8/341.8 kB 10.7 MB/s eta 0:00:00
Collecting coloramaDownloading colorama-0.4.6-py2.py3-none-any.whl (25 kB)
Collecting click>=7.0Downloading click-8.1.4-py3-none-any.whl (98 kB)---------------------------------------- 98.2/98.2 kB 5.5 MB/s eta 0:00:00
Collecting h11>=0.8Downloading h11-0.14.0-py3-none-any.whl (58 kB)---------------------------------------- 58.3/58.3 kB 3.0 MB/s eta 0:00:00
Collecting multidict<7.0,>=4.5Downloading multidict-6.0.4-cp310-cp310-win_amd64.whl (28 kB)
Collecting charset-normalizer<4.0,>=2.0Downloading charset_normalizer-3.2.0-cp310-cp310-win_amd64.whl (96 kB)---------------------------------------- 96.9/96.9 kB 5.4 MB/s eta 0:00:00
Collecting aiosignal>=1.1.2Downloading aiosignal-1.3.1-py3-none-any.whl (7.6 kB)
Collecting yarl<2.0,>=1.0Downloading yarl-1.9.2-cp310-cp310-win_amd64.whl (61 kB)---------------------------------------- 61.0/61.0 kB 3.2 MB/s eta 0:00:00
Collecting async-timeout<5.0,>=4.0.0a3Downloading async_timeout-4.0.2-py3-none-any.whl (5.8 kB)
Collecting attrs>=17.3.0Downloading attrs-23.1.0-py3-none-any.whl (61 kB)---------------------------------------- 61.2/61.2 kB ? eta 0:00:00
Collecting frozenlist>=1.1.1Downloading frozenlist-1.3.3-cp310-cp310-win_amd64.whl (33 kB)
Collecting starlette<0.28.0,>=0.27.0Downloading starlette-0.27.0-py3-none-any.whl (66 kB)---------------------------------------- 67.0/67.0 kB ? eta 0:00:00
Collecting pydantic-core==2.1.2Downloading pydantic_core-2.1.2-cp310-none-win_amd64.whl (1.5 MB)---------------------------------------- 1.5/1.5 MB 9.5 MB/s eta 0:00:00
Collecting annotated-types>=0.4.0Downloading annotated_types-0.5.0-py3-none-any.whl (11 kB)
Collecting certifiDownloading certifi-2023.5.7-py3-none-any.whl (156 kB)-------------------------------------- 157.0/157.0 kB 9.2 MB/s eta 0:00:00
Collecting httpcore<0.18.0,>=0.15.0Downloading httpcore-0.17.3-py3-none-any.whl (74 kB)---------------------------------------- 74.5/74.5 kB 4.0 MB/s eta 0:00:00
Collecting idnaDownloading idna-3.4-py3-none-any.whl (61 kB)---------------------------------------- 61.5/61.5 kB ? eta 0:00:00
Collecting sniffioDownloading sniffio-1.3.0-py3-none-any.whl (10 kB)
Collecting fonttools>=4.22.0Downloading fonttools-4.40.0-cp310-cp310-win_amd64.whl (1.9 MB)---------------------------------------- 1.9/1.9 MB 10.4 MB/s eta 0:00:00
Collecting pyparsing<3.1,>=2.3.1Downloading pyparsing-3.0.9-py3-none-any.whl (98 kB)---------------------------------------- 98.3/98.3 kB 5.5 MB/s eta 0:00:00
Collecting cycler>=0.10Downloading cycler-0.11.0-py3-none-any.whl (6.4 kB)
Collecting kiwisolver>=1.0.1Downloading kiwisolver-1.4.4-cp310-cp310-win_amd64.whl (55 kB)---------------------------------------- 55.3/55.3 kB 3.0 MB/s eta 0:00:00
Collecting contourpy>=1.0.1Downloading contourpy-1.1.0-cp310-cp310-win_amd64.whl (470 kB)-------------------------------------- 470.4/470.4 kB 9.8 MB/s eta 0:00:00
Collecting urllib3<3,>=1.21.1Downloading urllib3-2.0.3-py3-none-any.whl (123 kB)-------------------------------------- 123.6/123.6 kB 7.1 MB/s eta 0:00:00
Collecting mpmath>=0.19Downloading mpmath-1.3.0-py3-none-any.whl (536 kB)------------------------------------- 536.2/536.2 kB 11.2 MB/s eta 0:00:00
Collecting anyio<5.0,>=3.0Downloading anyio-3.7.1-py3-none-any.whl (80 kB)---------------------------------------- 80.9/80.9 kB ? eta 0:00:00
Collecting jsonschema-specifications>=2023.03.6Downloading jsonschema_specifications-2023.6.1-py3-none-any.whl (17 kB)
Collecting referencing>=0.28.4Downloading referencing-0.29.1-py3-none-any.whl (25 kB)
Collecting rpds-py>=0.7.1Downloading rpds_py-0.8.8-cp310-none-win_amd64.whl (180 kB)------------------------------------- 180.3/180.3 kB 11.3 MB/s eta 0:00:00
Collecting uc-micro-pyDownloading uc_micro_py-1.0.2-py3-none-any.whl (6.2 kB)
Collecting six>=1.5Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting exceptiongroupDownloading exceptiongroup-1.1.2-py3-none-any.whl (14 kB)
Using legacy 'setup.py install' for ffmpy, since package 'wheel' is not installed.
Installing collected packages: tokenizers, sentencepiece, pytz, pydub, mpmath, ffmpy, cpm_kernels, websockets, urllib3, uc-micro-py, tzdata, typing-extensions, toolz, sympy, sniffio, six, semantic-version, rpds-py, regex, pyyaml, python-multipart, pyparsing, pygments, psutil, protobuf, pillow, packaging, orjson, numpy, networkx, multidict, mdurl, markupsafe, markdown, latex2mathml, kiwisolver, idna, h11, fsspec, frozenlist, fonttools, filelock, exceptiongroup, cycler, colorama, charset-normalizer, certifi, attrs, async-timeout, annotated-types, aiofiles, yarl, tqdm, requests, referencing, python-dateutil, pydantic-core, mdtex2html, markdown-it-py, linkify-it-py, jinja2, contourpy, click, anyio, aiosignal, uvicorn, torch, starlette, pydantic, pandas, mdit-py-plugins, matplotlib, jsonschema-specifications, huggingface-hub, httpcore, aiohttp, transformers, jsonschema, httpx, fastapi, accelerate, gradio-client, altair, gradioRunning setup.py install for ffmpy: startedRunning setup.py install for ffmpy: finished with status 'done'
Successfully installed accelerate-0.20.3 aiofiles-23.1.0 aiohttp-3.8.4 aiosignal-1.3.1 altair-5.0.1 annotated-types-0.5.0 anyio-3.7.1 async-timeout-4.0.2 attrs-23.1.0 certifi-2023.5.7 charset-normalizer-3.2.0 click-8.1.4 colorama-0.4.6 contourpy-1.1.0 cpm_kernels-1.0.11 cycler-0.11.0 exceptiongroup-1.1.2 fastapi-0.100.0 ffmpy-0.3.0 filelock-3.12.2 fonttools-4.40.0 frozenlist-1.3.3 fsspec-2023.6.0 gradio-3.36.1 gradio-client-0.2.7 h11-0.14.0 httpcore-0.17.3 httpx-0.24.1 huggingface-hub-0.16.4 idna-3.4 jinja2-3.1.2 jsonschema-4.18.0 jsonschema-specifications-2023.6.1 kiwisolver-1.4.4 latex2mathml-3.76.0 linkify-it-py-2.0.2 markdown-3.4.3 markdown-it-py-2.2.0 markupsafe-2.1.3 matplotlib-3.7.2 mdit-py-plugins-0.3.3 mdtex2html-1.2.0 mdurl-0.1.2 mpmath-1.3.0 multidict-6.0.4 networkx-3.1 numpy-1.25.0 orjson-3.9.2 packaging-23.1 pandas-2.0.3 pillow-10.0.0 protobuf-4.23.4 psutil-5.9.5 pydantic-2.0.2 pydantic-core-2.1.2 pydub-0.25.1 pygments-2.15.1 pyparsing-3.0.9 python-dateutil-2.8.2 python-multipart-0.0.6 pytz-2023.3 pyyaml-6.0 referencing-0.29.1 regex-2023.6.3 requests-2.31.0 rpds-py-0.8.8 semantic-version-2.10.0 sentencepiece-0.1.99 six-1.16.0 sniffio-1.3.0 starlette-0.27.0 sympy-1.12 tokenizers-0.13.3 toolz-0.12.0 torch-2.0.1 tqdm-4.65.0 transformers-4.27.1 typing-extensions-4.7.1 tzdata-2023.3 uc-micro-py-1.0.2 urllib3-2.0.3 uvicorn-0.22.0 websockets-11.0.3 yarl-1.9.2[notice] A new release of pip available: 22.2.2 -> 23.1.2
[notice] To update, run: python.exe -m pip install --upgrade pip

预训练文件

下载INT4量化后的预训练结果文件，INT4量化的预训练文件下载地址：https://huggingface.co/THUDM/chatglm-6b-int4/tree/main。需要注意的是在GitHub上官方提供了模型在清华云上的下载地址，但是那个只包含预训练结果文件，即bin文件，但实际上ChatGLM-6B的运行需要模型的配置文件，即config.json等，如下图所示：
在这里插入图片描述

从HuggingFace上下载所有文件到本地。上述文件全部下载之后保存到本地的一个目录下即可，我保存在：E:\ikbp\chatglm-6b-int4
在这里插入图片描述

如果以上代码输出的是True，那么恭喜你，你安装的是cuda版本的torch（注意，有显卡也需要大家下载cuda和cudann安装成功才可以，这部分大家可以去网上找教程）。

在这里插入图片描述
##Win+GPU部署方案

Win+GPU方案的必备条件

部署GPU版本的ChatGLM-6B需要安装cuda版本的torch，大家需要检测自己的torch是否正确，可以通过如下命令检查（下面是python代码）：

pip install torch

在这里插入图片描述

下载cuda和cudann安装

win10安装CUDA和cuDNN详解

下载和安装CUDA

下载CUDA
下载地址：https://developer.nvidia.com/cuda-downloads

直接终端输入命令nvcc --version，即可显示

下载和安装cuDNN

下载cuDNN
下载地址：https://developer.nvidia.com/rdp/cudnn-download

在这里插入图片描述
选择合适的版本下载就行啦，注意：一定要选择和你安装的CUDA匹配的版本。

在这里插入图片描述

附件

https://online2023.worldaic.com.cn/exhibition
ChatGLM-6B模型——Windows+6GB显卡本地部署
ChatGLM-6B/blob/main/requirements.txt
THUDM chatglm-6b-int4
win10安装CUDA和cuDNN详解
CUDA Toolkit 12.2 Downloads
cuDNN Archive