1 下载CUDA和cudnn

详见文章

CUDA与CUDNN在Windows下的安装与配置（超级详细版）_windows cudnn安装-CSDN博客

我电脑的CUDA下载链接如下

​​​​​https://developer.nvidia.com/cuda-12-1-0-download-archive?target_os=Windows&target_arch=x86_64&target_version=11&target_type=exe_local

安装成功nvcc -V命令显示如下内容

PS D:\Downloads> nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2023 NVIDIA Corporation
Built on Wed_Feb__8_05:53:42_Coordinated_Universal_Time_2023
Cuda compilation tools, release 12.1, V12.1.66
Build cuda_12.1.r12.1/compiler.32415258_0 

我电脑的cudnn下载链接如下

https://developer.nvidia.com/cudnn-downloads?target_os=Windows&target_arch=x86_64&target_version=Agnostic&cuda_version=12

将cudnn-windows-x86_64-9.2.0.82_cuda12-archive里的对应bin、lib、include三个文件内容合并到C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.1即可

打开C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.1\extras\demo_suite，查看是否有文件bandwidthTest.exe以及deviceQuery.exe，若存在则在该文件中打开cmd运行两.exe文件

2 安装相关python包

1) Clone or download

https://github.com/oobabooga/text-generation-webui/archive/refs/heads/main.zip

2) Run the start_windows.bat

这里会运行text-generation-webui\one_click.py会显示详细的安装内容

其中会根据cuda的版本安装对于的pytorch 如下所示

pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121

具体的下载路径如下

https://download.pytorch.org/whl/cu121/torch-2.2.1%2Bcu121-cp311-cp311-win_amd64.whl

之后下载\text-generation-webui\的requirements.txt内容

Command '"D:\Qwen2\text-generation-webui\installer_files\conda\condabin\conda.bat" activate "D:\Qwen2\text-generation-webui\installer_files\env" >nul && python -m pip install -r temp_requirements.txt --upgrade'

Looking in indexes: https://download.pytorch.org/whl/cu121
Collecting torch==2.2.1
  Downloading https://download.pytorch.org/whl/cu121/torch-2.2.1%2Bcu121-cp311-cp311-win_amd64.whl (2454.8 MB)
     ---------------------------------------- 2.5/2.5 GB 241.9 kB/s eta 0:00:00
Collecting torchvision==0.17.1
  Downloading https://download.pytorch.org/whl/cu121/torchvision-0.17.1%2Bcu121-cp311-cp311-win_amd64.whl (5.7 MB)
     ---------------------------------------- 5.7/5.7 MB 262.4 kB/s eta 0:00:00
Collecting torchaudio==2.2.1
  Downloading https://download.pytorch.org/whl/cu121/torchaudio-2.2.1%2Bcu121-cp311-cp311-win_amd64.whl (4.1 MB)
     ---------------------------------------- 4.1/4.1 MB 271.9 kB/s eta 0:00:00
Collecting filelock (from torch==2.2.1)
  Downloading https://download.pytorch.org/whl/filelock-3.13.1-py3-none-any.whl (11 kB)
Collecting typing-extensions>=4.8.0 (from torch==2.2.1)
  Downloading https://download.pytorch.org/whl/typing_extensions-4.9.0-py3-none-any.whl (32 kB)
Collecting sympy (from torch==2.2.1)
  Downloading https://download.pytorch.org/whl/sympy-1.12-py3-none-any.whl (5.7 MB)
     ---------------------------------------- 5.7/5.7 MB 269.3 kB/s eta 0:00:00
Collecting networkx (from torch==2.2.1)
  Downloading https://download.pytorch.org/whl/networkx-3.2.1-py3-none-any.whl (1.6 MB)
     ---------------------------------------- 1.6/1.6 MB 260.7 kB/s eta 0:00:00
Collecting jinja2 (from torch==2.2.1)
  Downloading https://download.pytorch.org/whl/Jinja2-3.1.3-py3-none-any.whl (133 kB)
     ---------------------------------------- 133.2/133.2 kB 1.6 MB/s eta 0:00:00
Collecting fsspec (from torch==2.2.1)
  Downloading https://download.pytorch.org/whl/fsspec-2024.2.0-py3-none-any.whl (170 kB)
     ---------------------------------------- 170.9/170.9 kB 791.7 kB/s eta 0:00:00
Collecting numpy (from torchvision==0.17.1)
  Downloading https://download.pytorch.org/whl/numpy-1.26.3-cp311-cp311-win_amd64.whl (15.8 MB)
     ---------------------------------------- 15.8/15.8 MB 265.2 kB/s eta 0:00:00
Collecting pillow!=8.3.*,>=5.3.0 (from torchvision==0.17.1)
  Downloading https://download.pytorch.org/whl/pillow-10.2.0-cp311-cp311-win_amd64.whl (2.6 MB)
     ---------------------------------------- 2.6/2.6 MB 270.4 kB/s eta 0:00:00
Collecting MarkupSafe>=2.0 (from jinja2->torch==2.2.1)
  Downloading https://download.pytorch.org/whl/MarkupSafe-2.1.5-cp311-cp311-win_amd64.whl (17 kB)
Collecting mpmath>=0.19 (from sympy->torch==2.2.1)
  Downloading https://download.pytorch.org/whl/mpmath-1.3.0-py3-none-any.whl (536 kB)
     ---------------------------------------- 536.2/536.2 kB 255.1 kB/s eta 0:00:00
Installing collected packages: mpmath, typing-extensions, sympy, pillow, numpy, networkx, MarkupSafe, fsspec, filelock, jinja2, torch, torchvision, torchaudio
Successfully installed MarkupSafe-2.1.5 filelock-3.13.1 fsspec-2024.2.0 jinja2-3.1.3 mpmath-1.3.0 networkx-3.2.1 numpy-1.26.3 pillow-10.2.0 sympy-1.12 torch-2.2.1+cu121 torchaudio-2.2.1+cu121 torchvision-0.17.1+cu121 typing-extensions-4.9.0Collecting py-cpuinfo==9.0.0
  Downloading https://pypi.tuna.tsinghua.edu.cn/packages/e0/a9/023730ba63db1e494a271cb018dcd361bd2c917ba7004c3e49d5daf795a2/py_cpuinfo-9.0.0-py3-none-any.whl (22 kB)
Installing collected packages: py-cpuinfo
Successfully installed py-cpuinfo-9.0.0

*******************************************************************
* Updating the local copy of the repository with "git pull"
*******************************************************************

Already up to date.

*******************************************************************
* Installing webui requirements from file: requirements.txt
*******************************************************************

TORCH: 2.2.1+cu121

Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Ignoring aqlm: markers 'platform_system == "Linux"' don't match your environment
Ignoring llama-cpp-python: markers 'platform_system == "Linux" and platform_machine == "x86_64" and python_version == "3.11"' don't match your environment
Ignoring llama-cpp-python: markers 'platform_system == "Linux" and platform_machine == "x86_64" and python_version == "3.10"' don't match your environment
Collecting llama-cpp-python==0.2.75+cpuavx2 (from -r temp_requirements.txt (line 40))