在 MacOS 上跑 kaldi

categories: [asr]
tags: C++ asr kaldi

在MacOS 下跑 kaldi

brew install automake llvm cmake sox libtool subversion

基本安装

Common build problems · pyenv/pyenv Wiki;

brew install pyenv
pyenv install -v 2.7.18# Set the python version.pyenv global 2.7.18
# Export PATH if necessary.export PATH="$(pyenv root)/shims:${PATH}"
Add if necessary.:echo 'PATH=$(pyenv root)/shims:$PATH' >> ~/.zshrc

还得是用 xcode 的 sdk, brew 很多都显示找不到对应的包, 用了 CPPFLAG也不行

 ==> /opt/homebrew/bin/pyenv install 2.7.18
Downloading openssl-1.1.1v.tar.gz...
-> https://www.openssl.org/source/openssl-1.1.1v.tar.gz
Installing openssl-1.1.1v...
Installed openssl-1.1.1v to /Users/zorch/.pyenv/versions/2.7.18
Downloading readline-8.0.tar.gz...
-> https://ftpmirror.gnu.org/readline/readline-8.0.tar.gz
Installing readline-8.0...
Installed readline-8.0 to /Users/zorch/.pyenv/versions/2.7.18
Downloading Python-2.7.18.tar.xz...
-> https://www.python.org/ftp/python/2.7.18/Python-2.7.18.tar.xz
Installing Python-2.7.18...
patching file configure
patching file configure.ac
patching file setup.py
patching file 'Mac/Tools/pythonw.c'
patching file setup.py
patching file 'Doc/library/ctypes.rst'
patching file 'Lib/test/test_str.py'
patching file 'Lib/test/test_unicode.py'
patching file 'Modules/_ctypes/_ctypes.c'
patching file 'Modules/_ctypes/callproc.c'
patching file 'Modules/_ctypes/ctypes.h'
patching file 'Modules/_ctypes/callproc.c'
patching file setup.py
patching file 'Mac/Modules/qt/setup.py'
patching file setup.py
python-build: use zlib from xcode sdk
Installed Python-2.7.18 to /Users/zorch/.pyenv/versions/2.7.18

还需要加入环境变量

 ==> pyenv global 2.7.18 √  ~/code/c-cpp_proj/kaldi/tools==> python2
Python 2.7.18 (default, Dec 16 2023, 14:23:06)
[GCC Apple LLVM 15.0.0 (clang-1500.1.0.2.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>==> which python2
/Users/zorch/.pyenv/shims/python2

安装 openblas

gfortran 11 ARM-darwin (Apple M1) build failure · Issue #3222 · OpenMathLib/OpenBLAS;

改版本

 ==> vi extras/install_openblas.shOPENBLAS_VERSION=0.3.20

之前是 0.3.13

 ==> sw_vers -productVersion
14.1
 √  ~/code/c-cpp_proj/kaldi/tools==> MACOSX_DEPLOYMENT_TARGET=14.1 extras/install_openblas.sh

结果

OpenBLAS build complete. (BLAS CBLAS LAPACK LAPACKE)OS               ... DarwinArchitecture     ... arm64BINARY           ... 64bitC compiler       ... CLANG  (cmd & version : Apple clang version 15.0.0 (clang-1500.1.0.2.5))Fortran compiler ... GFORTRAN  (cmd & version : GNU Fortran (Homebrew GCC 13.2.0) 13.2.0)
-n   Library Name     ... libopenblas_armv8-r0.3.20.a(Single-threading)
WARNING: If you plan to use the dynamic library libopenblas_armv8-r0.3.20.dylib, you must run:"make PREFIX=/your_installation_path/ install".(or set PREFIX in Makefile.rule and run make install.
If you want to move the .dylib to a new location later, make sure you change
the internal name of the dylib with:install_name_tool -id /new/absolute/path/to/libopenblas_armv8-r0.3.20.dylib libopenblas_armv8-r0.3.20.dylibTo install the library, you can run "make PREFIX=/path/to/your/installation install"./Applications/Xcode.app/Contents/Developer/usr/bin/make -j 11 -f Makefile.install install
Generating openblas_config.h in /Users/zorch/code/c-cpp_proj/kaldi/tools/OpenBLAS/install/include
Generating f77blas.h in /Users/zorch/code/c-cpp_proj/kaldi/tools/OpenBLAS/install/include
Generating cblas.h in /Users/zorch/code/c-cpp_proj/kaldi/tools/OpenBLAS/install/include
Copying LAPACKE header files to /Users/zorch/code/c-cpp_proj/kaldi/tools/OpenBLAS/install/include
Copying the static library to /Users/zorch/code/c-cpp_proj/kaldi/tools/OpenBLAS/install/lib
Copying the shared library to /Users/zorch/code/c-cpp_proj/kaldi/tools/OpenBLAS/install/lib
Generating openblas.pc in /Users/zorch/code/c-cpp_proj/kaldi/tools/OpenBLAS/install/lib/pkgconfig
Generating OpenBLASConfig.cmake in /Users/zorch/code/c-cpp_proj/kaldi/tools/OpenBLAS/install/lib/cmake/openblas
Generating OpenBLASConfigVersion.cmake in /Users/zorch/code/c-cpp_proj/kaldi/tools/OpenBLAS/install/lib/cmake/openblas
Install OK!
OpenBLAS is installed successfully.

issue 永远滴神

检验

 ==> extras/check_dependencies.sh
extras/check_dependencies.sh: all OK.

正餐开始

# optional, exec if you have built the old version
# make -j clean depend
cd src/
./configure --use-cuda=no
make -j11

跑一个例子试试

cd egs/yesno/s5==> make test -j10

更新: 2024-11-06

最新的编译不用这么麻烦了, 安装python2 之后直接用 cmake 安装即可(看 INSTALL.md)

vi cmake/INSTALL.md## Build with Ninja Generator
```bash
mkdir -p build && cd build
cmake -GNinja -DCMAKE_INSTALL_PREFIX=../dist ..
cmake --build . --target install
```执行之后会在`./dist/bin` 中生成可执行的工具

安装之后有个小坑, 我这里环境是 MacOS14.6 的, m2 , 执行下面的测试脚本之后提示 dylib 找不到

cd  egs/yesno/s5
./run.shdyld[90630]: Library not loaded: @rpath/libfstscript.16.dylibReferenced from: <6A97203D-EF56-3A63-A568-9B24B38894CF> /Users/xx/code/asr_learning/kaldi/dist/bin/fstcompileReason: no LC_RPATH's found
zsh: abort

但是这个共享库是存在的, 后来问了一下 GPT, 用下面的脚本给填了一下 rpath 就可以执行了

for i in `ls -l ~/code/asr_learning/kaldi/dist/bin | tr -s ' ' | cut  -d' ' -f9`; 
do
install_name_tool -add_rpath /Users/x/code/asr_learning/kaldi/dist/lib "/Users/x/code/asr_learning/kaldi/dist/bin/$i" ;
done

注意这里用绝对路径, 否则找不到, 用户名改成自己的.

最后的部分日志信息.

fstdeterminizestar --use-log=true
fsttablecompose data/lang_test_tg/L_disambig.fst data/lang_test_tg/G.fst
fstminimizeencoded
fstpushspecial
fstisstochastic data/lang_test_tg/tmp/LG.fst
0.534295 0.533859
[info]: LG not stochastic.
fstcomposecontext --context-size=1 --central-position=0 --read-disambig-syms=data/lang_test_tg/phones/disambig.int --write-disambig-syms=data/lang_test_tg/tmp/disambig_ilabels_1_0.int data/lang_test_tg/tmp/ilabels_1_0.6521 data/lang_test_tg/tmp/LG.fst
fstisstochastic data/lang_test_tg/tmp/CLG_1_0.fst
0.534295 0.533859
[info]: CLG not stochastic.
make-h-transducer --disambig-syms-out=exp/mono0a/graph_tgpr/disambig_tid.int --transition-scale=1.0 data/lang_test_tg/tmp/ilabels_1_0 exp/mono0a/tree exp/mono0a/final.mdl
fstminimizeencoded
fsttablecompose exp/mono0a/graph_tgpr/Ha.fst data/lang_test_tg/tmp/CLG_1_0.fst
fstdeterminizestar --use-log=true
fstrmsymbols exp/mono0a/graph_tgpr/disambig_tid.int
fstrmepslocal
fstisstochastic exp/mono0a/graph_tgpr/HCLGa.fst
0.5342 -0.000144671
HCLGa is not stochastic
add-self-loops --self-loop-scale=0.1 --reorder=true exp/mono0a/final.mdl exp/mono0a/graph_tgpr/HCLGa.fst
steps/decode.sh --nj 1 --cmd utils/run.pl exp/mono0a/graph_tgpr data/test_yesno exp/mono0a/decode_test_yesno
decode.sh: feature type is delta
steps/diagnostic/analyze_lats.sh --cmd utils/run.pl exp/mono0a/graph_tgpr exp/mono0a/decode_test_yesno
steps/diagnostic/analyze_lats.sh: see stats in exp/mono0a/decode_test_yesno/log/analyze_alignments.log
Overall, lattice depth (10,50,90-percentile)=(1,1,1) and mean=1.1
steps/diagnostic/analyze_lats.sh: see stats in exp/mono0a/decode_test_yesno/log/analyze_lattice_depth_stats.log
local/score.sh --cmd utils/run.pl data/test_yesno exp/mono0a/graph_tgpr exp/mono0a/decode_test_yesno
local/score.sh: scoring with word insertion penalty=0.0,0.5,1.0
%WER 0.00 [ 0 / 232, 0 in , 0 del, 0  ub ] exp/mono0a/decode_te t_ye no/wer_11_1.0