编译py-faster-rcnn,花费了好几天,中间遇到好多问题,今天终于成功编译。下面详述我的整个编译过程。
【注记:】其实下面的依赖库可以安装在统一的一个本地目录下,相关安装指南,可以参考《深度学习:21天实战caffe》的第5天那一章。 (这也是我后来才知道的。)
部分软件见百度云盘:链接: https://pan.baidu.com/s/1kwucs_pDqM-5vN6NbVuLNg 密码: fk0v
- centos6.9
- 8块Titan xp显卡
[关于nvidia显卡的对比,可以查看《nvidia显卡对比分析》]
软件情况
-
boost 1.41
关于boost的情况,虽然caffe官网要求boost>=1.55,但是似乎对于低版本的也没有出现什么问题,如果要安装boost,可以参考《centos 安装boost》,切记要原封不动地安装如上说明去装,否则有可能只装上头文件,却没有动态库.就装在默认目录下即可。
一般安装好以后,头文件在/usr/include/boost,lib文件在/usr/lib64/boost 。 -
OpenBlas
-
CUDA8.0
-
cudnn v6 (如果没有安装,可以自己装在home目录下,可参考< cuda、cudnn相关问题链接>)
查看相应版本号:
cuda 版本
cat /usr/local/cuda/version.txt cudnn 版本
cat /usr/local/cuda/include/cudnn.h | grep CUDNN_MAJOR -A 2
- glog
- gflags
注意要先安装gflags,再安装glog。安装后者时,注意指定前者的路径,如下:
./configure --prefix=/home/caiyong.wang/bin/caffe_local_install/ CPPFLAGS="-I/home/caiyong.wang/bin/caffe_local_install/include" LDFLAGS="-L/home/caiyong.wang/bin/caffe_local_install/lib"
- leveldb
安装可以使用yum install
sudo yum install leveldb-devel
或者参考《levelDB的安装-Linux》
- lmdb
使用pip安装:
pip install lmdb
- snappy
首先下载snappy-1.1.1,然后解压,配置:./configure --prefix=/home/**/
最后编译安装: make && make install
以上关于依赖库的安装,可以参考:RHEL / Fedora / CentOS Installation
安装依赖库
- anaconda2
首先安装anaconda2,安装方法很简单,就是下载软件,然后运行bash Anaconda2-5.0.0.1-Linux-x86_64.sh
,接着安装过程中,需要指定安装路径和选择加入path环境变量(填yes),即可。重启后,发现python变成anaconda的版本。
- opencv-python
使用pip安装:
pip install opencv-python ,最新版为opencv3
安装后
>>> import cv2
>>> print cv2.__version__
注意:在caffe中不用麻烦地安装源码装opencv,会很麻烦,要装一堆依赖库,而且很多需要sudo权限,
请参考< linux 安装python-opencv>
3. easydict
pip install easydict
使用上述命令即可安装成功。
4. protobuf 2.6.1
首先先确认系统是否有装protobuf ,
查看方法:
$ protoc --version
libprotoc 2.3.0
如果没有显示,则说明没有装,另外我们期望装2.6.1(caffe-master使用的protobuf版本是2.5.0及以上),所以版本不符,就需要重装。
注意要装两个probobuf,一个是python的,一个是系统的。而且最好安装2.6.1,比较稳定。
由于anaconda默认没有装protobuf,所以可以使用pip安装指定版本的protobuf。
[python版本]
pip install protobuf==2.6.1
[系统]
从http://download.csdn.net/download/liangyihuai/9534593下载protobuf-2.6.1或从其他地方下载protobuf-2.6.1·.tar.gz。
认真阅读gitHub上给出的安装教程。
$./autogen.sh(在csdn下载的可以省略)
$./configure --prefix=/home/**/protobuf (自己的目录下)
$ make
$ make check
$ make install
编译成功后将export PATH=/home/**/protobuf/bin:$PATH加入到环境变量中
输入 protoc --version命令,如显示protobuf 2.6.1则安装成功。
$ protoc --version
libprotoc 2.6.1
最后进入caffe根目录,修改Makefile.config,在INCLUDE_DIRS后面加入/home//protobuf/include
LIBRARY_DIRS 后面加入 /home//protobuf/lib。
【注意:】
在向path中添加变量的时候,我总结一个原则,即:如果系统的路径,比如/usr/local/bin等,就这样写:
export PATH=$PATH:/usr/local/bin
or
export PATH=/usr/local/bin:$PATH
如果是自己安装的路径,就:
export PATH=/home/**/protobuf/bin:$PATH
也就是一个在前一个在后的问题。之所以强调这个,是因为有时候系统已经安装了某个软件,如果你想覆盖它,则在path中必须写在它的前面。因此对于自己安装的路径,一般我们都想覆盖原来的软件,所以写在path前面。,而系统安装的软件只有一个,写在前后都可以。我们可以查看echo $PATH
来看最后的path路径结果,并在需要的时候,在~/.bashrc中做相应地调整。一定要注意:
对于含有两个安装位置的软件,如果我们想先读某一个,一定要注意在path的位置。
make check的结果:
5. hdf5
由于前面安装过 anaconda ,所以hdf5已经默认安装。只需要把/*/anaconda2/lib 加入~/.bashrc中,即:
export LD_LIBRARY_PATH=/*/anaconda2/lib:$LD_LIBRARY_PATH
便于搜索hdf5的lib,尤其是 libhdf5_hl.so.100 ,若在后面的编译中仍然找不到此lib的话,只能在home目录下重新安装,参考:http://blog.csdn.net/xiamentingtao/article/details/78266153
6. 关于numpy的版本问题
根据py-faster-rcnn的要求:
numpy>=1.7.1
但是anaconda最新默认装的numpy的版本在1.13.1,虽然符合要求,但是https://github.com/rbgirshick/py-faster-rcnn/issues/480 ,最好numpy的版本为1.11.0,我们可以使用如下命令降级:
pip install -U numpy==1.11.0
下载py-faster-rcnn
克隆Faster R-CNN仓库:
# Make sure to clone with --recursive
git clone --recursive https://github.com/rbgirshick/py-faster-rcnn.git
–recursive :是为了递归下载caffe-master
继续安装python依赖库
将上面的克隆的文件放在安装目录下,这里我们用FRCN_ROOT指 py-faster-rcnn的路径,也就是
进入根目录,然后进入py-faster-rcnn/caffe-fast-rcnn/python中,caffe-fast-rcnn/python/requirement.txt文件列出了pycaffe的所有依赖库,要把requirements下的依赖都装一遍
Cython>=0.19.2
numpy>=1.7.1
scipy>=0.13.2
scikit-image>=0.9.3
matplotlib>=1.3.1
ipython>=3.0.0
h5py>=2.2.0
leveldb>=0.191
networkx>=1.8.1
nose>=1.3.0
pandas>=0.12.0
python-dateutil>=1.4,<2
protobuf>=2.5.0
python-gflags>=2.0
pyyaml>=3.10
Pillow>=2.3.0
six>=1.1.0
执行bash命令:
$for req in $(cat requirements.txt); do pip install $req; done
如果安装完成,可以再运行一遍上述命令,发现全部装好:
运行pip list可以列出相应地版本号,要注意与上面说的相符,尤其是:protobuf 2.6.1
到目前为止,我们可以对照caffe 官网的install,检测所有必须的依赖库。
合版本
根据《使用cuDNN5编译py-faster-rcnn错误:cudnn.hpp(126): error: argument of type “int” is incompatible …》的要求去操作即可。
或者参考http://blog.csdn.net/u010733679/article/details/52221404 更新最新的cudnn支持。
编译Cython模块
cd $FRCN_ROOT/lib
make
如果出现问题,参考:《 编译py-faster-rcnn的问题汇总及解决方法》.
编译caffe& pycaffe
正如caffe官网-install所说,在编译前有两种方法可以配置build,
第一种就是常见的手动设置Makefile.(我所采用的)
cp Makefile.config.example Makefile.config
# Adjust Makefile.config (for example, if using Anaconda Python, or if cuDNN is desired)
make all
make test
make runtest
make pycaffe
另一种就是先用cmake(版本>=2.8.7)自动检测各依赖项位置,生成Makefile.也可以自己设置。使用cmake-gui可能更容易些。
mkdir build
cd build
#编译CPU版本
#cmake -DCPU_ONLY=ON -DBLAS=Open .
#编译GPU版本
$cmake -DBLAS=Open -DCUDA_NVCC_FLAGS=--Wno-deprecated-gpu-targets ..
make all
make install
make runtest
make pycaffe
下面给出我自己配置的Makefile.config文件:
## Refer to http://caffe.berkeleyvision.org/installation.html
# Contributions simplifying and improving our build system are welcome!# cuDNN acceleration switch (uncomment to build with cuDNN).USE_CUDNN := 1# CPU-only switch (uncomment to build without GPU support).
# CPU_ONLY := 1# uncomment to disable IO dependencies and corresponding data layersUSE_OPENCV := 0USE_LEVELDB := 1USE_LMDB := 1# uncomment to allow MDB_NOLOCK when reading LMDB files (only if necessary)
# You should not set this flag if you will be reading LMDBs with any
# possibility of simultaneous read and write
# ALLOW_LMDB_NOLOCK := 1# Uncomment if you're using OpenCV 3
# OPENCV_VERSION := 3# To customize your choice of compiler, uncomment and set the following.
# N.B. the default for Linux is g++ and the default for OSX is clang++
# CUSTOM_CXX := g++# CUDA directory contains bin/ and lib/ directories that we need.
CUDA_DIR := /usr/local/cuda
# On Ubuntu 14.04, if cuda tools are installed via
# "sudo apt-get install nvidia-cuda-toolkit" then use this instead:
# CUDA_DIR := /usr# CUDA architecture setting: going with all of them.
# For CUDA < 6.0, comment the *_50 lines for compatibility.
CUDA_ARCH := -gencode arch=compute_20,code=sm_20 \-gencode arch=compute_20,code=sm_21 \-gencode arch=compute_30,code=sm_30 \-gencode arch=compute_35,code=sm_35 \-gencode arch=compute_50,code=sm_50 \-gencode arch=compute_50,code=compute_50# BLAS choice:
# atlas for ATLAS (default)
# mkl for MKL
# open for OpenBlas
BLAS := open
# Custom (MKL/ATLAS/OpenBLAS) include and lib directories.
# Leave commented to accept the defaults for your choice of BLAS
# (which should work)!
BLAS_INCLUDE := /opt/OpenBLAS/include
BLAS_LIB := /opt/OpenBLAS/lib# Homebrew puts openblas in a directory that is not on the standard search path
# BLAS_INCLUDE := $(shell brew --prefix openblas)/include
# BLAS_LIB := $(shell brew --prefix openblas)/lib# This is required only if you will compile the matlab interface.
# MATLAB directory should contain the mex binary in /bin.
# MATLAB_DIR := /data1/caiyong.wang/bin/matlab
# MATLAB_DIR := /Applications/MATLAB_R2012b.app# NOTE: this is required only if you will compile the python interface.
# We need to be able to find Python.h and numpy/arrayobject.h.
# PYTHON_INCLUDE := /usr/include/python2.7 \
# /usr/lib/python2.7/dist-packages/numpy/core/include
# Anaconda Python distribution is quite popular. Include path:
# Verify anaconda location, sometimes it's in root.ANACONDA_HOME := /data1/caiyong.wang/bin/anaconda2PYTHON_INCLUDE := $(ANACONDA_HOME)/include \$(ANACONDA_HOME)/include/python2.7 \$(ANACONDA_HOME)/lib/python2.7/site-packages/numpy/core/include \# Uncomment to use Python 3 (default is Python 2)
# PYTHON_LIBRARIES := boost_python3 python3.5m
# PYTHON_INCLUDE := /usr/include/python3.5m \
# /usr/lib/python3.5/dist-packages/numpy/core/include# We need to be able to find libpythonX.X.so or .dylib.
# PYTHON_LIB := /usr/libPYTHON_LIB := $(ANACONDA_HOME)/lib# Homebrew installs numpy in a non standard path (keg only)
# PYTHON_INCLUDE += $(dir $(shell python -c 'import numpy.core; print(numpy.core.__file__)'))/include
# PYTHON_LIB += $(shell brew --prefix numpy)/lib# Uncomment to support layers written in Python (will link against Python libs)WITH_PYTHON_LAYER := 1# Whatever else you find you need goes here.
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include /data1/caiyong.wang/bin/hdf5-1.10.1/hdf5/include /data1/caiyong.wang/bin/protobuf/include
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib /data1/caiyong.wang/bin/hdf5-1.10.1/hdf5/lib /data1/caiyong.wang/bin/protobuf/lib# If Homebrew is installed at a non standard location (for example your home directory) and you use it for general dependencies
INCLUDE_DIRS += $(shell brew --prefix)/include
LIBRARY_DIRS += $(shell brew --prefix)/lib # Uncomment to use `pkg-config` to specify OpenCV library paths.
# (Usually not necessary -- OpenCV libraries are normally installed in one of the above $LIBRARY_DIRS.)
# USE_PKG_CONFIG := 1BUILD_DIR := build
DISTRIBUTE_DIR := distribute# Uncomment for debugging. Does not work on OSX due to https://github.com/BVLC/caffe/issues/171
# DEBUG := 1# The ID of the GPU that 'make runtest' will use to run unit tests.
TEST_GPUID := 0# enable pretty build (comment to see full commands)
Q ?= @
需要注意的是我禁掉了opencv,并且 加入了一些额外的搜索路径,
至于到底是否需要增加,我也不清楚。也许不用添加也可以。你可以自己尝试。
下面再贴出我的~/.bashrc的内容:
# .bashrc# Source global definitions
if [ -f /etc/bashrc ]; then. /etc/bashrc
fi# User specific aliases and functionsalias matlab='/data1/caiyong.wang/bin/matlab/bin/matlab'
export PYTHONPATH=/data1/caiyong.wang/program/faster_rcnn/py-faster-rcnn/caffe-fast-rcnn/python:$PYTHONPATHexport LD_LIBRARY_PATH=/usr/local/cuda-8.0/lib64:$LD_LIBRARY_PATH# libglog
export LD_LIBRARY_PATH=/usr/local/lib/:$LD_LIBRARY_PATH# hdf5 (其实可以省略,因为anaconda的lib含有,而且在下面一句已经加上,且加在了自安装的HDF5的前面,所以实际上使用的仍然是anaconda的 版本,如果不行就加上呗。)
#export LD_LIBRARY_PATH=/data1/caiyong.wang/bin/hdf5-1.10.1/hdf5/lib:$LD_LIBRARY_PATHexport LD_LIBRARY_PATH=/data1/caiyong.wang/bin/anaconda2/lib:$LD_LIBRARY_PATHexport PATH=/usr/local/cuda-8.0/bin:$PATH######## add protobuf lib path ########
##(动态库搜索路径) 程序加载运行期间查找动态链接库时指定除了系统默认路径之外的其他路径
#export LD_LIBRARY_PATH=/data1/caiyong.wang/bin/protobuf2/lib/:$LD_LIBRARY_PATH
##(静态库搜索路径) 程序编译期间查找动态链接库时指定查找共享库的路径
#export LIBRARY_PATH=/data1/caiyong.wang/bin/protobuf2/lib/:$LIBRARY_PATH
##执行程序搜索路径
#export PATH=/data1/caiyong.wang/bin/protobuf2/bin/:$PATH
##c程序头文件搜索路径
#export C_INCLUDE_PATH=/data1/caiyong.wang/bin/protobuf2/include/:$C_INCLUDE_PATH
##c++程序头文件搜索路径
#export CPLUS_INCLUDE_PATH=/data1/caiyong.wang/bin/protobuf2/include/:$CPLUS_INCLUDE_PATH
##pkg-config 路径
#export PKG_CONFIG_PATH=/data1/caiyong.wang/bin/protobuf2/lib/pkgconfig/
#######################################
#alias protoc='/data1/caiyong.wang/bin/protobuf2/bin/protoc'
export PATH=/data1/caiyong.wang/bin/protobuf/bin:$PATH
export PATH=/usr/local/bin:$PATH
export LD_LIBRARY_PATH=/data1/caiyong.wang/bin/protobuf/lib:$LD_LIBRARY_PATH# added by Anaconda2 installer
export PATH=/data1/caiyong.wang/bin/anaconda2/bin:$PATH
一般情况下,这样编译就可以通过。这里make runtest花费时间很多,我用了近9个小时。如果不想花费这么长时间,也可以不运行这个指令。
最后为了在python中可以载入caffe模块,我们需要在bashrc设置PYTHONPATH。
export PYTHONPATH=/*/py-faster-rcnn/caffe-fast-rcnn/python:$PYTHONPATH
然后source ~/.bashrc生效。
最终查看python接口是否编译成功:
进入python环境,进行import操作
# python
>>> import caffe
>>> print caffe.__version__
1.0.0
如果没有提示错误,则编译成功。
测试demo
- 下载预计算的R-CNN检测器
cd $FRCN_ROOT
./data/scripts/fetch_faster_rcnn_models.sh
这个模型解压出来750M,下载的话大概695M,而且很慢。。
为了方便大家,我把模型上传到了百度云 链接: https://pan.baidu.com/s/1eSKoVAi 密码: 59ga
注意下载后,解压到data目录下。
2. 运行
cd $FRCN_ROOT
./tools/demo.py
出现结果: (共5个图片来源)
[外链图片转存失败(img-Dv5lyU7I-1567741174724)(https://img-blog.csdn.net/20171019152755856?watermark/2/text/aHR0cDovL2Jsb2cuY3Nkbi5uZXQveGlhbWVudGluZ3Rhbw==/font/5a6L5L2T/fontsize/400/fill/I0JBQkFCMA==/dissolve/70/gravity/SouthEast)]
成功~~~
最后说明
如果安装过程中还有问题,可以参考:http://blog.csdn.net/xiamentingtao/article/details/78266153
参考文献
- rbgirshick/py-faster-rcnn
- cuda8+cudnn4 Faster R-CNN安装塈运行demo
- 使用cuDNN5编译py-faster-rcnn错误:cudnn.hpp(126): error: argument of type “int” is incompatible …
- Caffe-faster-rcnn demo测试
- caffe官网-install
- Caffe学习系列(13):数据可视化环境(python接口)配置