容器化部署fastdfs文件存储

一、软件信息

二、构建fastdfs镜像

三、docker 启动fdfs服务

四、k8s部署fdfs服务

1、fdfs部署文件

五、外部服务访问

一、软件信息

fastdfs版本：fastdfs:V5.11
libfastcommon版本: V1.0.36
fastdfs-nginx-module版本：V1.20
nginx版本：1.18.0

二、构建fastdfs镜像

为了减少操作了文件数，我是直接把启动文件和dockerfile写一起了，也可以把启动脚本 start.sh 单独于dockerfile；然后在构建镜像的时候把启动文件 COPY 到镜像里。

以下是包括了启动文件为一起的dockerfile构建文件。直接使用该dockerfile即可完成镜像构建。然后按照自己的实际情况来修改配置文件和挂载，即可完成容器化运行fastdfs件存储服务。

FROM centos:7LABEL creator=tudou date="2024-06-02"ENV FASTDFS_PATH=/var/fastdfs \TZ=Asia/Shanghai \TRACKER_PORT= \STORAGE_PORT= \TRACKER_SERVER= \GROUP_NAME= #get all the dependences
RUN yum install -y git wget zip unzip gcc make \perl openssl openssl-devel pcre pcre-devel zlib zlib-devel libevent libevent-devel \&& yum clean all \&& mkdir -p ${FASTDFS_PATH}/filesRUN git clone --branch V1.0.36 --depth 1 https://github.com/happyfish100/libfastcommon.git ${FASTDFS_PATH}/libfastcommon \&& cd ${FASTDFS_PATH}/libfastcommon \&& ./make.sh \&& ./make.sh install \&& rm -rf ${FASTDFS_PATH}/libfastcommonRUN git clone --branch V5.11 --depth 1 https://github.com/happyfish100/fastdfs.git ${FASTDFS_PATH}/fastdfs \&& cd ${FASTDFS_PATH}/fastdfs  \&& ./make.sh \&& ./make.sh install \&& cp -r  conf/*.* /etc/fdfs/ \&& rm -rf ${FASTDFS_PATH}/fastdfs RUN git clone --branch V1.20 https://github.com/happyfish100/fastdfs-nginx-module.git /usr/local/fastdfs-nginx-module \&& cp /usr/lib64/libfastcommon.so /usr/local/lib/libfastcommon.so \&& sed -i "s@/usr/local/include@/usr/include/fastdfs /usr/include/fastcommon@g" /usr/local/fastdfs-nginx-module/src/config \&& cd ${FASTDFS_PATH} \&& wget https://nginx.org/download/nginx-1.18.0.tar.gz \&& tar -xzvf nginx-1.18.0.tar.gz \&& cd nginx-1.18.0 \&& ./configure --prefix=/usr/local/nginx --add-module=/usr/local/fastdfs-nginx-module/src \&& make \&& make install \&& cp /usr/local/fastdfs-nginx-module/src/mod_fastdfs.conf  /etc/fdfs/ \&& rm -rf ${FASTDFS_PATH}/nginx-1.18.0* WORKDIR ${FASTDFS_PATH}# 启动脚本 start.sh
RUN cat > /usr/bin/start.sh <<-EOF
#!/bin/bashcd \$(dirname \$0)# start.sh trackerd 
# start.sh storaged# # 自定义环境变量初始化，若无自定义环境变量，则直接跳过使用配置文件的默认值
function init_config(){if [[ -n \${STORAGE_PORT} ]];thensed -i "s|^port=.*$|port=\${STORAGE_PORT}|g" /etc/fdfs/storage.conffiif [[ -n \${TRACKER_PORT} ]];thensed -i "s|^port=.*$|port=\${TRACKER_PORT}|g" /etc/fdfs/tracker.conffiif [[ -n \${GROUP_NAME} ]];thensed -i "s|group_name=.*$|group_name=\${GROUP_NAME}|g" /etc/fdfs/storage.confsed -i "s|group_name=.*$|group_name=\${GROUP_NAME}|g" /etc/fdfs/mod_fastdfs.confsed -i "s|url_have_group_name =.*$|url_have_group_name = true|g" /etc/fdfs/mod_fastdfs.conffiif [[ -n \${TRACKER_SERVER} ]];thensed -i "s|tracker_server=.*$|tracker_server=\${TRACKER_SERVER}|g" /etc/fdfs/storage.confsed -i "s|tracker_server=.*$|tracker_server=\${TRACKER_SERVER}|g" /etc/fdfs/client.confsed -i "s|tracker_server=.*$|tracker_server=\${TRACKER_SERVER}|g" /etc/fdfs/mod_fastdfs.conffised -i "s|base_path=.*$|base_path=\${FASTDFS_PATH}|g" /etc/fdfs/tracker.confsed -i "s|base_path=.*$|base_path=\${FASTDFS_PATH}|g" /etc/fdfs/storage.confsed -i "s|base_path=.*$|base_path=\${FASTDFS_PATH}|g" /etc/fdfs/client.confsed -i "s|base_path=.*$|base_path=\${FASTDFS_PATH}|g" /etc/fdfs/mod_fastdfs.conf
}init_configcase \$1 intrackerd)echo "start trackerd ..."fdfs_trackerd /etc/fdfs/tracker.conf start  && tail -f ${FASTDFS_PATH}/logs/trackerd.log;;storaged)sed -i "s|store_path0.*$|store_path0=\${FASTDFS_PATH}/files|g" /etc/fdfs/storage.confsed -i "s|store_path0.*$|store_path0=\${FASTDFS_PATH}/files|g" /etc/fdfs/mod_fastdfs.confecho "start storaged ..."fdfs_storaged /etc/fdfs/storage.conf start \&& /usr/local/nginx/sbin/nginx -c /usr/local/nginx/conf/nginx.conf \&& tail -f ${FASTDFS_PATH}/logs/storaged.log;;*)echo "USAGE sh $0 [ trackerd|storaged ]"exit 1;;
esacEOF
# 启动脚本 start.sh# 默认fastdfs端口
EXPOSE 22122 23000 8080 8888 80RUN chmod a+x /usr/bin/start.sh ENTRYPOINT ["/usr/bin/start.sh"]
CMD ["trackerd"]
#CMD ["storaged"]

# 构建 fdfs 应用镜像# docker build -t fdfs:v1.0 -f dockerfile . --no-cachedocker build -t fdfs:v1.0 -f dockerfile .

三、docker 启动fdfs服务

# 容器启动 trackerd 服务，并且自定义环境变量值,可不指定环境变量值docker run -itd --name trackerd -e TRACKER_PORT=22122 -e TRACKER_SERVER=192.168.159.132:22122 -p 22122:22122   fdfs:v1.0 trackerd# 容器启动 storaged 服务，并且自定义环境变量值,可不指定环境变量值docker run -itd --name storage -e STORAGE_PORT=23000 -e TRACKER_SERVER=192.168.159.132:22122 -p 23000:23000   fdfs:v1.0 storaged

# 分别查看容器应用的日志docker logs trackerddocker logs storage

四、k8s部署fdfs服务

1、fdfs部署文件

由于storage存储是各自独立的，所以要使用有状态部署。tracker是透明无状态的，可以使用无状态负载deploy部署，也可以使用有状态负载StatefulSet部署。

# kubectl apply -f fdfs.yaml -n fdfs# fdfs 配置文件
---
kind: ConfigMap
apiVersion: v1
metadata:name: fdfs-confignamespace: fdfsannotations:kubesphere.io/creator: admin
data:mod_fastdfs.conf: |-base_path=/var/fastdfstracker_server=tracker-svc:22122storage_server_port=23000group_name=group1url_have_group_name = truestore_path_count=1store_path0=/var/fastdfs/filesgroup_count = 1# 在末尾增加组的具体信息 [group1] group_name=group1 storage_server_port=23000 store_path_count=1 store_path0=/var/fastdfs/files[group2] group_name=group2 storage_server_port=23000 store_path_count=1 store_path0=/var/fastdfs/filesnginx.conf: |-user root;worker_processes  2;events {worker_connections  65535;use epoll;}http {include       mime.types;default_type  application/octet-stream;sendfile        on;keepalive_timeout  65;server {listen       80;server_name  localhost;charset utf-8;server_tokens off;location ~group[0-9]/M00 {root /var/fastdfs/files/data;ngx_fastdfs_module;}error_page   500 502 503 504  /50x.html;location = /50x.html {root   html;}}}storage.conf: |-group_name=group1port=23000base_path=/var/fastdfsstore_path_count=1store_path0=/var/fastdfs/filestracker_server=tracker-svc:22122http.server_port=80tracker.conf: |-disabled=falseport=22122base_path=/var/fastdfshttp.server_port=80---# kubectl apply -f fdfs-trackerd.yaml -n fdfs
---
apiVersion: apps/v1
kind: Deployment
metadata:labels:app: fdfs-trackerdname: fdfs-trackerdnamespace: fdfs
spec:replicas: 1selector:matchLabels:app: fdfs-trackerdstrategy:rollingUpdate:maxSurge: 25%maxUnavailable: 25%type: RollingUpdatetemplate:metadata:labels:app: fdfs-trackerdspec:containers:- args:- trackerdcommand:- /usr/bin/start.shimage: swr.cn-south-1.myhuaweicloud.com/tudou/fastdfs:latestimagePullPolicy: IfNotPresentname: trackerdports:- containerPort: 22122name: http-0protocol: TCP- containerPort: 8080name: http-1protocol: TCPresources: {}terminationMessagePath: /dev/termination-logterminationMessagePolicy: FilevolumeMounts:- mountPath: /etc/fdfs/tracker.confname: volume-b8e30ireadOnly: truesubPath: tracker.confdnsPolicy: ClusterFirstrestartPolicy: AlwaysschedulerName: default-schedulersecurityContext: {}serviceAccount: defaultserviceAccountName: defaultterminationGracePeriodSeconds: 30volumes:- configMap:defaultMode: 420name: fdfs-configname: volume-b8e30i
---
---
apiVersion: v1
kind: Service
metadata:labels:app: fdfs-trackerdname: fdfs-trackerdnamespace: fdfs
spec:ports:- name: port-1port: 22122protocol: TCPtargetPort: 22122- name: port-2port: 8080protocol: TCPtargetPort: 8080selector:app: fdfs-trackerdtype: ClusterIP
---# kubectl apply -f fdfs-storaged.yaml -n fdfs
---
apiVersion: apps/v1
kind: StatefulSet
metadata:labels:app: fdfs-storagedname: fdfs-storagednamespace: fdfs
spec:podManagementPolicy: OrderedReadyreplicas: 2revisionHistoryLimit: 10selector:matchLabels:app: fdfs-storagedserviceName: fdfs-storagedtemplate:metadata:labels:app: fdfs-storagedspec:containers:- args:- storagedcommand:- /usr/bin/start.shimage: swr.cn-south-1.myhuaweicloud.com/tudou/fastdfs:latestimagePullPolicy: IfNotPresentname: storageports:- containerPort: 23000name: http-0protocol: TCP- containerPort: 8888name: http-1protocol: TCP- containerPort: 80name: http-2protocol: TCPresources: {}terminationMessagePath: /dev/termination-logterminationMessagePolicy: FilevolumeMounts:- mountPath: /etc/fdfs/storage.confname: volume-ssmj62readOnly: truesubPath: storage.conf- mountPath: /etc/fdfs/mod_fastdfs.confname: volume-zlknkzreadOnly: truesubPath: mod_fastdfs.conf- mountPath: /usr/local/nginx/conf/nginx.confname: volume-q4ylwbreadOnly: truesubPath: nginx.conf- mountPath: /var/fastdfsname: fdfs-storage-datadnsPolicy: ClusterFirstrestartPolicy: AlwaysschedulerName: default-schedulersecurityContext: {}serviceAccount: defaultserviceAccountName: defaultterminationGracePeriodSeconds: 30volumes:- configMap:defaultMode: 420name: fdfs-configname: volume-ssmj62- configMap:defaultMode: 420name: fdfs-configname: volume-zlknkz- configMap:defaultMode: 420name: fdfs-configname: volume-q4ylwbupdateStrategy:rollingUpdate:partition: 0type: RollingUpdatevolumeClaimTemplates:- apiVersion: v1kind: PersistentVolumeClaimmetadata:name: fdfs-storage-dataspec:accessModes:- ReadWriteOnceresources:requests:storage: 120GistorageClassName: localvolumeMode: Filesystem
---
---
apiVersion: v1
kind: Service
metadata:labels:app: fdfs-storagedname: fdfs-storagednamespace: fdfs
spec:clusterIP: NoneclusterIPs:- NoneinternalTrafficPolicy: ClusteripFamilies:- IPv4ipFamilyPolicy: SingleStackports:- name: http-0port: 23000protocol: TCPtargetPort: 23000- name: http-1port: 8888protocol: TCPtargetPort: 8888- name: http-2port: 80protocol: TCPtargetPort: 80selector:app: fdfs-storagedsessionAffinity: Nonetype: ClusterIP
---