文章目录
- 一、docker-compose方式
- 二、Dockerfile方式
- 三、docker run方式
- 四、查看检查日志
一、docker-compose方式
在docker-compose中加入healthcheck
healthcheck 支持下列选项:
test:健康检查命令,例如 ["CMD", "curl", "-f", "http://localhost/actuator/health"]
interval:健康检查的间隔,默认为 30 秒,单位(h/m/s);
timeout:健康检查命令运行超时时间,如果超过这个时间,本次健康检查就被视为失败,默认 30 秒,单位(h/m/s);
retries:当连续失败指定次数后,则将容器状态视为 unhealthy,默认 3 次。
start-period:应用的启动的初始化时间,在启动过程中的健康检查失效不会计入,默认 0 秒; (从17.05)引入
说明:在此期间的探测失败将不计入最大重试次数。但是,如果健康检查在启动期间成功,则认为容器已启动,所有连续的失败都将计入最大重试次数。和 CMD, ENTRYPOINT 一样,HEALTHCHECK 只可以出现一次,如果写了多个,只有最后一个生效。
在 HEALTHCHECK [选项] CMD
后面的命令,格式和 ENTRYPOINT
一样,分为 shell
格式,和 exec
格式。命令的返回值决定了该次健康检查的成功与否:
0
:成功;1
:失败;2
:保留值,不要使用
容器启动之后,初始状态会为 starting
(启动中)。Docker Engine会等待 interval
时间,开始执行健康检查命令,并周期性执行。如果单次检查返回值非0或者运行需要比指定 timeout
时间还长,则本次检查被认为失败。如果健康检查连续失败超过了 retries
重试次数,状态就会变为 unhealthy
(不健康)。
注:
- 一旦有一次健康检查成功,Docker会将容器置回
healthy
(健康)状态 - 当容器的健康状态发生变化时,Docker Engine会发出一个
health_status
事件。
healthcheck:test: ["CMD","curl","-f","http://localhost:28025/v1/health/check"]# 每次间隔30秒检查一次interval: 30s# 每次检查的超时时间timeout: 10s# 最多重试检查次数retries: 3# redis检查示例 healthcheck:test: ["CMD", "redis-cli", "ping"]interval: 1stimeout: 3sretries: 30# postgresql检查 healthcheck:test: ["CMD", "pg_isready"]interval: 1stimeout: 3sretries: 30# mysql检查 healthcheck:test: ["CMD", "mysql" ,"-h", "mysql", "-P", "3306", "-u", "root", "-e", "SELECT 1", "cache"]interval: 1stimeout: 3sretries: 30
version: '3'
services:mysql:hostname: mysqlimage: 10.10.3.5/cta/mysql_base:01container_name: mysqlvolumes:- ./mysql/mysql.cnf:/etc/mysql/conf.d/mysql.cnf- ./mysql/mysqld.cnf:/etc/mysql/mysql.conf.d/mysqld.cnf- ./mysql/init:/docker-entrypoint-initdb.d/#- ./mysql/data:/var/lib/mysql# 给容器赋予root权限 privileged: truenetwork_mode: "host"environment:- MYSQL_ROOT_PASSWORD=123- TZ=Asia/Shanghai- LANG=en_US.UTF-8# 健康检查 healthcheck:# mysqladmin -uroot -p123 ping -h127.0.0.1test: [ "CMD", "mysqladmin" ,"ping", "-h", "localhost" ]
# test: [ "CMD", "mysqladmin","-u","root","-p","123" ,"ping", "-h", "127.0.0.1" ]timeout: 45sinterval: 10sretries: 10nginx:hostname: nginximage: nginx:1.20.2volumes:- ./nginx/nginx.conf:/etc/nginx/nginx.conf- ./nginx/platform.conf:/etc/nginx/conf.d/platform.conf- ./nginx/svrshell:/root/svrshell- ./nginx/aiviewer:/root/aiviewer
# 增加cta- ./nginx/dwcta:/root/dwcta
# 增加input挂载- /data1/inputdata:/data1/inputdatacontainer_name: nginxprivileged: truenetwork_mode: "host"environment:- TZ=Asia/Shanghai- LANG=en_US.UTF-8zk:
# zookeeper 修改了zoo.cfg的data和datalog路径 /opt/zk1/zookeeper-3.4.10hostname: zkimage: 10.10.3.5/cta/zookeeper:3.4.10healthcheck:
# test: ["CMD-SHELL", "curl -sS 127.0.0.1:2181 || exit 1"]test: ["CMD-SHELL", "echo 'ruok' | curl -s telnet://localhost:2181 || exit 1"]interval: 1m30stimeout: 10sretries: 3start_period: 40sdepends_on:mysql:# mysql健康检查通过后,才能启动zk服务condition: service_healthy# condition: service_startedvolumes:- ./zk/zoo.cfg:/conf/zoo.cfg
# - /opt/zk1/zookeeper-3.4.10/data:/data
# - /opt/zk1/zookeeper-3.4.10/datalog:/datalogcontainer_name: zkprivileged: truenetwork_mode: "host"# ports:# - 2181:2181es_python:hostname: es_pythonimage: es_python:1.1volumes:- /data1/es_data:/usr/share/elasticsearch/datacontainer_name: es_pythonprivileged: truenetwork_mode: "host"environment:- TZ=Asia/Shanghai- LANG=en_US.UTF-8# 应用aiserverconfig:
# 20882 11383hostname: aiserverconfigimage: 10.10.3.5/cta/aiserverconfig-cta_1215_20220112_002:latesthealthcheck:test: ["CMD-SHELL", "curl -sS 127.0.0.1:11383 || exit 1"]interval: 1m30stimeout: 10sretries: 3start_period: 40sdepends_on:mysql:condition: service_healthyzk:condition: service_healthyvolumes:- /etc/sign.conf:/etc/sign.conf- /data1/container-root/aiserverconfig/log:/root/aiserverconfig/log
# 增加input挂载- /data1/inputdata:/data1/inputdatacontainer_name: aiserverconfigprivileged: truenetwork_mode: "host"aiserver:
# 20880 11381hostname: aiserverimage: 10.10.3.5/cta/aiserver-cta_1215_20220112_002:latesthealthcheck:test: ["CMD-SHELL", "curl -sS 127.0.0.1:11381/aiserver/ai/v1/echo || exit 1"]interval: 1m30stimeout: 10sretries: 3start_period: 40sdepends_on:mysql:condition: service_healthyzk:condition: service_healthyaiserverconfig:condition: service_healthyvolumes:- /etc/sign.conf:/etc/sign.conf- /data1/container-root/aiserver/log:/root/aiserver/log
# 增加input挂载- /data1/inputdata:/data1/inputdatacontainer_name: aiserverprivileged: truenetwork_mode: "host"pacsserver:
# 20881 22222 11380hostname: pacsserverimage: 10.10.3.5/cta/pacsserver-cta_1215_20220112_002:latesthealthcheck:test: ["CMD-SHELL", "curl -sS 127.0.0.1:11384/pacs/v2/dicom/echo || exit 1"]interval: 1m30stimeout: 10sretries: 3start_period: 40sdepends_on:mysql:condition: service_healthyzk:condition: service_healthyaiserverconfig:condition: service_healthyaiserver:condition: service_healthyvolumes:- /etc/sign.conf:/etc/sign.conf- /data1/container-root/pacsserver/log:/root/pacsserver/log
# 增加input挂载- /data1/inputdata:/data1/inputdatacontainer_name: pacsserverprivileged: truenetwork_mode: "host"platform:
# 11380 11345hostname: platformimage: 10.10.3.5/cta/platform-cta_1215_20220112_002:latesthealthcheck:test: ["CMD-SHELL", "curl -sS 127.0.0.1:11380/echo || exit 1"]interval: 1m30stimeout: 10sretries: 3start_period: 40sdepends_on:mysql:condition: service_healthyzk:condition: service_healthyaiserverconfig:condition: service_healthyaiserver:condition: service_healthyvolumes:- /etc/sign.conf:/etc/sign.conf- /data1/container-root/platform/log:/root/platform/log
# 增加input挂载- /data1/inputdata:/data1/inputdatacontainer_name: platformprivileged: truenetwork_mode: "host"ocrsvr:
# 28800hostname: ocrsvrimage: 10.10.3.5/cta/ocrsvr-cta_1215_20220112_002:latestdepends_on:mysql:condition: service_healthyzk:condition: service_healthyaiserverconfig:condition: service_healthyaiserver:condition: service_healthyvolumes:- /etc/sign.conf:/etc/sign.conf- /data1/container-root/ocrsvr/log:/root/ocrsvr/log
# 增加input挂载- /data1/inputdata:/data1/inputdatacontainer_name: ocrsvrprivileged: truenetwork_mode: "host"lung:
# 20880 11381hostname: lungimage: 10.10.3.5/cta/lung-cta_1215_20220112_002:latestdepends_on:mysql:condition: service_healthyzk:condition: service_healthyaiserverconfig:condition: service_healthyaiserver:condition: service_healthyvolumes:- /etc/sign.conf:/etc/sign.conf- /data1/container-root/lung/log:/root/lung/log
# 增加input挂载- /data1/inputdata:/data1/inputdata- /tmp/.X11-unix:/tmp/.X11-unixcontainer_name: lungprivileged: truenetwork_mode: "host"environment:- NVIDIA_DRIVER_CAPABILITIES=all- DISPLAY- XAUTHORITYdeploy:resources:reservations:devices:- driver: nvidiacount: allcapabilities: [gpu]
二、Dockerfile方式
假设我们有个镜像是个最简单的 Web 服务,我们希望增加健康检查来判断其 Web 服务是否在正常工作,我们可以用 curl
来帮助判断,其 Dockerfile
的 HEALTHCHECK
可以这么写:
Dockerfile
FROM elasticsearch:5.5HEALTHCHECK --interval=5s --timeout=2s --retries=12 \CMD curl --silent --fail localhost:9200/_cluster/health || exit 1
docker build -t test/elasticsearch:5.5 .docker run --rm -d \--name=elasticsearch \test/elasticsearch:5.5
我们可以通过 docker ps,来发现过了几秒之后,Elasticsearch容器从 starting
状态进入了 healthy
状态
#!/bin/bash
# 检测tomcat端口 细心的小伙伴可以在awk把最后一个:后面的端口取出来 会更准确
netstat -luntp |awk '{print $4}' |grep 8080
if [ $? == 0 ]
thenecho $?exit 0
elseecho $?exit 1
fi
FROM registry.cn-shanghai.aliyuncs.com/yjk-datag/tomcat:v1
MAINTAINER YJK Enterprise Container Images <min@hz-health.cn>
ENV LANG C.UTF-8
ENV PATH /usr/bin:$PATH
RUN mv /srv/tomcat/tomcat8/ /srv/tomcat/tomcat-haozhuo-video
ADD ROOT.war /srv/tomcat/tomcat-haozhuo-video/webapps/
ADD health_check.sh /opt/
EXPOSE 8080 20920
HEALTHCHECK --start-period=60s --interval=60s --timeout=5s --retries=3 CMD /bin/bash /opt/health_check.sh
# 依赖基础镜像centos7版jdk1.8FROM openjdk:8
# 标签
LABEL maintainer="2022-12-23"
# 传输文件,将search里面的内容考入/opt/search/中
#COPY search /usr/CommandCenter/gis/search
#添加search.tar.gz压缩文件到/opt目录下,并同时完成解压,完成后的目录为opt/search/xxxx
ADD fps.tar.gz /usr/CommandCenter/vcs
# 给启动脚本授可执行权限
RUN chmod +x /usr/fps/bin/docker_startup.sh
RUN chmod +x /usr/fps/bin/get-nacos-status.sh
# 安装curl
RUN apk update && apk add curl && apk add busybox-extras
# 工作目录
WORKDIR /usr/CommandCenter
# 启动startup.sh
CMD ["/bin/sh","-c","/usr/bin/docker_startup.sh"]
#CMD ["/bin/sh","-c","ps&&tail -f /dev/null"]
三、docker run方式
另外一种方法是在 docker run
命令中,直接指明healthcheck相关策略
$ docker run --rm -d \--name=elasticsearch \--health-cmd="curl --silent --fail localhost:9200/_cluster/health || exit 1" \--health-interval=5s \--health-retries=12 \--health-timeout=2s \elasticsearch:5.5
四、查看检查日志
为了帮助排障,健康检查命令的输出(包括 stdout 以及 stderr)都会被存储于健康状态里,可以用 docker inspect 来查看。我们可以通过如下命令,来获取过去5个容器的健康检查结果
docker inspect --format='{{json .State.Health}}' 容器名docker inspect --format='{{json .State.Health}}' commandcenter.public.configcenter
或者
docker inspect 容器名 | jq ".[].State.Health"
日志示例
{"Status": "healthy","FailingStreak": 0,"Log": [{"Start": "2017-08-19T09:12:53.393598805Z","End": "2017-08-19T09:12:53.452931792Z","ExitCode": 0,"Output": "..."},...
}
由于应用的开发者会更加了解应用的SLA,一般建议在Dockerfile中声明相应的健康检查策略,这样可以方便镜像的使用。对于应用的部署和运维人员,可以通过命令行参数和REST API针对部署场景对健康检查策略按需进行调整。