效果图

环境准备

安装 influxdb

安装我们刚刚下载 influxdb rpm文件

查看默认配置

修改参数

启动 influxdb

查看启动状态

设置基本配置

influxdb 其他命令扩展

安装 jmxtrans

可能遇到的异常

验证jmxtrans是否成功运行

安装 Grafana

安装

influxDB 与 Grafana 监控模板

效果图

先看下

环境准备

JDK：1.8

jmxtrans 安装包

# 我们系统是CentOS，这里选择rpm
https://github.com/downloads/jmxtrans/jmxtrans/jmxtrans-20121016.145842.6a28c97fbb-0.noarch.rpm
# 其他系统在下面链接中可以找到
https://github.com/jmxtrans/jmxtrans/downloads# 下载源码，后面编译使用
https://github.com/jmxtrans/jmxtrans/releases
# 我这里使用的271版本
https://github.com/jmxtrans/jmxtrans/archive/jmxtrans-parent-271.tar.gz

influxdb 安装包

# 下载安装包
https://dl.influxdata.com/influxdb/releases/influxdb-1.8.0.x86_64.rpm
# 官网下载页面
https://portal.influxdata.com/downloads/

Grafana 安装包

https://dl.grafana.com/oss/release/grafana-6.7.3-1.x86_64.rpm

安装 influxdb

安装我们刚刚下载 influxdb rpm文件

rpm -ivh influxdb-1.8.0.x86_64.rpm

查看默认配置

> influxd configMerging with configuration at: /etc/influxdb/influxdb.conf
reporting-disabled = false
bind-address = "127.0.0.1:8088"[meta]dir = "/var/lib/influxdb/meta"retention-autocreate = truelogging-enabled = true[data]dir = "/var/lib/influxdb/data"index-version = "inmem"wal-dir = "/var/lib/influxdb/wal"wal-fsync-delay = "0s"validate-keys = falsequery-log-enabled = truecache-max-memory-size = 1073741824cache-snapshot-memory-size = 26214400cache-snapshot-write-cold-duration = "10m0s"compact-full-write-cold-duration = "4h0m0s"compact-throughput = 50331648compact-throughput-burst = 50331648max-series-per-database = 1000000max-values-per-tag = 100000max-concurrent-compactions = 0max-index-log-file-size = 1048576series-id-set-cache-size = 100series-file-max-concurrent-snapshot-compactions = 0trace-logging-enabled = falsetsm-use-madv-willneed = false[coordinator]write-timeout = "10s"max-concurrent-queries = 0query-timeout = "0s"log-queries-after = "0s"max-select-point = 0max-select-series = 0max-select-buckets = 0[retention]enabled = truecheck-interval = "30m0s"[shard-precreation]enabled = truecheck-interval = "10m0s"advance-period = "30m0s"[monitor]store-enabled = truestore-database = "_internal"store-interval = "10s"[subscriber]enabled = truehttp-timeout = "30s"insecure-skip-verify = falseca-certs = ""write-concurrency = 40write-buffer-size = 1000[http]enabled = truebind-address = ":8086"auth-enabled = falselog-enabled = truesuppress-write-log = falsewrite-tracing = falseflux-enabled = falseflux-log-enabled = falsepprof-enabled = truepprof-auth-enabled = falsedebug-pprof-enabled = falseping-auth-enabled = falsehttps-enabled = falsehttps-certificate = "/etc/ssl/influxdb.pem"https-private-key = ""max-row-limit = 0max-connection-limit = 0shared-secret = ""realm = "InfluxDB"unix-socket-enabled = falseunix-socket-permissions = "0777"bind-socket = "/var/run/influxdb.sock"max-body-size = 25000000access-log-path = ""max-concurrent-write-limit = 0max-enqueued-write-limit = 0enqueued-write-timeout = 30000000000[logging]format = "auto"level = "info"suppress-logo = false[[graphite]]enabled = falsebind-address = ":2003"database = "graphite"retention-policy = ""protocol = "tcp"batch-size = 5000batch-pending = 10batch-timeout = "1s"consistency-level = "one"separator = "."udp-read-buffer = 0[[collectd]]enabled = falsebind-address = ":25826"database = "collectd"retention-policy = ""batch-size = 5000batch-pending = 10batch-timeout = "10s"read-buffer = 0typesdb = "/usr/share/collectd/types.db"security-level = "none"auth-file = "/etc/collectd/auth_file"parse-multivalue-plugin = "split"[[opentsdb]]enabled = falsebind-address = ":4242"database = "opentsdb"retention-policy = ""consistency-level = "one"tls-enabled = falsecertificate = "/etc/ssl/influxdb.pem"batch-size = 1000batch-pending = 5batch-timeout = "1s"log-point-errors = true[[udp]]enabled = falsebind-address = ":8089"database = "udp"retention-policy = ""batch-size = 5000batch-pending = 10read-buffer = 0batch-timeout = "1s"precision = ""[continuous_queries]log-enabled = trueenabled = truequery-stats-enabled = falserun-interval = "1s"[tls]min-version = ""max-version = ""

修改参数

默认influxDB使用以下端口

8086: 用于客户端和服务端交互的HTTP API
8088: 用于提供备份和恢复的RPC服务

我这里修改配置文件,使用8087端口，我这里8088和其他服务冲突了

同时修改了数据保存的路径

> vim /etc/influxdb/influxdb.confbind-address = "127.0.0.1:8087"
# metadata 保存路径
dir = "/root/jast/influxdb/meta"
#数据保存路径
dir = "/root/jast/influxdb/data"
#`write-ahead-log(WAL)保存路径
wal-dir = "/root/jast/influxdb/wal"

注意：选择的路径要有权限，否则会启动失败

启动 influxdb

systemctl start influxdb

查看启动状态

systemctl status influxdb

此时influxdb已经启动成功

我们也可以指定配置文件启动，在/etc/influxdb/influxdb.conf ,是默认的目录也可以不指定

influxd -config /etc/influxdb/influxdb.conf

设置基本配置

在服务器输入 influx 进入交互页面

[root@ecs-t-001-0001 influx]# influx
Connected to http://localhost:8086 version 1.8.0
InfluxDB shell version: 1.8.0
>

创建用户

CREATE USER "admin" WITH PASSWORD '123456' WITH ALL PRIVILEGES

创建数据库（后面保存数据用）

create database "jmxDB"

查看是否创建成功

[root@ecs-t-001-0001 jmxtrans]# influx
Connected to http://localhost:8086 version 1.8.0
InfluxDB shell version: 1.8.0
> show databases
name: databases
name
----
_internal
jmxDB
>

influxdb 其他命令扩展

#创建数据库
create database "db_name"
#显示所有的数据库
show databases
#删除数据库
drop database "db_name"
#使用数据库
use db_name
#显示该数据库中所有的表
show measurements
#创建表，直接在插入数据的时候指定表名
insert test,host=127.0.0.1,monitor_name=test count=1
#删除表
drop measurement "measurement_name"
#退出
quit

安装 jmxtrans

安装我们刚刚下载jmxtrans rpm文件

rpm -ivh jmxtrans-20121016.145842.6a28c97fbb-0.noarch.rpm

安装完成后默认安装目录在

[root@ecs-t-001-0001 jmxtrans]# whereis jmxtrans
jmxtrans: /usr/share/jmxtrans

这里我们先简单配置Kafka 的Memory 监控，其他配置在文末统一整理

我们创建json文件供jmxtrans读取，json文件名称自己根据业务取名即可

{"servers" : [ {"port" : "9393","host" : "172.11.0.1","queries" : [ {"obj" : "java.lang:type=Memory","attr" : [ "HeapMemoryUsage", "NonHeapMemoryUsage" ],"resultAlias":"jvmMemory","outputWriters" : [ {"@class" : "com.googlecode.jmxtrans.model.output.InfluxDbWriterFactory","url" : "http://172.11.0.1:8086/","username" : "admin","password" : "123456","database" : "jmxDB","tags"     : {"application" : "kafka_server"}} ]} ]} ]
}

简单解释一下上面的说明

port: 我们要监控的Kafka JMX端口
host：我们要监控的Kafka host
resultAlias：自定义表名，收集到的数据会存入influxdb的定义的表中，自动创建
outputWriters为连接influxdb的配置
@class不需要修改
url：influxdb的机器+端口，默认端口8086
username和password：influxdb的用户和密码
database：influxdb数据库（我们刚刚创建的）

启动之前我们把 /usr/share/jmxtrans 目录下的所有 .json 文件换个名，因为它会默认会读取 /usr/share/jmxtrans 目录下的所有json文件

在 /usr/share/jmxtrans 目录下启动 jmxtrans.sh

jmxtrans.sh start

到这里正常来说就是要启动成功了，我们先说下可能遇到的异常

可能遇到的异常

异常1

[root@ecs-t-001-0001 jmxtrans]# Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=384m; support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=384m; support was removed in 8.0
MaxTenuringThreshold of 16 is invalid; must be between 0 and 15
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.

MaxTenuringThreshold 这个参数用于控制对象能经历多少次Minor GC才晋升到旧生代

提示设置的是16，但是范围在0-15，我们直接修改一下启动脚本 jmxtrans.sh

> vim jmxtrans.shGC_OPTS=${GC_OPTS:-"-Xms${HEAP_SIZE}M -Xmx${HEAP_SIZE}M -XX:+UseConcMarkSweepGC -XX:NewRatio=${NEW_RATIO} -XX:NewSize=${NEW_SIZE}m -XX:MaxNewSize=${NEW_SIZE}m -XX:MaxTenuringThreshold=15 -XX:GCTimeRatio=9 -XX:PermSize=${PERM_SIZE}m -XX:MaxPermSize=${MAX_PERM_SIZE}m -XX:+UseTLAB -XX:CMSInitiatingOccupancyFraction=${IO_FRACTION} -XX:+CMSIncrementalMode -XX:+CMSIncrementalPacing -XX:ParallelGCThreads=${CPU_CORES} -Dsun.rmi.dgc.server.gcInterval=28800000 -Dsun.rmi.dgc.client.gcInterval=28800000"}

异常2

[root@ecs-t-001-0001 jmxtrans]# Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=384m; support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=384m; support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: Using incremental CMS is deprecated and will likely be removed in a future release
Exception in thread "main" com.googlecode.jmxtrans.util.LifecycleException: com.googlecode.jmxtrans.util.LifecycleException: Error parsing json: /var/lib/jmxtrans/kafka.jsonat com.googlecode.jmxtrans.JmxTransformer.start(JmxTransformer.java:146)at com.googlecode.jmxtrans.JmxTransformer.doMain(JmxTransformer.java:107)at com.googlecode.jmxtrans.JmxTransformer.main(JmxTransformer.java:92)
Caused by: com.googlecode.jmxtrans.util.LifecycleException: Error parsing json: /var/lib/jmxtrans/kafka.jsonat com.googlecode.jmxtrans.JmxTransformer.processFilesIntoServers(JmxTransformer.java:358)at com.googlecode.jmxtrans.JmxTransformer.startupSystem(JmxTransformer.java:301)at com.googlecode.jmxtrans.JmxTransformer.start(JmxTransformer.java:142)... 2 more
Caused by: java.lang.IllegalArgumentException: Invalid type id 'com.googlecode.jmxtrans.model.output.InfluxDbWriterFactory' (for id type 'Id.class'): no such class foundat org.codehaus.jackson.map.jsontype.impl.ClassNameIdResolver.typeFromId(ClassNameIdResolver.java:89)at org.codehaus.jackson.map.jsontype.impl.TypeDeserializerBase._findDeserializer(TypeDeserializerBase.java:73)at org.codehaus.jackson.map.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromObject(AsPropertyTypeDeserializer.java:65)at org.codehaus.jackson.map.deser.AbstractDeserializer.deserializeWithType(AbstractDeserializer.java:81)at org.codehaus.jackson.map.deser.CollectionDeserializer.deserialize(CollectionDeserializer.java:118)at org.codehaus.jackson.map.deser.CollectionDeserializer.deserialize(CollectionDeserializer.java:93)at org.codehaus.jackson.map.deser.CollectionDeserializer.deserialize(CollectionDeserializer.java:25)at org.codehaus.jackson.map.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:149)at org.codehaus.jackson.map.deser.SettableBeanProperty$MethodProperty.deserializeAndSet(SettableBeanProperty.java:237)at org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:496)at org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:350)at org.codehaus.jackson.map.deser.CollectionDeserializer.deserialize(CollectionDeserializer.java:116)at org.codehaus.jackson.map.deser.CollectionDeserializer.deserialize(CollectionDeserializer.java:93)at org.codehaus.jackson.map.deser.CollectionDeserializer.deserialize(CollectionDeserializer.java:25)at org.codehaus.jackson.map.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:149)at org.codehaus.jackson.map.deser.SettableBeanProperty$MethodProperty.deserializeAndSet(SettableBeanProperty.java:237)at org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:496)at org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:350)at org.codehaus.jackson.map.deser.CollectionDeserializer.deserialize(CollectionDeserializer.java:116)at org.codehaus.jackson.map.deser.CollectionDeserializer.deserialize(CollectionDeserializer.java:93)at org.codehaus.jackson.map.deser.CollectionDeserializer.deserialize(CollectionDeserializer.java:25)at org.codehaus.jackson.map.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:149)at org.codehaus.jackson.map.deser.SettableBeanProperty$MethodProperty.deserializeAndSet(SettableBeanProperty.java:237)at org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:496)at org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:350)at org.codehaus.jackson.map.ObjectMapper._readMapAndClose(ObjectMapper.java:1980)at org.codehaus.jackson.map.ObjectMapper.readValue(ObjectMapper.java:1225)at com.googlecode.jmxtrans.util.JmxUtils.getJmxProcess(JmxUtils.java:494)at com.googlecode.jmxtrans.JmxTransformer.processFilesIntoServers(JmxTransformer.java:352)... 4 more

说是解析 com.googlecode.jmxtrans.util.LifecycleException 异常，这里需要我们自己编译一下jar包，在文章开头我们下载过jmxtrans源码

在项目目录下进行编译，文末有我编译好的jar包

mvn clean package -Dmaven.test.skip=true -DskipTests=true;

编译完成我们需要的jar包在 jmxtrans-jmxtrans-parent-271\jmxtrans\target 目录下

jmxtrans-271-all.jar 就是我们需要用的jar包

将jar包传到jmxtrans目录下

我们对比一下发现我们编译的包是有这个类的，而他自带的那个没有

[root@ecs-t-001-0001 jmxtrans]# grep 'com.googlecode.jmxtrans.model.output.InfluxDbWriterFactory' ./jmxtrans-271-all.jar 
Binary file ./jmxtrans-271-all.jar matches
[root@ecs-t-001-0001 jmxtrans]# grep 'com.googlecode.jmxtrans.model.output.InfluxDbWriterFactory' ./jmxtrans-all.jar 
[root@ecs-t-001-0001 jmxtrans]#

替换 jmxtrans.sh 中应用的 jmxtrans jar包名称

#JAR_FILE=${JAR_FILE:-"jmxtrans-all.jar"}
JAR_FILE=${JAR_FILE:-"jmxtrans-271-all.jar"}

再次启动即可

验证jmxtrans是否成功运行

进入 influx jmxDB数据库(我们之前创建的)，查看表 show MEASUREMENTS，我们发现自动创了jvmMemory

[root@ecs-t-001-0001 jmxtrans]# influx
Connected to http://localhost:8086 version 1.8.0
InfluxDB shell version: 1.8.0
> show databases
name: databases
name
----
_internal
jmxDB
> use jmxDB
Using database jmxDB
> show 
ERR: error parsing query: found EOF, expected CONTINUOUS, DATABASES, DIAGNOSTICS, FIELD, GRANTS, MEASUREMENT, MEASUREMENTS, QUERIES, RETENTION, SERIES, SHARD, SHARDS, STATS, SUBSCRIPTIONS, TAG, USERS at line 1, char 6
> show MEASUREMENTS
name: measurements
name
----
jvmMemory

具体查看数据，发现有数据写入

至此 jmxtrans 已成功监控 Kafka JMX端口，离成功更近了

安装 Grafana

安装

yum install grafana-6.7.3-1.x86_64.rpm

配置文件默认路径 /etc/grafana/grafana.ini

修改下web端口

> vim /etc/grafana/grafana.ini# web页面端口默认3000
http_port = 9099

启动服务并设置为开机启动

systemctl start grafana-server
systemctl enable grafana-server

查看启动状态

systemctl  status grafana-server

访问 web页面，第一次登陆账号密码是 admin/admin ，登陆完成后会提示你设置密码

开始配置Grafana显示模板

点击DataSource

选择添加 InfluxDB ，并填写基本信息

填写 influxDB 数据库信息

点击保存

此时InfluxDB配置完成，我们几区创建仪表盘

选择添加一个查询

选择我们上面设置的KafkaMonitor（因为我们只有一个默认的也是这个）

简单修改一下sql，因为我们上面只监控了JMX中的内存信息

注意：如果上面使用tag进行划分，这里就不要设置了，否则都叫Jvm内存使用了

简单配置完成，我们对比一下监控与jconsole监控对比

influxDB 与 Grafana 监控模板

模板下载

链接：https://pan.baidu.com/s/1ld-Yhv7wVutRxslV084GoQ 
提取码：0pzr 
复制这段内容后打开百度网盘手机App，操作更方便哦

Kafka JMX 监控之 jmxtrans + influxdb + grafana

效果图

环境准备

安装 influxdb

安装我们刚刚下载 influxdb rpm文件

查看默认配置

修改参数

启动 influxdb

查看启动状态

设置基本配置

influxdb 其他命令扩展

安装 jmxtrans

可能遇到的异常

验证jmxtrans是否成功运行

安装 Grafana

安装

influxDB 与 Grafana 监控模板

相关文章

实例浅析epoll的水平触发和边缘触发，以及边缘触发为什么要使用非阻塞IO

UML序列图

OpenTSDB 开发指南之查询数据

libcurl使用方法

OpenTSDB 开发指南之 Grafana 展示OpenTSDB监控数据

CDH 版本 Kafka 外网设置

OpenTSDB 安装

HBase原理 – snapshot 快照

linux如何自动化部署脚本实现免密登录并访问资源

时序数据库技术体系 – InfluxDB TSM存储引擎之数据读取

java多线程之生产者和消费者问题

时序数据库技术体系 – InfluxDB TSM存储引擎之数据写入

zookeeper结构和命令详解

时序数据库技术体系 – InfluxDB 多维查询之倒排索引

OSG框架分析

在Eclipse中如何操作zookpeer

Linux 系统进程守护工具 cesi + superviosr

Docker 服务器安装（一）

Docker 入门使用（二）

ElasticSearch sql 插件安装

Kafka JMX 监控 之 jmxtrans + influxdb + grafana

效果图

环境准备

安装 influxdb

安装我们刚刚下载 influxdb rpm文件

查看默认配置

修改参数

启动 influxdb

查看启动状态

设置基本配置

influxdb 其他命令扩展

安装 jmxtrans

可能遇到的异常

验证jmxtrans是否成功运行

安装 Grafana

安装

influxDB 与 Grafana 监控模板

相关文章

Kafka JMX 监控之 jmxtrans + influxdb + grafana