EMQX Operator 简介
EMQX Broker/Enterprise 是一个云原生的 MQTT 消息中间件。 我们提供了 EMQX Kubernetes Operator 来帮助您在 Kubernetes 的环境上快速创建和管理 EMQX Broker/Enterprise 集群。 它可以大大简化部署和管理 EMQX 集群的流程，对于管理和配置的知识要求也更低。它把部署和管理的工作变成一种低成本的、标准化的、可重复性的能力。
EMQX Operator 包括但不限于以下功能：
简化 EMQX 部署：通过 EMQX 自定义资源声明 EMQX 集群，并快速的部署，更多的内容，请查看快速开始。
管理 EMQX 集群：对 EMQX 进行自动化运维操作，包括集群升级、运行时数据持久化、根据 EMQX 的状态更新 Kubernetes 的资源等，更多的内容，请查看管理 EMQX 集群。
EMQX 与 EMQX Operator 的兼容性列表
EMQX 企业版
| EMQX 企业版 | EMQX Operator Version | APIVersion | Kind |
|---|---|---|---|
| 4.3.x （包含） ～ 4.4 | 1.2.1, 1.2.2, 1.2.3 | apps.emqx.io/v1beta3 | EmqxEnterprise |
| 4.4.6 （包含） ～ 4.4.8 | 1.2.5 | apps.emqx.io/v1beta3 | EmqxEnterprise |
| 4.4.8 （包含） ～ 4.4.14 | 1.2.6, 1.2.7, 1.2.8, 2.0.0, 2.0.1, 2.0.2, 2.0.3 | apps.emqx.io/v1beta3 | EmqxEnterprise |
| 4.4.14 （包含） 或更高 4.4.x | 2.1.0, 2.1.1 | apps.emqx.io/v1beta4 | EmqxEnterprise |
| 5.0.0 （包含） ～ 5.0.23 | 2.0.0, 2.0.1, 2.0.2, 2.0.3, 2.1.0, 2.1.1 | apps.emqx.io/v2alpha1 | EMQX |
| 5.1.1 或更高 | 2.2.0 | apps.emqx.io/v2beta1 | EMQX |
EMQX 开源版
| EMQX 开源版 | EMQX Operator Version | APIVersion | Kind |
|---|---|---|---|
| 4.3.x （包含） ～ 4.4 | 1.2.1, 1.2.2, 1.2.3 | apps.emqx.io/v1beta3 | EmqxBroker |
| 4.4.6 （包含） ～ 4.4.8 | 1.2.5 | apps.emqx.io/v1beta3 | EmqxBroker |
| 4.4.8 （包含） ～ 4.4.14 | 1.2.6, 1.2.7, 1.2.8, 2.0.0, 2.0.1, 2.0.2, 2.0.3 | apps.emqx.io/v1beta3 | EmqxBroker |
| 4.4.14 或更高 4.4.x | 2.1.0, 2.1.1 | apps.emqx.io/v1beta4 | EmqxBroker |
| 5.0.6 （包含） ～ 5.0.8 | 2.0.0, 2.0.1, 2.0.3 | apps.emqx.io/v2alpha1 | EMQX |
| 5.0.8 （包含） ～ 5.0.14 | 2.0.2 | apps.emqx.io/v2alpha1 | EMQX |
| 5.0.14 （包含） ～ 5.0.23 | 2.1.0, 2.1.1 | apps.emqx.io/v2alpha1 | EMQX |
| 5.1.1 或更高 | 2.2.0 | apps.emqx.io/v2beta1 | EMQX |
如何选择 Kubernetes 版本
EMQX Operator 要求 Kubernetes 集群的版本号 >=1.24。
| Kubernetes 版本 | EMQX Operator 兼容性 | 注释 |
|---|---|---|
| 1.24 更高 | 支持所有功能 | |
| 1.22 ( 包含) ～ 1.23 | 支持，但是不包含 MixedProtocolLBService | EMQX 集群只能在 LoadBalancer 类型的 Service 中使用一个协议，例如 TCP 或 UDP。 |
| 1.21 ( 包含) ～ 1.22 | 支持，但是不包含 Pod 删除开销 | EMQX Core + Replicant 模式集群时，更新 EMQX 集群无法准确的删除 Pod。 |
| 1.20 ( 包含) ～ 1.21 | 支持，但是如果使用 NodePort 类型的 Service，需要手动管理 .spec.ports[].nodePort | 更多的详情，请查看 Kubernetes changelog. |
| 1.16 ( 包含) ～ 1.20 | 支持，但是不推荐，因为缺乏足够的测试 | |
| 低于 1.16 | 不支持 | 低于 1.16 版本的 Kubernetes 不支持 apiextensions/v1 APIVersion。 |
快速开始
在本文中，我们将指导您完成高效设置 EMQX Operator 环境、安装 EMQX Operator，然后使用它部署 EMQX 所需的步骤。通过遵循本节中概述的指南，您将能够使用 EMQX Operator 有效地安装和管理 EMQX。
准备环境
在部署 EMQX Operator 之前，请确认以下组件已经准备就绪：
- 一个正在运行的 Kubernetes 集群，关于 Kubernetes 的版本，请查看如何选择 Kubernetes 版本
- 一个可以访问 Kubernetes 集群的 kubectl 工具。您可以使用
kubectl cluster-info命令检查 Kubernetes 集群的状态。 - Helm 3 或更高
安装 EMQX Operator
-
安装
cert-manger。TIP
需要
cert-manager版本1.1.6或更高。如果cert-manager已经安装并启动，请跳过此步骤。你可以使用 Helm 来安装
cert-manager。
$ helm repo add jetstack https://charts.jetstack.io
$ helm repo update
$ helm upgrade --install cert-manager jetstack/cert-manager \--namespace cert-manager \--create-namespace \--set installCRDs=true
-
或者按照 cert-manager 安装指南来安装它。
WARNING
如果您在 Google Kubernetes Engine（GKE） 上安装它。那么通过默认配置安装可能会导致 bootstraping 问题。所以通过增加
--set global.leaderElection.namespace=cert-manager这个配置为 leader 选举使用不同的命名空间。查看 cert-manager 兼容性 -
运行以下命令来安装 EMQX Operator。
bash
$ helm repo add emqx https://repos.emqx.io/charts $ helm repo update $ helm upgrade --install emqx-operator emqx/emqx-operator \--namespace emqx-operator-system \--create-namespace -
等待 EMQX Operator 就绪。
$ kubectl wait --for=condition=Ready pods -l "control-plane=controller-manager" -n emqx-operator-systempod/emqx-operator-controller-manager-57bd7b8bd4-h2mcr condition met
现在你已经成功的安装 EMQX Operator，你可以继续下一步了。在部署 EMQX 部分中，您将学习如何使用 EMQX Operator 来部署 EMQX。
部署 EMQX
- EMQX Enterprise 5
apiVersion: apps.emqx.io/v2beta1
kind: EMQX
metadata:name: emqx-ee
spec:image: emqx/emqx-enterprise:5.6
-
EMQX Open Source 5
apiVersion: apps.emqx.io/v2beta1 kind: EMQX metadata:name: emqx spec:image: emqx:5 -
EMQX Enterprise 4
apiVersion: apps.emqx.io/v1beta4 kind: EmqxEnterprise metadata:name: emqx-ee spec:template:spec:emqxContainer:image:repository: emqx/emqx-eeversion: 4.4.19 -
EMQX Open Source 4
apiVersion: apps.emqx.io/v1beta4
kind: EmqxBroker
metadata:name: emqx
spec:template:spec:emqxContainer:image:repository: emqxversion: 4.4.19
- 将下面的 YAML 配置文件保存为
emqx.yaml。
apiVersion: apps.emqx.io/v2beta1
kind: EMQX
metadata:name: emqx-ee
spec:image: emqx/emqx-enterprise:5.6
并使用 kubectl apply 命令来部署 EMQX。
$ kubectl apply -f emqx.yaml
关于 EMQX 自定义资源的更多信息，请查看 API 参考
- 检查 EMQX 集群状态，请确保 STATUS 为 Running，这可能需要一些时间等待 EMQX 集群准备就绪。
$ kubectl get emqxNAME IMAGE STATUS AGE
emqx-ee emqx/emqx-enterprise:5.1.0 Running 2m55s
在OCP上部署EMQX
1.安装 cert-manger。
cert-manager Operator for Red Hat OpenShift
安装完
namespace : cert-manager-operator下效果如下
namespace : cert-manager下效果如下
2.安装EMQX Operator
版本选择：
EMQX 开源版 ：4.4.14
EMQX Operator Version ： 2.2.22
$ helm search repo emqx/emqx-operator --versions
$ helm pull emqx/emqx-operator --version 2.2.22
# 解压后安装
$ helm install emqx-operator -f values.yaml .
$ oc wait --for=condition=Ready pods -l "control-plane=controller-manager" -n emqx-operator-system
pod/emqx-operator-controller-manager condition met
3.安装EMQX
storageClassName 填写为自己的。
apiVersion: apps.emqx.io/v1beta4
kind: EmqxBroker
metadata:name: emqx
spec:replicas: 1persistent:metadata:name: emqxspec:accessModes:- ReadWriteOnceresources:requests:storage: 200MistorageClassName: xxxxtemplate:spec:emqxContainer:resources:limits:cpu: '1'memory: 4Giimage:repository: emqxversion: 4.4.14volumeMounts:- mountPath: /opt/emqx/etc/emqx.confname: emqx-config-volumesubPath: emqx.confvolumes:- configMap:name: emqx-configname: emqx-config-volume
声明emqx-config,可以根据需求自己修改。
##不允许匿名登录
allow_anonymous = false
TCP协议透传 [需要nginx负载均衡, 无则不开启]
listener.tcp.external.proxy_protocol = on
## WS协议路径
listener.ws.external.mqtt_path = /ws
## WSS协议路径
listener.wss.external.mqtt_path = /ws
kind: ConfigMap
apiVersion: v1
metadata:name: emqx-config
data:emqx.conf: |-cluster.name = emqxclcluster.proto_dist = inet_tcpcluster.discovery = manualcluster.autoheal = oncluster.autoclean = 5mnode.name = emqx@127.0.0.1node.cookie = emqxsecretcookienode.data_dir = datanode.global_gc_interval = 15mnode.crash_dump = log/crash.dumpnode.dist_use_interface = 0.0.0.0node.dist_listen_min = 6369node.dist_listen_max = 6369node.backtrace_depth = 16rpc.mode = asyncrpc.async_batch_size = 256rpc.port_discovery = statelessrpc.connect_timeout = 5srpc.send_timeout = 5srpc.authentication_timeout = 5srpc.call_receive_timeout = 15srpc.socket_keepalive_idle = 900srpc.socket_keepalive_interval = 75srpc.socket_keepalive_count = 9rpc.socket_sndbuf = 1MBrpc.socket_recbuf = 1MBrpc.socket_buffer = 1MBlog.to = filelog.level = warninglog.dir = loglog.file = emqx.loglog.formatter = textlog.rotation = onlog.rotation.size = 10MBlog.rotation.count = 5listener.tcp.external.proxy_protocol = onallow_anonymous = trueacl_nomatch = allowacl_file = etc/acl.confenable_acl_cache = onacl_cache_max_size = 32acl_cache_ttl = 1macl_deny_action = ignoreflapping_detect_policy = 30, 1m, 5mauth_order = noneacl_order = nonemqtt.max_packet_size = 1MBmqtt.max_clientid_len = 65535mqtt.max_topic_levels = 128mqtt.max_qos_allowed = 2mqtt.max_topic_alias = 65535mqtt.retain_available = truemqtt.wildcard_subscription = truemqtt.shared_subscription = truemqtt.exclusive_subscription = falsemqtt.ignore_loop_deliver = falsemqtt.strict_mode = falsezone.external.idle_timeout = 15szone.external.enable_acl = onzone.external.enable_ban = onzone.external.enable_stats = onzone.external.acl_deny_action = ignorezone.external.force_gc_policy = 16000|16MBzone.external.keepalive_backoff = 0.75zone.external.max_subscriptions = 0zone.external.upgrade_qos = offzone.external.max_inflight = 32zone.external.retry_interval = 30szone.external.max_awaiting_rel = 100zone.external.await_rel_timeout = 300szone.external.session_expiry_interval = 2hzone.external.max_mqueue_len = 1000zone.external.mqueue_priorities = nonezone.external.mqueue_default_priority = highestzone.external.mqueue_store_qos0 = truezone.external.enable_flapping_detect = offzone.external.use_username_as_clientid = falsezone.external.ignore_loop_deliver = falsezone.external.strict_mode = falsezone.internal.allow_anonymous = truezone.internal.enable_stats = onzone.internal.enable_acl = offzone.internal.acl_deny_action = ignorezone.internal.max_subscriptions = 0zone.internal.max_inflight = 128zone.internal.max_awaiting_rel = 1000zone.internal.max_mqueue_len = 10000zone.internal.mqueue_store_qos0 = truezone.internal.enable_flapping_detect = offzone.internal.ignore_loop_deliver = falsezone.internal.strict_mode = falsezone.internal.bypass_auth_plugins = truelistener.tcp.external = 0.0.0.0:1883listener.tcp.external.acceptors = 8listener.tcp.external.max_connections = 1024000listener.tcp.external.max_conn_rate = 1000listener.tcp.external.active_n = 100listener.tcp.external.zone = externallistener.tcp.external.access.1 = allow alllistener.tcp.external.backlog = 1024listener.tcp.external.send_timeout = 15slistener.tcp.external.send_timeout_close = onlistener.tcp.external.nodelay = truelistener.tcp.external.reuseaddr = truelistener.tcp.internal = 127.0.0.1:11883listener.tcp.internal.acceptors = 4listener.tcp.internal.max_connections = 1024000listener.tcp.internal.max_conn_rate = 1000listener.tcp.internal.active_n = 1000listener.tcp.internal.zone = internallistener.tcp.internal.backlog = 512listener.tcp.internal.send_timeout = 5slistener.tcp.internal.send_timeout_close = onlistener.tcp.internal.recbuf = 64KBlistener.tcp.internal.sndbuf = 64KBlistener.tcp.internal.nodelay = falselistener.tcp.internal.reuseaddr = truelistener.ssl.external = 8883listener.ssl.external.acceptors = 16listener.ssl.external.max_connections = 102400listener.ssl.external.max_conn_rate = 500listener.ssl.external.active_n = 100listener.ssl.external.zone = externallistener.ssl.external.access.1 = allow alllistener.ssl.external.handshake_timeout = 15slistener.ssl.external.keyfile = etc/certs/key.pemlistener.ssl.external.certfile = etc/certs/cert.pemlistener.ssl.external.cacertfile = etc/certs/cacert.pemcrl_cache_http_timeout = 15scrl_cache_refresh_interval = 15mlistener.ssl.external.ciphers = TLS_AES_256_GCM_SHA384,TLS_AES_128_GCM_SHA256,TLS_CHACHA20_POLY1305_SHA256,TLS_AES_128_CCM_SHA256,TLS_AES_128_CCM_8_SHA256,ECDHE-ECDSA-AES256-GCM-SHA384,ECDHE-RSA-AES256-GCM-SHA384,ECDHE-ECDSA-AES256-SHA384,ECDHE-RSA-AES256-SHA384,ECDHE-ECDSA-DES-CBC3-SHA,ECDH-ECDSA-AES256-GCM-SHA384,ECDH-RSA-AES256-GCM-SHA384,ECDH-ECDSA-AES256-SHA384,ECDH-RSA-AES256-SHA384,DHE-DSS-AES256-GCM-SHA384,DHE-DSS-AES256-SHA256,AES256-GCM-SHA384,AES256-SHA256,ECDHE-ECDSA-AES128-GCM-SHA256,ECDHE-RSA-AES128-GCM-SHA256,ECDHE-ECDSA-AES128-SHA256,ECDHE-RSA-AES128-SHA256,ECDH-ECDSA-AES128-GCM-SHA256,ECDH-RSA-AES128-GCM-SHA256,ECDH-ECDSA-AES128-SHA256,ECDH-RSA-AES128-SHA256,DHE-DSS-AES128-GCM-SHA256,DHE-DSS-AES128-SHA256,AES128-GCM-SHA256,AES128-SHA256,ECDHE-ECDSA-AES256-SHA,ECDHE-RSA-AES256-SHA,DHE-DSS-AES256-SHA,ECDH-ECDSA-AES256-SHA,ECDH-RSA-AES256-SHA,AES256-SHA,ECDHE-ECDSA-AES128-SHA,ECDHE-RSA-AES128-SHA,DHE-DSS-AES128-SHA,ECDH-ECDSA-AES128-SHA,ECDH-RSA-AES128-SHA,AES128-SHAlistener.ssl.external.recbuf = 4KBlistener.ssl.external.sndbuf = 4KBlistener.ssl.external.reuseaddr = truelistener.ws.external = 8083listener.ws.external.mqtt_path = /wslistener.ws.external.acceptors = 4listener.ws.external.max_connections = 102400listener.ws.external.max_conn_rate = 1000listener.ws.external.active_n = 100listener.ws.external.zone = externallistener.ws.external.access.1 = allow alllistener.ws.external.backlog = 1024listener.ws.external.send_timeout = 15slistener.ws.external.send_timeout_close = onlistener.ws.external.nodelay = truelistener.ws.external.mqtt_piggyback = multiplelistener.ws.external.check_origin_enable = falselistener.ws.external.allow_origin_absence = truelistener.ws.external.check_origins = http://localhost:18083, http://127.0.0.1:18083listener.wss.external = 8084listener.wss.external.mqtt_path = /wslistener.wss.external.acceptors = 4listener.wss.external.max_connections = 102400listener.wss.external.max_conn_rate = 1000listener.wss.external.active_n = 100listener.wss.external.zone = externallistener.wss.external.access.1 = allow alllistener.wss.external.keyfile = etc/certs/key.pemlistener.wss.external.certfile = etc/certs/cert.pemlistener.wss.external.ciphers = TLS_AES_256_GCM_SHA384,TLS_AES_128_GCM_SHA256,TLS_CHACHA20_POLY1305_SHA256,TLS_AES_128_CCM_SHA256,TLS_AES_128_CCM_8_SHA256,ECDHE-ECDSA-AES256-GCM-SHA384,ECDHE-RSA-AES256-GCM-SHA384,ECDHE-ECDSA-AES256-SHA384,ECDHE-RSA-AES256-SHA384,ECDHE-ECDSA-DES-CBC3-SHA,ECDH-ECDSA-AES256-GCM-SHA384,ECDH-RSA-AES256-GCM-SHA384,ECDH-ECDSA-AES256-SHA384,ECDH-RSA-AES256-SHA384,DHE-DSS-AES256-GCM-SHA384,DHE-DSS-AES256-SHA256,AES256-GCM-SHA384,AES256-SHA256,ECDHE-ECDSA-AES128-GCM-SHA256,ECDHE-RSA-AES128-GCM-SHA256,ECDHE-ECDSA-AES128-SHA256,ECDHE-RSA-AES128-SHA256,ECDH-ECDSA-AES128-GCM-SHA256,ECDH-RSA-AES128-GCM-SHA256,ECDH-ECDSA-AES128-SHA256,ECDH-RSA-AES128-SHA256,DHE-DSS-AES128-GCM-SHA256,DHE-DSS-AES128-SHA256,AES128-GCM-SHA256,AES128-SHA256,ECDHE-ECDSA-AES256-SHA,ECDHE-RSA-AES256-SHA,DHE-DSS-AES256-SHA,ECDH-ECDSA-AES256-SHA,ECDH-RSA-AES256-SHA,AES256-SHA,ECDHE-ECDSA-AES128-SHA,ECDHE-RSA-AES128-SHA,DHE-DSS-AES128-SHA,ECDH-ECDSA-AES128-SHA,ECDH-RSA-AES128-SHA,AES128-SHAlistener.wss.external.backlog = 1024listener.wss.external.send_timeout = 15slistener.wss.external.send_timeout_close = onlistener.wss.external.mqtt_piggyback = multiplelistener.wss.external.check_origin_enable = falselistener.wss.external.allow_origin_absence = truelistener.wss.external.check_origins = https://localhost:8084, https://127.0.0.1:8084modules.loaded_file = data/loaded_modulesmodule.presence.qos = 1plugins.etc_dir = etc/plugins/plugins.loaded_file = data/loaded_pluginsplugins.expand_plugins_dir = etc/plugins/broker.sys_interval = 1mbroker.sys_heartbeat = 30sbroker.session_locking_strategy = quorumbroker.shared_subscription_strategy = randombroker.shared_dispatch_ack_enabled = falsebroker.route_batch_clean = offsysmon.long_gc = 0sysmon.long_schedule = 240mssysmon.large_heap = 8MBsysmon.busy_port = falsesysmon.busy_dist_port = trueos_mon.cpu_check_interval = 60sos_mon.cpu_high_watermark = 80%os_mon.cpu_low_watermark = 60%os_mon.mem_check_interval = 60sos_mon.sysmem_high_watermark = 70%os_mon.procmem_high_watermark = 5%vm_mon.check_interval = 30svm_mon.process_high_watermark = 80%vm_mon.process_low_watermark = 60%alarm.actions = log,publishalarm.size_limit = 1000alarm.validity_period = 24h默认的Operator不会安装dashborad，所以下一步我们需要安装
service.yaml
kind: Service
apiVersion: v1
metadata:name: emqx
spec:ports:- name: http-management-8081protocol: TCPport: 8081targetPort: 8081- name: http-dashboard-18083protocol: TCPport: 18083targetPort: 18083- name: ws-8083protocol: TCPport: 8083targetPort: 8083type: ClusterIPselector:apps.emqx.io/instance: emqxapps.emqx.io/managed-by: emqx-operatorroute.yaml
host字段需要修改
kind: Route
apiVersion: route.openshift.io/v1
metadata:name: emqx-dashborad
spec:host: xxxxxxto:kind: Servicename: emqxweight: 100port:targetPort: http-dashboard-18083wildcardPolicy: Nonetls: null
登入验证：
http://${host}
用户名：admin
密码：public
![Servlet[springmvc]的Servlet.init()引发异常](https://i-blog.csdnimg.cn/direct/1e69e9a5f2704fa79c22356498f346fc.png)