目录

一. 前言

二. 使用 SSL 的加密和身份验证

2.4. 生产中的常见缺陷（Common Pitfalls in Production）

2.4.1. 扩展密钥用法（Extended Key Usage）

2.4.2. 中间证书（Intermediate Certificates）

2.4.3. 复制扩展字段失败（Failure to copy extension fields）

2.5. 配置 Kafka Broker（Configuring Kafka Brokers）

2.6. 配置 Kafka 客户端（Configuring Kafka Clients）

---

一. 前言

    接上一篇《（二）Kafka 安全之使用 SSL 的加密和身份验证》，本文从 2.4 小节开始。

二. 使用 SSL 的加密和身份验证

2.4. 生产中的常见缺陷（Common Pitfalls in Production）

原文引用：The above paragraphs show the process to create your own CA and use it to sign certificates for your cluster. While very useful for sandbox, dev, test, and similar systems, this is usually not the correct process to create certificates for a production cluster in a corporate environment. Enterprises will normally operate their own CA and users can send in CSRs to be signed with this CA, which has the benefit of users not being responsible to keep the CA secure as well as a central authority that everybody can trust. However it also takes away a lot of control over the process of signing certificates from the user. Quite often the persons operating corporate CAs will apply tight restrictions on certificates that can cause issues when trying to use these certificates with Kafka.

    上面的段落显示了创建自己的 CA 并使用它为集群签名证书的过程。虽然这对沙箱、开发、测试和类似系统非常有用，但通常不是在公司环境中为生产集群创建证书的正确过程。企业通常会运营自己的 CA，用户可以发送 CSRs 与该 CA 进行签名，这样做的好处是用户不必负责维护 CA 的安全，并且是每个人都可以信任的中央机构。然而，它也剥夺了用户对证书签名过程的大量控制权。通常，运营公司 CA 的人员会对证书施加严格的限制，这可能会在尝试将这些证书与 Kafka 一起使用时引发问题。

2.4.1. 扩展密钥用法（Extended Key Usage）

原文引用：Certificates may contain an extension field that controls the purpose for which the certificate can be used. If this field is empty, there are no restrictions on the usage, but if any usage is specified in here, valid SSL implementations have to enforce these usages.

Relevant usages for Kafka are:

• Client authentication
• Server authentication

    证书可能包含一个扩展字段，用于控制证书的使用目的。如果此字段为空，则对用法没有限制，但如果此处指定了任何用法，则有效的 SSL 实现必须强制执行这些用法。

Kafka 的相关用法有：

• Client authentication（客户端身份验证）。
• Server authentication（服务器身份验证）。

原文引用：Kafka brokers need both these usages to be allowed, as for intra-cluster communication every broker will behave as both the client and the server towards other brokers. It is not uncommon for corporate CAs to have a signing profile for webservers and use this for Kafka as well, which will only contain the serverAuth usage value and cause the SSL handshake to fail.

    Kafka Broker 需要允许这两种用途，因为对于集群内通信，每个 Broker 都将作为客户端和服务器对其他 Broker 进行操作。公司 CAs 为 Web 服务器提供签名配置文件，并将其用于 Kafka，这并不罕见，因为它只包含 serverAuth 使用值，并导致 SSL 握手失败。

2.4.2. 中间证书（Intermediate Certificates）

原文引用：Corporate Root CAs are often kept offline for security reasons. To enable day-to-day usage, so called intermediate CAs are created, which are then used to sign the final certificates. When importing a certificate into the keystore that was signed by an intermediate CA it is necessary to provide the entire chain of trust up to the root CA. This can be done by simply cating the certificate files into one combined certificate file and then importing this with keytool.

    出于安全原因，公司 Root CAs 通常保持离线。为了实现日常使用，创建了所谓的中间 CAs，然后用于签署最终证书。将证书导入由中间 CA 签名的密钥库时，有必要将整个信任链提供给 Root CA。这可以通过将证书文件简单地定位到一个组合的证书文件中，然后使用 keytool 导入来完成。

2.4.3. 复制扩展字段失败（Failure to copy extension fields）

原文引用：CA operators are often hesitant to copy and requested extension fields from CSRs and prefer to specify these themselves as this makes it harder for a malicious party to obtain certificates with potentially misleading or fraudulent values. It is advisable to double check signed certificates, whether these contain all requested SAN fields to enable proper hostname verification. The following command can be used to print certificate details to the console, which should be compared with what was originally requested:

    CA 运营商通常对从 CSRs 中复制和请求扩展字段犹豫不决，更喜欢自己指定这些字段，因为这会使恶意方更难获得具有潜在误导或欺诈价值的证书。建议仔细检查已签名的证书，无论这些证书是否包含所有请求的 SAN 字段，以启用正确的主机名验证。以下命令可用于将证书详细信息打印到控制台，应将其与最初请求的内容进行比较：

> openssl x509 -in certificate.crt -text -noout

2.5. 配置 Kafka Broker（Configuring Kafka Brokers）

原文引用：If SSL is not enabled for inter-broker communication (see below for how to enable it), both PLAINTEXT and SSL ports will be necessary.

    如果没有为 Broker 程序间通信启用 SSL（请参阅下面的启用方法），则 PLAINTEXT 和 SSL 端口都是必需的。

listeners=PLAINTEXT://host.name:port,SSL://host.name:port

Broker 端需要以下 SSL 配置（Following SSL configs are needed on the broker side）

ssl.keystore.location=/var/private/ssl/server.keystore.jks
ssl.keystore.password=test1234
ssl.key.password=test1234
ssl.truststore.location=/var/private/ssl/server.truststore.jks
ssl.truststore.password=test1234

原文引用：Note: ssl.truststore.password is technically optional but highly recommended. If a password is not set access to the truststore is still available, but integrity checking is disabled. Optional settings that are worth considering:

1. ssl.client.auth=none ("required" => client authentication is required, "requested" => client authentication is requested and client without certs can still connect. The usage of "requested" is discouraged as it provides a false sense of security and misconfigured clients will still connect successfully.)
2. ssl.cipher.suites (Optional). A cipher suite is a named combination of authentication, encryption, MAC and key exchange algorithm used to negotiate the security settings for a network connection using TLS or SSL network protocol. (Default is an empty list)
3. ssl.enabled.protocols=TLSv1.2,TLSv1.1,TLSv1 (list out the SSL protocols that you are going to accept from clients. Do note that SSL is deprecated in favor of TLS and using SSL in production is not recommended)
4. ssl.keystore.type=JKS
5. ssl.truststore.type=JKS
6. ssl.secure.random.implementation=SHA1PRNG

If you want to enable SSL for inter-broker communication, add the following to the server.properties file (it defaults to PLAINTEXT)

注意：ssl.truststore.password 在技术上是可选的，但强烈建议使用。如果未设置密码，则对信任库的访问仍然可用，但完整性检查被禁用。值得考虑的可选设置：

1. ssl.client.auth=none（“required”=>需要客户端身份验证，“requested”=>请求客户端身份验证并且没有证书的客户端仍然可以连接。不鼓励使用“requested”，因为它提供了虚假的安全感，并且配置错误的客户端仍然会成功连接。）
2. ssl.cipher.suites（可选）。密码套件是身份验证、加密、MAC 和密钥交换算法的命名组合，用于协商使用 TLS 或 SSL 网络协议的网络连接的安全设置。（默认为空列表）
3. ssl.enabled.procols=TLSv1.2、TLSv1.1、TLSv1（列出您将从客户端接受的 SSL 协议。请注意，SSL 已被弃用，取而代之的是 TLS，不建议在生产中使用 SSL
4. ssl.keystore.type=JKS
5. ssl.truststore.type=JKS
6. ssl.secure.random.implementation=SHA1PRNG

    如果您想为 Broker 间通信启用 SSL，请将以下内容添加到 server.properties 文件中（默认为PLAINTEXT）

security.inter.broker.protocol=SSL

原文引用：Due to import regulations in some countries, the Oracle implementation limits the strength of cryptographic algorithms available by default. If stronger algorithms are needed (for example, AES with 256-bit keys), the JCE Unlimited Strength Jurisdiction Policy Files must be obtained and installed in the JDK/JRE. See the JCA Providers Documentation for more information.

    由于某些国家/地区的进口法规，Oracle 的实施限制了默认情况下可用的加密算法的强度。如果需要更强的算法（例如，具有256位密钥的 AES），则必须获取 JCE 无限制强度管辖权策略文件并将其安装在 JDK/JRE 中。有关更多信息，请参阅 JCA 提供商文档。

原文引用：The JRE/JDK will have a default pseudo-random number generator (PRNG) that is used for cryptography operations, so it is not required to configure the implementation used with the ssl.secure.random.implementation. However, there are performance issues with some implementations (notably, the default chosen on Linux systems, NativePRNG, utilizes a global lock). In cases where performance of SSL connections becomes an issue, consider explicitly setting the implementation to be used. The SHA1PRNG implementation is non-blocking, and has shown very good performance characteristics under heavy load (50 MB/sec of produced messages, plus replication traffic, per-broker).

Once you start the broker you should be able to see in the server.log

    JRE/JDK 将有一个用于加密操作的默认伪随机数生成器（PRNG），因此不需要配置与ssl.secure.random.implementation 一起使用的实现。然而，有些实现存在性能问题（特别是，Linux 系统上选择的默认 NativePRNG 使用全局锁）。如果 SSL 连接的性能成为一个问题，请考虑明确设置要使用的实现。SHA1PRNG 实现是非阻塞的，并且在重负载（每个代理产生 50 MB/秒的消息，加上复制流量）下显示出非常好的性能特征。

    启动 Broker 后，您应该能够在 server.log 中看到

with addresses: PLAINTEXT -> EndPoint(192.168.64.1,9092,PLAINTEXT),SSL -> EndPoint(192.168.64.1,9093,SSL)

原文引用：To check quickly if the server keystore and truststore are setup properly you can run the following command

要快速检查服务器密钥库和信任库是否设置正确，可以运行以下命令

> openssl s_client -debug -connect localhost:9093 -tls1

原文引用：(Note: TLSv1 should be listed under ssl.enabled.protocols)
In the output of this command you should see server's certificate:

（注意：TLSv1 应列在 ssl.enabled 协议下）

在该命令的输出中，您应该看到服务器的证书：

-----BEGIN CERTIFICATE-----
{variable sized random bytes}
-----END CERTIFICATE-----
subject=/C=US/ST=CA/L=Santa Clara/O=org/OU=org/CN=Sriharsha Chintalapani
issuer=/C=US/ST=CA/L=Santa Clara/O=org/OU=org/CN=kafka/emailAddress=test@test.com

原文引用：If the certificate does not show up or if there are any other error messages then your keystore is not setup properly.

如果证书没有显示，或者存在任何其他错误消息，则表明您的密钥库设置不正确。

2.6. 配置 Kafka 客户端（Configuring Kafka Clients）

原文引用：SSL is supported only for the new Kafka Producer and Consumer, the older API is not supported. The configs for SSL will be the same for both producer and consumer.

If client authentication is not required in the broker, then the following is a minimal configuration example:

    SSL 仅支持新的 Kafka Producer 和 Consumer，不支持旧的 API。SSL 的配置对于生产者和消费者都是相同的。

    如果 Broker 中不需要客户端身份验证，那么下面是一个最小配置示例：

security.protocol=SSL
ssl.truststore.location=/var/private/ssl/client.truststore.jks
ssl.truststore.password=test1234

原文引用：Note: ssl.truststore.password is technically optional but highly recommended. If a password is not set access to the truststore is still available, but integrity checking is disabled. If client authentication is required, then a keystore must be created like in step 1 and the following must also be configured:

注意：ssl.truststore.password 在技术上是可选的，但强烈建议使用。如果未设置密码，则对信任库的访问仍然可用，但完整性检查被禁用。如果需要客户端身份验证，则必须像步骤1中那样创建密钥库，并且还必须配置以下内容：

ssl.keystore.location=/var/private/ssl/client.keystore.jks
ssl.keystore.password=test1234
ssl.key.password=test1234

原文引用：Other configuration settings that may also be needed depending on our requirements and the broker configuration:

1. ssl.provider (Optional). The name of the security provider used for SSL connections. Default value is the default security provider of the JVM.
2. ssl.cipher.suites (Optional). A cipher suite is a named combination of authentication, encryption, MAC and key exchange algorithm used to negotiate the security settings for a network connection using TLS or SSL network protocol.
3. ssl.enabled.protocols=TLSv1.2,TLSv1.1,TLSv1. It should list at least one of the protocols configured on the broker side
4. ssl.truststore.type=JKS
5. ssl.keystore.type=JKS

Examples using console-producer and console-consumer:

根据我们的要求和 Broker 配置，可能还需要其他配置设置：

1. ssl.provider（可选）。用于 SSL 连接的安全提供程序的名称。默认值是 JVM 的默认安全提供程序。
2. ssl.cipher.suites（可选）。密码套件是身份验证、加密、MAC 和密钥交换算法的命名组合，用于协商使用 TLS 或 SSL 网络协议的网络连接的安全设置。
3. ssl.enabled.procols=TLSv1.2、TLSv1.1、TLSv1。它应该至少列出一个在 Broker 端配置的协议
4. ssl.truststore.type=JKS
5. ssl.keystore.type=JKS

使用控制台生产者和控制台消费者的示例：

> kafka-console-producer.sh --bootstrap-server localhost:9093 --topic test --producer.config client-ssl.properties
> kafka-console-consumer.sh --bootstrap-server localhost:9093 --topic test --consumer.config client-ssl.properties