版本信息:
操作系统:GUN Linux操作系统AARCH64架构。
istio-proxy版本:istio-proxy1.15.2
编译环境搭建:
设置代理,确保可以访问Google等外网,这里envoy的第一次编译需要从外网下载依赖库。// 备注:这里一定要设置一下,可以避免很多麻烦
2. 安装G++ 10.3.1 // 其他版本应该也可以,本人安装了这个版本
3.安装cmake // 本人安装的是 3.22.0版本
4.安装ninja //本人安装版本是1.10.1
5.安装libstdc++静态库 // yum install -y libstdc++-static
安装完之后,到/usr/lib/gcc/aarch64-linux-gnu/10.3.1下面能够看到stdlibc++.a这个文件,就算成功了。(单独安装的原因参考后面的编译错误。)
特别说明:安装命令用的是 yum install -y XXX的方式。
6.安装bazel,这里没有特别的,按照官方文档进行操作就可以了。
https://github.com/envoyproxy/envoy/blob/v1.28.0/bazel/README.md#supported-compiler-versions# sudo wget -O /usr/local/bin/bazel https://github.com/bazelbuild/bazelisk/releases/latest/download/bazelisk-linux-$([ $(uname -m) = "aarch64" ] && echo "arm64" || echo "amd64") Resolving github.com (github.com)... 20.205.243.166 Connecting to github.com (github.com)|20.205.243.166|:443... connected. HTTP request sent, awaiting response... 302 Found Location: https://github.com/bazelbuild/bazelisk/releases/download/v1.19.0/bazelisk-linux-arm64 [following]Reusing existing connection to github.com:443. HTTP request sent, awaiting response... 302 Found Location: https://objects.githubusercontent.com/github-production-release-asset-2e65be/149661467/04314521-c57c-42fe-8b8c-5173058c7c76?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20231219%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20231219T024713Z&X-Amz-Expires=300&X-Amz-Signature=3094802b498630fdb9035addf38a543e035d086cd90755d84581b9ab535406ab&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=149661467&response-content-disposition=attachment%3B%20filename%3Dbazelisk-linux-arm64&response-content-type=application%2Foctet-stream [following]Resolving objects.githubusercontent.com (objects.githubusercontent.com)... 185.199.109.133, 185.199.111.133, 185.199.110.133,... Connecting to objects.githubusercontent.com (objects.githubusercontent.com)|185.199.109.133|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 4980736 (4.8M) [application/octet-stream] Saving to: ‘/usr/local/bin/bazel’/usr/local/bin/bazel 100%[=====================================================>] 4.75M 3.71MB/s in 1.3s.......
编译envoy:
下载代码到本地:
git clone XXX // XXX:对应的是github或者你自己库里面的istio-proxy的代码
2.执行编译命令:
set -e
# output_user_root 设置编译缓存目录,设置了之后,外网下载的东西就不需要每次都下载了,并且后续编译是增量编译
make build_envoy BAZEL_STARTUP_ARGS="--output_user_root=/home/istio-proxy/cache"
3.编译问题一:
错误:Note: Either `libc++` or `libstdc++-7-dev` (or higher) must be installed.
原因和解决办法:libstdc++静态库需要单独安装,因为GUN Linux操作系统在安装G++的时候,并没有自动生成stdlibc++.a这个静态库,而是生成了一个叫做stdlibc++fs.a的库,这会导致后面envoy编译的时候,提示找不到stdlibc++.a这个库。
4.编译问题二:
错误:
bazel-out/aarch64-opt/bin/extensions/common/wasm/nlohmann_json.hpp:8494:24: error: comparison is always true due to limited range of data type [-Werror=type-limits]
8494 | if ('\x00' <= c and c <= '\x1F')
| ~~~~~~~^~~~
cc1plus: all warnings being treated as errors
原因:
参考:https://www.cnblogs.com/ejutuo/archive/2012/08/23/2653359.html
解决办法:
本人通过直接修改对应的bazel编译文件移除"-Werror"来达到目地。如下所示:
# git diff envoy/bazel/envoy_internal.bzl
diff --git a/envoy/bazel/envoy_internal.bzl b/envoy/bazel/envoy_internal.bzl
index 76bce7f..2308273 100644
--- a/envoy/bazel/envoy_internal.bzl
+++ b/envoy/bazel/envoy_internal.bzl
@@ -7,7 +7,6 @@ def envoy_copts(repository, test = False):posix_options = ["-Wall","-Wextra",
- "-Werror", // 备注,这里要移除,因为操作系统的 字符串兼容问题,不然编译会卡死在类似下面问题上"-Wnon-virtual-dtor","-Woverloaded-virtual","-Wold-style-cast",
5.编译问题三:与编译问题二类似,不过是出现在级联的第三方库里面。
错误:
opt/bin/external/com_github_datadog_dd_opentracing_cpp/_virtual_includes/3rd_party_nlohmann/nlohmann/json.hpp:8494:24: error: comparison is always true due to limited range of data type [-Werror=type-limits]8494 | if ('\x00' <= c and c <= '\x1F')| ~~~~~~~^~~~
cc1plus: all warnings being treated as errors
Target //src/envoy:envoy failed to build
解决方法:本人使用了缓存的方式,所以直接修改了缓存内的第三方编译的bazel文件 // 最好的方法是通过设置环境变量来避免掉这里的修改,后续再研究
解决办法:手动魔改出问题第三方库里面的make编译参数 /root/.cache/bazel/_bazel_root/2d6185edcef03d5d7233a743a6490d13/external/com_github_datadog_dd_opentracing_cpp/BUILD.bazel 移除对应的"-Werror"参数
6.envoy对应的二进制文件生成
上面问题解决掉之后,后续就比较顺利了,envoy 二进制文件直接生成到:
../istio-proxy1.15.2/bazel-bin/src/envoy下面
通过#file envoy 可以确认envoy二进制的版本确实是 ARM aarch64版本。
总结:
网上搜了一圈,也没有找到关于envoy arm 的编译方式,踩了很多坑,便整理了这篇文章,希望对envoy开发和使用的朋友们有帮助。