根据 istio 文档 可知,编译 istio 源码分为两种方式,一种是直接在 linux 机器上编译源码,另外一种是通过 docker 镜像工具包编译与构建。
编译 istio 1.6.14 centos 7 rpm 依赖包
istio 1.6.x 系列版本是没有提供 centos 7 物理机的 envoy rpm 依赖包的,我们通过编译 istio 源码来产生 1.6.14 版本的 rpm 依赖包。在 /root/go/src/istio.io
上克隆 istio 镜像仓库 git clone https://github.com/istio/istio.git && cd istio
,切换到 1.6.14 分支,git checkout -b 1.6.14 1.6.14
。
备注:尽量最好请先自行解决网络问题,解决翻墙问题,需要下载镜像 gcr.io/istio-testing/build-tools:release-1.6-2020-11-13T15-30-50
执行 make VERSION=1.6.14 rpm/builder-image
,如果编译镜像出现以下错误,目前是注释了 tools/packaging/rpm/Dockerfile.build
中的 /usr/bin/ninja-build /usr/bin/ninja
软链接操作即可。
Step 5/11 : RUN curl -o /usr/local/bin/bazel -L https://github.com/bazelbuild/bazelisk/releases/download/v1.1.0/bazelisk-linux-amd64 && chmod +x /usr/local/bin/bazel
---> Using cache
---> e59b3bc8b8ca
Step 6/11 : RUN ln -s /usr/bin/cmake3 /usr/bin/cmake && ln -s /usr/bin/ninja-build /usr/bin/ninja
---> Running in c46f7d52d3f5
ln: failed to create symbolic link '/usr/bin/ninja': File exists
The command '/bin/sh -c ln -s /usr/bin/cmake3 /usr/bin/cmake && ln -s /usr/bin/ninja-build /usr/bin/ninja' returned a non-zero code: 1
tools/packaging/rpm/rpm.mk:32: recipe for target 'rpm/builder-image' failed
make[1]: *** [rpm/builder-image] Error 1
make: *** [Makefile:46: rpm/builder-image] Error 2
编译镜像成功,如下所示:
[root@mesh-10-20-11-190 istio]# make VERSION=1.6.14 rpm/builder-image
docker build -t istio-rpm-builder -f /work/tools/packaging/rpm/Dockerfile.build /work/tools/packaging/rpm
Sending build context to Docker daemon 27.14kB
Step 1/11 : FROM centos:7
---> 8652b9f0cb4c
Step 2/11 : RUN yum install -y centos-release-scl epel-release && yum update -y && yum install -y fedpkg sudo devtoolset-7-gcc devtoolset-7-gcc-c++ devtoolset-7-binutils java-1.8.0-openjdk-headless rsync rh-git218 wget unzip which make cmake3 patch ninja-build devtoolset-7-libatomic-devel openssl python27 libtool autoconf && yum clean all
---> Using cache
---> 8fb1d6dd0c17
Step 3/11 : RUN curl -o /root/go.tar.gz https://dl.google.com/go/go1.13.4.linux-amd64.tar.gz && tar zxf /root/go.tar.gz -C /usr/local
---> Using cache
---> a27dac7c7d03
Step 4/11 : ENV GOROOT=/usr/local/go PATH=/usr/local/go/bin:/opt/rh/rh-git218/root/usr/bin:/opt/rh/devtoolset-7/root/usr/bin:/opt/llvm/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:${PATH}
---> Using cache
---> 34db8194bcaa
Step 5/11 : RUN curl -o /usr/local/bin/bazel -L https://github.com/bazelbuild/bazelisk/releases/download/v1.1.0/bazelisk-linux-amd64 && chmod +x /usr/local/bin/bazel
---> Using cache
---> e59b3bc8b8ca
Step 6/11 : RUN ln -s /usr/bin/cmake3 /usr/bin/cmake
---> Using cache
---> 88a0535d8d22
Step 7/11 : RUN echo "/opt/rh/httpd24/root/usr/lib64" > /etc/ld.so.conf.d/httpd24.conf && ldconfig
---> Using cache
---> 05ec9f46546b
Step 8/11 : ENV LLVM_VERSION=9.0.0
---> Using cache
---> 4b2748bd32c5
Step 9/11 : ENV LLVM_DISTRO="x86_64-linux-sles11.3"
---> Using cache
---> ab821fe0a7d6
Step 10/11 : ENV LLVM_RELEASE="clang+llvm-${LLVM_VERSION}-${LLVM_DISTRO}"
---> Using cache
---> 8b092c276236
Step 11/11 : RUN curl -fsSL --output ${LLVM_RELEASE}.tar.xz https://releases.llvm.org/${LLVM_VERSION}/${LLVM_RELEASE}.tar.xz && tar Jxf ${LLVM_RELEASE}.tar.xz && mv ./${LLVM_RELEASE} /opt/llvm && chown -R root:root /opt/llvm && rm ./${LLVM_RELEASE}.tar.xz && echo "/opt/llvm/lib" > /etc/ld.so.conf.d/llvm.conf && ldconfig
---> Running in 266cf6894400
/usr/bin/ninja-build /usr/bin/ninjaRemoving intermediate container 266cf6894400
---> 5a26ceb0e73d
Successfully built 5a26ceb0e73d
Successfully tagged istio-rpm-builder:latest
执行 make VERSION=1.6.14 rpm/proxy
,执行命令的过程中,如果出现以下错误:
docker run --rm -it \
-v /go:/go \
-w /builder \
-e USER= \
-e ISTIO_ENVOY_VERSION=1ef6cb53abbb057185f4bcb60e28cf92c3a174ad \
-e ISTIO_GO=/work \
-e ISTIO_OUT=/work/out/linux_amd64 \
-e PACKAGE_VERSION=1.6.14 \
-e USER_ID=0 \
-e GROUP_ID=994 \
istio-rpm-builder \
/work/tools/packaging/rpm/build-proxy-rpm.sh
docker: Error response from daemon: OCI runtime create failed: container_linux.go:367: starting container process caused: exec: "/work/tools/packaging/rpm/build-proxy-rpm.sh": stat /work/tools/packaging/rpm/build-proxy-rpm.sh: no such file or directory: unknown.
ERRO[0003] error waiting for container: context canceled
tools/packaging/rpm/rpm.mk:18: recipe for target 'rpm/proxy' failed
需要更改 tools/packaging/rpm/rpm.mk
下的 docker 运行命令,将 -v /root/go/src/istio.io/istio:/work
挂载在容器中即可。需要注意的是挂载的目录需要与 istio 项目所在的文件夹的位置一致。
rpm/proxy:
docker run --rm -it \
-v ${GO_TOP}:${GO_TOP} \
-v /root/go/src/istio.io/istio:/work \
-w /builder \
-e USER=${USER} \
-e ISTIO_ENVOY_VERSION=${ISTIO_ENVOY_VERSION} \
-e ISTIO_GO=${ISTIO_GO} \
-e ISTIO_OUT=${ISTIO_OUT} \
-e PACKAGE_VERSION=${PACKAGE_VERSION} \
-e USER_ID=$(shell id -u) \
-e GROUP_ID=$(shell id -g) \
istio-rpm-builder \
${PWD}/tools/packaging/rpm/build-proxy-rpm.sh
如果 istio/proxy 仓库克隆不下来,则可以执行以下操作。
#git clone https://github.com/istio/proxy.git istio-proxy
git clone https://github.com.cnpmjs.org/istio/proxy.git istio-proxy
等待上述过程即可,编译的过程耗时较长,推荐使用 8c16G(包含以及以上) 的机器配置,可能耗费数小时。你可以使用 top 命令查看,物理机器 8c cpu 接近全部被打满。在 out/linux_amd64/rpm
目录下会生成 istio-proxy-1.6.14-1.el7.x86_64 rpm
依赖包。
编译 istio 镜像并推送到自己的私有仓库
在 /root/go/src/istio.io
上克隆 istio 镜像仓库,git clone https://github.com/istio/istio.git && cd istio
,
export USER="tanjunchen"
export HUB="registry.cn-hangzhou.aliyuncs.com/$USER"
export TAG="dev-tanjunchen"
需要自行登录私有仓库镜像的地址,此次以 master 分支,commitId 为 7cc1bb6eb59faa36b73cb64e40d50b588c1153e2 的 istio 提交记录作为构建与编译案例。make build
此操作将会拉取 gcr.io/istio-testing/build-tools:master-2021-04-12T17-40-14
镜像,需要翻墙拉取,并且镜像分支版本可能会不一样。
# 编译二进制文件
make build
root@tanjunchen:~/go/src/istio.io/istio# make build
useradd: UID 0 is not unique
mkdir -p /work/out/linux_amd64/logs
mkdir -p /work/out/linux_amd64/release
GOOS=linux GOARCH=amd64 LDFLAGS='-extldflags -static -s -w' common/scripts/gobuild.sh /work/out/linux_amd64/ ./istioctl/cmd/istioctl ./pilot/cmd/pilot-discovery ./pkg/test/echo/cmd/client ./pkg/test/echo/cmd/server ./operator/cmd/operator ./cni/cmd/istio-cni ./cni/cmd/istio-cni-repair ./cni/cmd/istio-cni-taint ./cni/cmd/install-cni ./tools/istio-iptables ./tools/bug-report
go: downloading golang.org/x/text v0.3.5
go: downloading sigs.k8s.io/gateway-api v0.3.0
go: downloading sigs.k8s.io/mcs-api v0.1.0
go: downloading github.com/pierrec/lz4 v2.0.5+incompatible
go: downloading github.com/aws/aws-sdk-go v1.38.3
go: downloading github.com/pierrec/lz4/v4 v4.0.3
go: downloading github.com/jmespath/go-jmespath v0.4.0
real 10m28.431s
user 12m8.419s
sys 1m18.972s
GOOS=linux GOARCH=amd64 LDFLAGS='-extldflags -static -s -w' common/scripts/gobuild.sh /work/out/linux_amd64/ -tags=agent ./pilot/cmd/pilot-agent
real 0m9.745s
user 0m29.966s
sys 0m3.806s
root@tanjunchen:~/go/src/istio.io/istio# ls
bin CODEOWNERS CONTRIBUTING.md go.mod istio.deps logo Makefile.overrides.mk out prow releasenotes SUPPORT.md
BUGS-AND-FEATURE-REQUESTS.md common docker go.sum LICENSE Makefile manifests pilot README.md samples tests
cni common-protos galley istioctl licenses Makefile.core.mk operator pkg release security tools
root@tanjunchen:~/go/src/istio.io/istio# tree out/
out/
└── linux_amd64
├── bug-report
├── client
├── envoy
├── envoy-centos
├── install-cni
├── istio-cni
├── istio-cni-repair
├── istio-cni-taint
├── istioctl
├── istio-iptables
├── istio_is_init
├── logs
├── operator
├── pilot-agent
├── pilot-discovery
├── release
│ ├── envoy
│ ├── envoy-759e940a5aee8d8973ffb4d9dad6773e9d605adf
│ ├── envoy-centos
│ ├── envoy-centos-759e940a5aee8d8973ffb4d9dad6773e9d605adf
│ ├── metadata_exchange-759e940a5aee8d8973ffb4d9dad6773e9d605adf.compiled.wasm
│ ├── metadata_exchange-759e940a5aee8d8973ffb4d9dad6773e9d605adf.wasm
│ ├── metadata-exchange-filter.compiled.wasm
│ ├── metadata-exchange-filter.wasm
│ ├── stats-759e940a5aee8d8973ffb4d9dad6773e9d605adf.compiled.wasm
│ ├── stats-759e940a5aee8d8973ffb4d9dad6773e9d605adf.wasm
│ ├── stats-filter.compiled.wasm
│ └── stats-filter.wasm
└── server
编译 istio docker 镜像,make docker build
详细流程如下所示:
mkdir -p /work/out/linux_amd64/logs
mkdir -p /work/out/linux_amd64/release
GOOS=linux GOARCH=amd64 LDFLAGS='-extldflags -static -s -w' common/scripts/gobuild.sh /work/out/linux_amd64/ ./istioctl/cmd/istioctl ./pilot/cmd/pilot-discovery ./pkg/test/echo/cmd/client ./pkg/test/echo/cmd/server ./operator/cmd/operator ./cni/cmd/istio-cni ./cni/cmd/istio-cni-repair ./cni/cmd/istio-cni-taint ./cni/cmd/install-cni ./tools/istio-iptables ./tools/bug-report
GOOS=linux GOARCH=amd64 LDFLAGS='-extldflags -static -s -w' common/scripts/gobuild.sh /work/out/linux_amd64/ -tags=agent ./pilot/cmd/pilot-agent
time (mkdir -p /work/out/linux_amd64/docker_build/docker.pilot && cp -rp pilot/docker/Dockerfile.pilot /work/out/linux_amd64/pilot-discovery /work/out/linux_amd64/docker_build/docker.pilot && cd /work/out/linux_amd64/docker_build/docker.pilot && chmod 755 pilot-discovery && docker build --build-arg BASE_VERSION=1.10-dev.1 --build-arg BASE_DISTRIBUTION=default -t registry.cn-hangzhou.aliyuncs.com/tanjunchen/pilot:dev-tanjunchen -f Dockerfile.pilot . );
Sending build context to Docker daemon 91.11MB
Step 1/8 : ARG BASE_DISTRIBUTION=default
Step 2/8 : ARG BASE_VERSION=latest
Step 3/8 : FROM gcr.io/istio-release/base:${BASE_VERSION} as default
---> d1542e974449
Step 4/8 : FROM gcr.io/istio-release/distroless:${BASE_VERSION} as distroless
---> e4794e4d4bb6
Step 5/8 : FROM ${BASE_DISTRIBUTION}
---> d1542e974449
Step 6/8 : COPY pilot-discovery /usr/local/bin/
---> Using cache
---> dd929463e8ec
Step 7/8 : USER 1337:1337
---> Using cache
---> c1582e1824ad
Step 8/8 : ENTRYPOINT ["/usr/local/bin/pilot-discovery"]
---> Using cache
---> 109a5ab0ba4b
Successfully built 109a5ab0ba4b
Successfully tagged registry.cn-hangzhou.aliyuncs.com/tanjunchen/pilot:dev-tanjunchen
time (mkdir -p /work/out/linux_amd64/docker_build/docker.proxyv2 && cp -rp /work/out/linux_amd64/release/metadata-exchange-filter.compiled.wasm /work/tools/packaging/common//envoy_bootstrap.json /work/tools/packaging/common//gcp_envoy_bootstrap.json /work/out/linux_amd64/release/envoy /work/out/linux_amd64/pilot-agent pilot/docker/Dockerfile.proxyv2 /work/out/linux_amd64/release/stats-filter.wasm /work/out/linux_amd64/release/stats-filter.compiled.wasm /work/out/linux_amd64/release/metadata-exchange-filter.wasm /work/out/linux_amd64/docker_build/docker.proxyv2 && cd /work/out/linux_amd64/docker_build/docker.proxyv2 && chmod 755 envoy pilot-agent && chmod 644 envoy_bootstrap.json gcp_envoy_bootstrap.json && docker build --build-arg proxy_version=istio-proxy:759e940a5aee8d8973ffb4d9dad6773e9d605adf --build-arg istio_version=1.10-dev --build-arg BASE_VERSION=1.10-dev.1 --build-arg SIDECAR=envoy --build-arg BASE_DISTRIBUTION=default -t registry.cn-hangzhou.aliyuncs.com/tanjunchen/proxyv2:dev-tanjunchen -f Dockerfile.proxyv2 . );
Sending build context to Docker daemon 148.4MB
Step 1/20 : ARG BASE_DISTRIBUTION=default
Step 2/20 : ARG BASE_VERSION=latest
Step 3/20 : FROM gcr.io/istio-release/base:${BASE_VERSION} as default
---> d1542e974449
Step 4/20 : FROM gcr.io/istio-release/iptables@sha256:8601b3cb13984e375d9a9a85687f23c88bc798e4f2ec9a80cca5e6abda66c6f9 as distroless
---> aa9db1ff07ce
Step 5/20 : FROM ${BASE_DISTRIBUTION}
---> d1542e974449
Step 6/20 : WORKDIR /
---> Using cache
---> 7921fae2382b
Step 7/20 : ARG proxy_version
---> Using cache
---> 57c7b2876f47
Step 8/20 : ARG istio_version
---> Using cache
---> d63eff22462b
Step 9/20 : ARG SIDECAR=envoy
---> Using cache
---> c2b6be0c50e0
Step 10/20 : COPY envoy_bootstrap.json /var/lib/istio/envoy/envoy_bootstrap_tmpl.json
---> Using cache
---> 13d79f257ec7
Step 11/20 : COPY gcp_envoy_bootstrap.json /var/lib/istio/envoy/gcp_envoy_bootstrap_tmpl.json
---> Using cache
---> 14bd9b60c532
Step 12/20 : COPY $SIDECAR /usr/local/bin/$SIDECAR
---> Using cache
---> 9214168d067c
Step 13/20 : ENV ISTIO_META_ISTIO_PROXY_SHA $proxy_version
---> Using cache
---> c9c7916e2dc7
Step 14/20 : ENV ISTIO_META_ISTIO_VERSION $istio_version
---> Using cache
---> 54c30796e3ac
Step 15/20 : COPY pilot-agent /usr/local/bin/pilot-agent
---> Using cache
---> 6b97155022dc
Step 16/20 : COPY stats-filter.wasm /etc/istio/extensions/stats-filter.wasm
---> Using cache
---> 4cba90d741d8
Step 17/20 : COPY stats-filter.compiled.wasm /etc/istio/extensions/stats-filter.compiled.wasm
---> Using cache
---> 599db691bef0
Step 18/20 : COPY metadata-exchange-filter.wasm /etc/istio/extensions/metadata-exchange-filter.wasm
---> Using cache
---> e6c764e8cc57
Step 19/20 : COPY metadata-exchange-filter.compiled.wasm /etc/istio/extensions/metadata-exchange-filter.compiled.wasm
---> Using cache
---> d867f4f5bf10
Step 20/20 : ENTRYPOINT ["/usr/local/bin/pilot-agent"]
---> Using cache
---> b276d91c589e
Successfully built b276d91c589e
Successfully tagged registry.cn-hangzhou.aliyuncs.com/tanjunchen/proxyv2:dev-tanjunchen
time (mkdir -p /work/out/linux_amd64/docker_build/docker.app && cp -rp /work/out/linux_amd64/docker_temp/certs pkg/test/echo/docker/Dockerfile.app /work/out/linux_amd64/client /work/out/linux_amd64/server /work/out/linux_amd64/docker_build/docker.app && cd /work/out/linux_amd64/docker_build/docker.app && chmod 755 server client && docker build --build-arg BASE_VERSION=1.10-dev.1 --build-arg BASE_DISTRIBUTION=default -t registry.cn-hangzhou.aliyuncs.com/tanjunchen/app:dev-tanjunchen -f Dockerfile.app . );
Sending build context to Docker daemon 31.23MB
Step 1/7 : ARG BASE_VERSION=latest
Step 2/7 : FROM gcr.io/istio-release/base:${BASE_VERSION}
---> d1542e974449
将编译后的 istio docker 镜像推送到私有仓库,make docker push
,详细流程如下所示:
mkdir -p /work/out/linux_amd64/logs
mkdir -p /work/out/linux_amd64/release
GOOS=linux GOARCH=amd64 LDFLAGS='-extldflags -static -s -w' common/scripts/gobuild.sh /work/out/linux_amd64/ ./istioctl/cmd/istioctl ./pilot/cmd/pilot-discovery ./pkg/test/echo/cmd/client ./pkg/test/echo/cmd/server ./operator/cmd/operator ./cni/cmd/istio-cni ./cni/cmd/istio-cni-repair ./cni/cmd/istio-cni-taint ./cni/cmd/install-cni ./tools/istio-iptables ./tools/bug-report
GOOS=linux GOARCH=amd64 LDFLAGS='-extldflags -static -s -w' common/scripts/gobuild.sh /work/out/linux_amd64/ -tags=agent ./pilot/cmd/pilot-agent
time (mkdir -p /work/out/linux_amd64/docker_build/docker.pilot && cp -rp pilot/docker/Dockerfile.pilot /work/out/linux_amd64/pilot-discovery /work/out/linux_amd64/docker_build/docker.pilot && cd /work/out/linux_amd64/docker_build/docker.pilot && chmod 755 pilot-discovery && docker build --build-arg BASE_VERSION=1.10-dev.1 --build-arg BASE_DISTRIBUTION=default -t registry.cn-hangzhou.aliyuncs.com/tanjunchen/pilot:dev-tanjunchen -f Dockerfile.pilot . );
Sending build context to Docker daemon 91.11MB
Step 1/8 : ARG BASE_DISTRIBUTION=default
Step 2/8 : ARG BASE_VERSION=latest
Step 3/8 : FROM gcr.io/istio-release/base:${BASE_VERSION} as default
---> d1542e974449
Step 4/8 : FROM gcr.io/istio-release/distroless:${BASE_VERSION} as distroless
---> e4794e4d4bb6
Step 5/8 : FROM ${BASE_DISTRIBUTION}
---> d1542e974449
Step 6/8 : COPY pilot-discovery /usr/local/bin/
---> Using cache
---> dd929463e8ec
Step 7/8 : USER 1337:1337
---> Using cache
---> c1582e1824ad
Step 8/8 : ENTRYPOINT ["/usr/local/bin/pilot-discovery"]
---> Using cache
---> 109a5ab0ba4b
Successfully built 109a5ab0ba4b
Successfully tagged registry.cn-hangzhou.aliyuncs.com/tanjunchen/pilot:dev-tanjunchen
......
f6dd38aa176a: Layer already exists
fe6d8881187d: Layer already exists
23135df75b44: Layer already exists
0933ae865aef: Layer already exists
dev-tanjunchen: digest: sha256:59383c79c515666f94238e82d16d691ba77f8f6c1297a432eb0b518a0eb2fee0 size: 2421
查看生成的 istio docker 私有镜像。
root@tanjunchen:~# docker images | grep dev-tanjunchen
registry.cn-hangzhou.aliyuncs.com/tanjunchen/app_sidecar_centos_7 dev-tanjunchen d88d2b86dfc5 43 minutes ago 581MB
registry.cn-hangzhou.aliyuncs.com/tanjunchen/app_sidecar_centos_8 dev-tanjunchen 1015cfa1cfc2 44 minutes ago 569MB
registry.cn-hangzhou.aliyuncs.com/tanjunchen/app_sidecar_debian_10 dev-tanjunchen 63f97cf191ce 45 minutes ago 379MB
registry.cn-hangzhou.aliyuncs.com/tanjunchen/app_sidecar_debian_9 dev-tanjunchen 0dddbc4081c0 45 minutes ago 363MB
registry.cn-hangzhou.aliyuncs.com/tanjunchen/app_sidecar_ubuntu_focal dev-tanjunchen 85c9102d27f5 46 minutes ago 343MB
registry.cn-hangzhou.aliyuncs.com/tanjunchen/app_sidecar_ubuntu_bionic dev-tanjunchen b803ec33ca1b 46 minutes ago 345MB
registry.cn-hangzhou.aliyuncs.com/tanjunchen/app_sidecar_ubuntu_xenial dev-tanjunchen 09fb9b7a4557 47 minutes ago 398MB
registry.cn-hangzhou.aliyuncs.com/tanjunchen/install-cni dev-tanjunchen f05f796de449 About an hour ago 335MB
registry.cn-hangzhou.aliyuncs.com/tanjunchen/operator dev-tanjunchen dd83e3528e07 About an hour ago 246MB
registry.cn-hangzhou.aliyuncs.com/tanjunchen/istioctl dev-tanjunchen 67d778570229 About an hour ago 248MB
registry.cn-hangzhou.aliyuncs.com/tanjunchen/app dev-tanjunchen f91c3a40ecb3 2 hours ago 184MB
registry.cn-hangzhou.aliyuncs.com/tanjunchen/proxyv2 dev-tanjunchen b276d91c589e 2 hours ago 301MB
registry.cn-hangzhou.aliyuncs.com/tanjunchen/pilot dev-tanjunchen 109a5ab0ba4b 2 hours ago 244MB
至此,所有 Istio 使用的镜像全部编译与构建完成,如上所示。
参考