k8s实战

部署harbor作为k8s镜像仓库

1.实验目标

部署k8s私有镜像仓库harbor把demo小项目需要的镜像上传到harbor上修改demo项目的资源配置清单，镜像地址修改为harbord的地址

2.再node1上安装harbor

[root@node1 ~]# cd /opt/#上传harbor软件包[root@node1 /opt]# rz -Erz waiting to receive.#解压[root@node1 /opt]# tar zxf harbor-offline-installer-v1.9.0-rc1.tgz#进入解压后的文件目录[root@node1 /opt]# cd harbor/

3.编辑harbor配置文件

#备份[root@node1 /opt/harbor]# cp harbor.yml harbor.yml.bak#编辑配置文件[root@node1 /opt/harbor]# vim harbor.yml #需要更改的地方hostname: 10.0.0.11port: 8888harbor_admin_password: 123456data_volume: /data/harbor

4.执行安装

#在安装harbor是许诺先安装docker-compose，否则报错[root@node1 /opt/harbor]# yum install docker-compose -y#安装harbor(注意命令执行的所在目录)[root@node1 /opt/harbor]# ./install.sh

5.浏览器访问

http://10.0.0.11:8888用户：admin密码：123456

6.建立镜像仓库

这里有2种访问级别：公开：任何人都可以直接访问并下载镜像私有：登陆授权后才允许下载镜像#注意如果创建私有仓库，k8s是不能直接下载的，需要配置安全文件

7. 所有节点都配置docker信任harbor仓库并重启docker 注意：所有节点

#配置信任仓库cat >/etc/docker/daemon.json <

8.docker登陆harbor ( 所有节点 都执行 )

[root@node1 /opt/harbor]# docker login 10.0.0.11:8888Username: adminPassword: #密码 123456WARNING! Your password will be stored unencrypted in /root/.docker/config.json.Configure a credential helper to remove this warning. Seehttps://docs.docker.com/engine/reference/commandline/login/#credentials-storeLogin Succeeded

9.下载镜像修改tag并push到harbor上 ( 注意：从节点执行 )

1.在主节点查询镜像存放的节点位置[root@node1 ~]# kubectl get pod -o wide NAME                     READY   STATUS    RESTARTS   AGE   IP         NODE    NOMINATED NODE   READINESS GATESmysql-8fcd9f64-vqkm9     1/1     Running   1          18m   10.2.1.4   node2   myweb-6f974fdbdc-gsncp   1/1     Running   1          18m   10.2.1.5   node2   myweb-6f974fdbdc-ngngv   1/1     Running   1          18m   10.2.2.3   node3   2.根据主节点获取的信息在从节点执行打标签[root@node2 ~]# docker tag kubeguide/tomcat-app:v1 10.0.0.11:8888/k8s/tomcat-app:v1[root@node2 ~]# docker tag mysql:5.7 10.0.0.11:8888/k8s/mysql:5.73.将打好的标签的镜像上传到harbor仓库[root@node2 ~]# docker push 10.0.0.11:8888/k8s/tomcat-app:v1 [root@node2 ~]# docker push 10.0.0.11:8888/k8s/mysql:5.7

10.节点上删除镜像

#注意需要先删除标签镜像在删除源镜像docker rmi 10.0.0.11:8888/k8s/mysql:5.7 docker rmi 10.0.0.11:8888/k8s/tomcat-app:v1docker rmi mysql:5.7 docker rmi kubeguide/tomcat-app:v1

11.删除以前的demo项目 注意：主节点执行

[root@node1 ~]# kubectl delete -f tomcat-demo.yamldeployment.apps "mysql" deletedservice "mysql" deleteddeployment.apps "myweb" deletedservice "myweb" deleted

12.修改demo项目的资源配置清单里的镜像地址

[root@node1 ~]# vim tomcat-demo.yaml   #注意更改的位置原来image: mysql:5.7  变更为： image: 10.0.0.11:8888/k8s/mysql:5.7原来image: k8s/tomcat-app:v1   变更为： image: 10.0.0.11:8888/k8s/tomcat-app:v1

13.应用资源配置清单

[root@node1 ~]# kubectl create -f tomcat-demo.yaml deployment.apps/mysql createdservice/mysql createddeployment.apps/myweb createdservice/myweb created

14.报错

#此时查看pod状态会发现镜像拉取失败了[root@node1 ~]# kubectl get podNAME                     READY   STATUS             RESTARTS   AGEmysql-7d746b5577-wtxtm   0/1     ErrImagePull       0          15smyweb-764df5ffdd-jvvmf   0/1     ImagePullBackOff   0          15smyweb-764df5ffdd-rc9pc   0/1     ImagePullBackOff   0          15s#查看pod创建的详细信息[root@node1 ~]# kubectl describe pod mysql-7d746b5577-可以tab自己的数据#关键报错信息：Failed to pull image "10.0.0.11:8888/k8s/mysql:5.7": rpc error: code = Unknown desc = Error response from daemon: pull access denied for 10.0.0.11:8888/k8s/mysql, repository does not exist or may require 'docker login'翻译：项目不出在或者需要登录

15.查看docker登陆的密码文件

[root@node1 ~]# docker login 10.0.0.11:8888Authenticating with existing credentials...WARNING! Your password will be stored unencrypted in /root/.docker/config.json.Configure a credential helper to remove this warning. Seehttps://docs.docker.com/engine/reference/commandline/login/#credentials-storeLogin Succeeded#查看加密密码文件[root@node1 ~]# cat /root/.docker/config.json{"auths": {"10.0.0.11:8888": {"auth": "YWRtaW46MTIzNDU2"}},"HttpHeaders": {"User-Agent": "Docker-Client/18.09.9 (linux)"}

16.将docker密码文件解码成base64编码 解码：base64

[root@node1 ~]# cat /root/.docker/config.json|base64ewoJImF1dGhzIjogewoJCSIxMC4wLjAuMTE6ODg4OCI6IHsKCQkJImF1dGgiOiAiWVdSdGFXNDZNVEl6TkRVMiIKCQl9Cgl9LAoJIkh0dHBIZWFkZXJzIjogewoJCSJVc2VyLUFnZW50IjogIkRvY2tlci1DbGllbnQvMTguMDkuOSAobGludXgpIgoJfQp9#每一个人的都不一样

17.创建并应用docker登陆的Secret资源

#注意！！！1.dockerconfigjson: xxx直接写base64的编码，不需要换行2.base64编码是一整行，不是好几行3.最后的type字段不能少[root@node1 ~]# cat >harbor-secret.yaml<

18.修改demo资源配置清单，添加拉取镜像的参数

查看命令帮助kubectl explain deployment.spec.template.spec.imagePullSecrets修改资源配置清单修改文件----------------------------      imagePullSecrets:       - name: harbor-secret----------------------------  #注意：mysql和tomcat都需要增加[root@node1 ~/demo]# cat tomcat-demo.yaml apiVersion: apps/v1kind: Deployment metadata:  name: mysqlspec:  replicas: 1  selector:    matchLabels:      app: mysql  template:    metadata:      labels:        app: mysql    spec:      containers:      - name: mysql        image: 10.0.0.11:8888/k8s/mysql:5.7        ports:        - containerPort: 3306        env:        - name: MYSQL_ROOT_PASSWORD          value: "123456"      imagePullSecrets:      - name: harbor-secret---apiVersion: v1kind: Servicemetadata:  name: mysqlspec:  ports:    - port: 3306  selector:    app: mysql---apiVersion: apps/v1kind: Deployment metadata:  name: mywebspec:  replicas: 2  selector:    matchLabels:      app: myweb  template:    metadata:      labels:        app: myweb    spec:      containers:      - name: myweb        image: 10.0.0.11:8888/k8s/tomcat-app:v1        ports:        - containerPort: 8080        env:        - name: MYSQL_SERVICE_HOST          value: 'mysql'        - name: MYSQL_SERVICE_PORT          value: '3306'      imagePullSecrets:      - name: harbor-secret---apiVersion: v1kind: Servicemetadata:  name: mywebspec:  type: NodePort  ports:    - port: 8080      nodePort: 30001  selector:    app: myweb

19.应用资源配置清单并查看

1.删除资源配置清单[root@node1 ~]# kubectl delete -f tomcat-demo.yaml2.创建新的资源[root@node1 ~]# kubectl create -f tomcat-demo.yaml deployment.apps/mysql createdservice/mysql createddeployment.apps/myweb createdservice/myweb created3.查询下载的资源kubectl get pod -o wide

20.浏览器查看

http://10.0.0.11:30001/demo

报错总结：

#报错总结：1.如果要删除的镜像正在被容器使用，那么你是删不了的2.harbor卸载不干净，/data/harbor/目录下的数据也要删除3.secret配置只写了一个dp，实际上有几个deployment就需要写几个重做k8s使用harbor作为私有仓库1.停止harbor正在运行的容器2.删除harbor的容器docker ps -a|grep "goharbor"|awk '{print "docker rm "$1}'3.删除harbor的镜像dockerimages|grep "goharbor"|awk '{print "docker rmi "$1":"$2}'4.解压并修改harbor配置文件hostname: 10.0.0.11port: 8888harbor_admin_password: 123456data_volume: /data/harbor5.执行安装并访问./install.shhttp://10.0.0.11:88886.创建一个私有仓库k8s7.配置docker信任仓库并重启--三台服务器都操作!!!    {      "registry-mirrors": ["https://ig2l319y.mirror.aliyuncs.com"],      "exec-opts": ["native.cgroupdriver=systemd"],      "insecure-registries" : ["http://10.0.0.11:8888"]    }systemctl restart docker注意！！！node1重启后harbor会失效，需要重启harborcd /opt/harbordocker-compose stopdocker-compose start8.docker登陆harbordocker login 10.0.0.11:88889.将docker登陆凭证转化为k8s能识别的base64编码[root@node1 ~]# cat /root/.docker/config.json|base64ewoJImF1dGhzIjogewoJCSIxMC4wLjAuMTE6ODg4OCI6IHsKCQkJImF1dGgiOiAiWVdSdGFXNDZNVEl6TkRVMiIKCQl9Cgl9LAoJIkh0dHBIZWFkZXJzIjogewoJCSJVc2VyLUFnZW50IjogIkRvY2tlci1DbGllbnQvMTguMDkuOSAobGludXgpIgoJfQp910.编写Secert资源配置清单[root@node1 ~/demo]# cat harbor-secret.yaml apiVersion: v1kind: Secretmetadata:  name: harbor-secretdata:  .dockerconfigjson: ewoJImF1dGhzIjogewoJCSIxMC4wLjAuMTE6ODg4OCI6IHsKCQkJImF1dGgiOiAiWVdSdGFXNDZNVEl6TkRVMiIKCQl9Cgl9LAoJIkh0dHBIZWFkZXJzIjogewoJCSJVc2VyLUFnZW50IjogIkRvY2tlci1DbGllbnQvMTguMDkuOSAobGludXgpIgoJfQp9type: kubernetes.io/dockerconfigjson11.应用Secret资源kubectl delete -f harbor-secret.yaml kubectl create -f harbor-secret.yamlkubectl get secrets12.修改镜像tag并上传到harbordocker tag kubeguide/tomcat-app:v1 10.0.0.11:8888/k8s/tomcat-app:v1docker tag mysql:5.7 10.0.0.11:8888/k8s/mysql:5.7docker push 10.0.0.11:8888/k8s/tomcat-app:v1docker push 10.0.0.11:8888/k8s/mysql:5.7 13.修改demo资源配置清单####mysqlimagePullSecrets:      - name: harbor-secret###tomcatimagePullSecrets:      - name: harbor-secret14.应用资源清单并查看kubectl apply -f .kubectl get pod