目录
1. 说明
2. 步骤
2.1 准备工作
2.2 安装
2.2.1 用jenkins原站for k8s的安装仓方法安装
2.2.2 Helm 安装
3. 相关命令
4. 遇到的问题
5. 参考
1. 说明
- 在k8s上部署jenkins，并用 jenkins.k8s-t2.com访问
- 在namespace为devops下安装在指定节点k8s-master-1，有指定持久化的PV/PVC/SC
- CI/DI 实践
2. 步骤
2.1 准备工作
设置代理，不然去git拿文件的时候会遇到麻烦
git config --global http.proxy 'socks5://192.168.0.108:1080'
git config --global https.proxy 'socks5://192.168.0.108:1080'git config --global --unset http.proxy
git config --global --unset https.proxy编辑客户机hosts, 映射子域名
192.168.0.103 jenkins.k8s-t2.com
2.2 安装
2.2.1 用jenkins原站for k8s的安装仓方法安装
获取
git clone https://github.com/scriptcamp/kubernetes-jenkins建个namespace
kubectl create ns devops-tools编辑 volume.yaml，设置/data0/jenkins-volume为存储地, 节点 k8s-master-0
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:name: local-storage
provisioner: kubernetes.io/no-provisioner
volumeBindingMode: WaitForFirstConsumer---
apiVersion: v1
kind: PersistentVolume
metadata:name: jenkins-pv-volumelabels:type: local
spec:storageClassName: local-storageclaimRef:name: jenkins-pv-claimnamespace: devops-toolscapacity:storage: 10GiaccessModes:- ReadWriteOncelocal:path: /data0/jenkins-volumenodeAffinity:required:nodeSelectorTerms:- matchExpressions:- key: kubernetes.io/hostnameoperator: Invalues:- k8s-master-0---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:name: jenkins-pv-claimnamespace: devops-tools
spec:storageClassName: local-storageaccessModes:- ReadWriteOnceresources:requests:storage: 3Gi建serviceAccount.yaml
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:name: jenkins-admin
rules:- apiGroups: [""]resources: ["*"]verbs: ["*"]---
apiVersion: v1
kind: ServiceAccount
metadata:name: jenkins-adminnamespace: devops-tools---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:name: jenkins-admin
roleRef:apiGroup: rbac.authorization.k8s.iokind: ClusterRolename: jenkins-admin
subjects:
- kind: ServiceAccountname: jenkins-adminnamespace: devops-tools建deployment.yaml ，为使得jenkins插件能科学安装，需进行deployment中的环境代理设置，否则牙痛 : )
apiVersion: apps/v1
kind: Deployment
metadata:name: jenkinsnamespace: devops-tools
spec:replicas: 1selector:matchLabels:app: jenkins-servertemplate:metadata:labels:app: jenkins-serverspec:securityContext:fsGroup: 1000 runAsUser: 1000serviceAccountName: jenkins-admincontainers:- name: jenkinsimage: jenkins/jenkins:ltsresources:limits:memory: "2Gi"cpu: "1000m"requests:memory: "500Mi"cpu: "500m"ports:- name: httpportcontainerPort: 8080- name: jnlpportcontainerPort: 50000livenessProbe:httpGet:path: "/login"port: 8080initialDelaySeconds: 90periodSeconds: 10timeoutSeconds: 5failureThreshold: 5readinessProbe:httpGet:path: "/login"port: 8080initialDelaySeconds: 60periodSeconds: 10timeoutSeconds: 5failureThreshold: 3volumeMounts:- name: jenkins-datamountPath: /var/jenkins_home env:- name: http_proxyvalue: http://192.168.0.108:1081- name: https_proxyvalue: http://192.168.0.108:1081- name: no_proxyvalue: aliyuncs.com,huaweicloud.com,k8s-master-0,k8s-master-1,k8s-worker-0,localhost,127.0.0.1,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16volumes:- name: jenkins-datapersistentVolumeClaim:claimName: jenkins-pv-claim建service.yaml
apiVersion: v1
kind: Service
metadata:name: jenkins-servicenamespace: devops-toolsannotations:prometheus.io/scrape: 'true'prometheus.io/path: /prometheus.io/port: '8080'
spec:selector: app: jenkins-servertype: NodePort ports:- port: 8080targetPort: 8080nodePort: 32000执行脚本k8s-jenkins.sh
#!/bin/bashkubectl label node k8s-master-0 app=jenkins-serverkubectl create namespace devops-toolskubectl apply -f /k8s_apps/kubernetes-jenkins/serviceAccount.yamlkubectl create -f /k8s_apps/kubernetes-jenkins/volume.yamlkubectl apply -f /k8s_apps/kubernetes-jenkins/deployment.yamlkubectl apply -f /k8s_apps/kubernetes-jenkins/service.yaml
成功后可查pod日志获取admin密码
2.2.2 Helm 安装
添加jenkins来源:
helm repo add jenkinsci https://charts.jenkins.io
helm repo updatehelm search repo jenkinsci
可知当前版本为:
NAME CHART VERSION APP VERSION DESCRIPTION
jenkinsci/jenkins 4.3.26 2.401.1 Jenkins - Build great things at any scale! The ...
获取到本地，并解压
helm fetch jenkinsci/jenkins
tar -zxvf jenkins.tgz
编辑 values.yaml：
ingress:#enabled: false
=>
ingress:enabled: true# See https://kubernetes.io/blog/2020/04/02/improvements-to-the-ingress-api-in-kubernetes-1.18/#specifying-the-class-of-an-ingress# 需注明用的是哪个ingress class，因为之前装的是ingress-nginx, 所以这里填nginxingressClassName: nginx# Set this path to jenkinsUriPrefix above or use annotations to rewrite pathhostName: jenkins.k8s-t2.com注意ingress需指定对应的 ingressClassName
执行安装
helm upgrade --install jenkins --namespace default \-f values.yaml \jenkins/jenkins# 过程大概要30分钟NOTES:
1. 获取admin登录密码 Get your 'admin' user password by running:kubectl exec --namespace default -it svc/jenkins -c jenkins -- /bin/cat /run/secrets/additional/chart-admin-password && echo
2. Visit http://jenkins.k8s-t2.com3. Login with the password from step 1 and the username: admin
4. Configure security realm and authorization strategy
5. Use Jenkins Configuration as Code by specifying configScripts in your values.yaml file, see documentation: http://jenkins.k8s-t2.com/configuration-as-code and examples: https://github.com/jenkinsci/configuration-as-code-plugin/tree/master/demos
获知部署后的情况
orangepi@k8s-master-1:/k8s_apps/jenkins$ kubectl describe ingress jenkins
Name: jenkins
Labels: app.kubernetes.io/component=jenkins-controllerapp.kubernetes.io/instance=jenkinsapp.kubernetes.io/managed-by=Helmapp.kubernetes.io/name=jenkinshelm.sh/chart=jenkins-4.3.23
Namespace: default
Address:
Ingress Class: nginx
Default backend: <default>
Rules:Host Path Backends---- ---- --------jenkins.k8s-t2.com/jenkins jenkins:8080 (10.244.2.7:8080)
Annotations: kubernetes.io/ingress.class: nginxmeta.helm.sh/release-name: jenkinsmeta.helm.sh/release-namespace: default
Events:Type Reason Age From Message---- ------ ---- ---- -------Normal Sync 11s (x2 over 5m36s) nginx-ingress-controller Scheduled for sync
然后就可以欢快地访问 jenkins.k8s-t2.com了
3. 相关命令
(jenkins_url)/safeRestart – 允许所有正在运行的作业完成。 重新启动完成后，新作业将保留在队列中运行。
(jenkins_url)/restart – 强制重启，无需等待生成完成。
4. 遇到的问题
- 启动pod时出现
default-scheduler 0/3 nodes are available: 1 node(s) had untolerated taint {node-role.kubernetes.io/control-plane: }
查找安装的目标Server是否有污点 "node-role.kubernetes.io/control-plane"，有则去掉或修改
- 在jenkins安装插件时，出现:
unable to find valid certification path to requested target
解决办法：手动安装 skip-certificate-check，到这里下载skip-certificate-check | Jenkins plugin安装最新版本，目前是1.1
设置git仓库时，提示 stderr: No ECDSA host key is known for and you have requested strict checking.
解决办法： Manage Jenkins => Security => Git-Host-Key-Verification 修改为 No verificationssh - stderr: No ECDSA host key is known for github.com and you have requested strict checking - Ask Ubuntu
5. 参考
Kubernetes
kubernetes(十四) 基于kubernetes的jenkins持续集成-腾讯云开发者社区-腾讯云
Managing Plugins
https://medium.com/javarevisited/deploying-a-spring-boot-application-on-kubernetes-using-jenkins-672961425a42
