在k8s+jenkins+github+dockerhub环境中用pipeline部署应用
更新:HHH   时间:2023-1-7


用k8s+jenkins+github+dockerhub部署应用:
jenkins有两种方式实现CI/CD
1、自由风格,通过shell代码实现
2、pipeline流水线,通过jenkins代码或者jenkinsfile文件实现
Pipeline 有两种创建方法:
1、可以直接在 Jenkins 的 Web UI 界面中输入jenkins代码;
2、也可以通过创建一个 Jenkinsfile 脚本文件放入项目源码库中

构建一个新项目步骤:
1、在github上新建一个项目
2、在本地将项目克隆下来
3、编写代码,dockerfile,jenkinsfile,部署该应用的yaml文件
4、将项目代码push到github上
5、使用jenkins实现CI/CD
示例:
1、在github上新建一个项目nginx-demo(略)
2、克隆代码到本地192.168.1.244
$ git clone https://github.com/dongyali521521/nginx-demo
除了第一次clone,以后每次clone都要先删除原项目再clone
$ rm -rf nginx-demo
3、修改代码
$ cd nginx-demo/
$ vim index.html
4、将修改后的代码push到github
$ git add .
$ git commit -m "2222"
$ git push #需要输入github的用户名和密码
5、在jenkins中新建任务实现CI/CD

一般情况下,除了应用程序代码之外,dockerfile,jenkinsfile和部署该应用的yaml文件都放在源码中一起管理。
一个项目内至少包括如下文件:都需要在实现CI/CD前准备好
$ cd nginx-demo/
$ ls
Dockerfile Jenkinsfile xxx.yaml index.html README.md

在k8s中使用jenkins实现CI/CD(pipeline方式)又包括如下步骤:
1、从git hub把项目克隆到jenkins slave pod的工作目录/home/jenkins/agent/workspace/pipeline-name
2、测试
3、使用Dockerfile构建镜像
4、把构建的镜像push到docker hub上
5、修改yaml文件中的镜像名为刚才构建的镜像
6、在k8s集群中部署应用
以上步骤的代码可以在jenkins master的任务配置界面输入,也可以写入Jenkinsfile文件在项目源码库中管理
前4步属于CI阶段,最后两步属于CD阶段

Jenkins Pipeline 有几个核心概念:
Node:节点,一个 Node 就是一个 Jenkins 节点,Master 或者 slave
Stage:阶段,一个 Pipeline 可以划分为若干个 Stage,每个 Stage 代表一组操作
Step:步骤,Step 是最基本的操作单元,可以是打印一句话,也可以是构建一个 Docker 镜像

用jenkins代码模拟CI/CD步骤
新建任务----名字----流水线
在流水线脚本中输入如下内容----保存立即构建

node('dongyali-jnlp') {
    stage('Clone') {
      echo "1.Clone Stage"
    }
    stage('Test') {
      echo "2.Test Stage"
    }
    stage('Build') {
      echo "3.Build Docker Image Stage"
    }
    stage('Push') {
      echo "4.Push Docker Image Stage"
    }
    stage('YAML') {
      echo "5. Change YAML File Stage"
    }
    stage('Deploy') {
      echo "6. Deploy Stage"
    }
}

假设项目nginx-demo已经开发完毕,并上传到了git hub上
文件包括:
Dockerkfile k8s.yaml index.html README.md
k8s.yaml是部署该应用的文件
$ cd nginx-demo/
$ cat Dockerfile

FROM docker.io/nginx
MAINTAINER dongyali
ADD index.html /usr/share/nginx/html/
EXPOSE 80
CMD ["nginx", "-g","daemon off;"]

$ cat k8s.yaml

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: nginx-demo
  namespace: default
spec:
  replicas: 3
  template:
    metadata:
      labels:
        app: nginx-demo
    spec:
      containers:
      - image: dongyali521521/nginx-demo:<BUILD_TAG>
        imagePullPolicy: IfNotPresent
        name: nginx-demo
        ports:
        - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: nginx-demo
  namespace: default
spec:
  selector:
    app: nginx-demo
  type: NodePort
  ports:
  - protocol: TCP
    port: 80
    targetPort: 80
    name: myapp-http

如果不指定namespace,默认会把应用部署到k8s集群的kube-ops空间,因为jenkins slave pod运行在kube-ops空间内。

下面开始在jenkins中部署上述项目:
因为每次git commit之后都要构建镜像重新部署应用,所以采用git commit的记录作为镜像的 tag
为了安全,把构建的镜像push到docker hub上时,需要输入加密的用户名和密码
凭据----Stores scoped to Jenkin下面的Jenkins----全局凭据----添加凭据:
用户名:docker hub的用户名
密码:输入docker hub的用户密码
ID:dockerHub,可以随便写,后面要用
在CD阶段开始之前需要人工干预,以便于部署到用户选择的环境
http://192.168.1.243:30003
新建任务----名字----流水线
在流水线脚本中输入如下内容----保存立即构建

node('dongyali-jnlp') {
    stage('Clone') {
        echo "1.Clone Stage"
        git url: "https://github.com/dongyali521521/nginx-demo.git"
        script {
            build_tag = sh(returnStdout: true, script: 'git rev-parse --short HEAD').trim()
        }
    }
    stage('Test') {
      echo "2.Test Stage"
    }
    stage('Build') {
        echo "3.Build Docker Image Stage"
        sh "docker build -t dongyali521521/nginx-demo:${build_tag} ."
    }
    stage('Push') {
        echo "4.Push Docker Image Stage"
        withCredentials([usernamePassword(credentialsId: 'dockerHub', passwordVariable: 'dockerHubPassword', usernameVariable: 'dockerHubUser')]) {
            sh "docker login -u ${dockerHubUser} -p ${dockerHubPassword}"
            sh "docker push dongyali521521/nginx-demo:${build_tag}"
        }
    }
    stage('Deploy') {
        echo "5. Deploy Stage"
        def userInput = input(
            id: 'userInput',
            message: 'Choose a deploy environment',
            parameters: [
                [
                    $class: 'ChoiceParameterDefinition',
                    choices: "Dev\nQA\nProd",
                    name: 'Env'
                ]
            ]
        )
        echo "This is a deploy step to ${userInput}"
        sh "sed -i 's/<BUILD_TAG>/${build_tag}/' k8s.yaml"
        if (userInput == "Dev") {
            // deploy dev stuff
        } else if (userInput == "QA"){
            // deploy qa stuff
        } else {
            // deploy prod stuff
        }
        sh "kubectl apply -f k8s.yaml --record"
    }
}

每次构建,都会启动一个新的jenkins slave pod
查看部署情况:
$ kubectl get pod
nginx-demo-65f6f679c6-5ljvd 1/1 Running 0 34m
nginx-demo-65f6f679c6-p4g6m 1/1 Running 0 34m
nginx-demo-65f6f679c6-xr7mx 1/1 Running 0 34m
$ kubectl get svc
nginx-demo NodePort 10.103.13.141 <none> 80:30185/TCP 18h
http://http://192.168.1.243:30185/

下面再使用jenkinsfile文件来部署应用
在项目目录nginx-demo下增加一个Jenkinsfile文件,并上传到git hub上
$ cd nginx-demo/
$ cat Jenkinsfile

node('dongyali-jnlp') {
    stage('Prepare') {
        echo "1.Prepare Stage"
        checkout scm
        script {
            build_tag = sh(returnStdout: true, script: 'git rev-parse --short HEAD').trim()
        }
    }
    stage('Test') {
      echo "2.Test Stage"
    }
    stage('Build') {
        echo "3.Build Docker Image Stage"
        sh "docker build -t dongyali521521/nginx-demo:${build_tag} ."
    }
    stage('Push') {
        echo "4.Push Docker Image Stage"
        withCredentials([usernamePassword(credentialsId: 'dockerHub', passwordVariable: 'dockerHubPassword', usernameVariable: 'dockerHubUser')]) {
            sh "docker login -u ${dockerHubUser} -p ${dockerHubPassword}"
            sh "docker push dongyali521521/nginx-demo:${build_tag}"
        }
    }
    stage('Deploy') {
        echo "5. Deploy Stage"
        def userInput = input(
            id: 'userInput',
            message: 'Choose a deploy environment',
            parameters: [
                [
                    $class: 'ChoiceParameterDefinition',
                    choices: "Dev\nQA\nProd",
                    name: 'Env'
                ]
            ]
        )
        echo "This is a deploy step to ${userInput}"
        sh "sed -i 's/<BUILD_TAG>/${build_tag}/' k8s.yaml"
        if (userInput == "Dev") {
            // deploy dev stuff
        } else if (userInput == "QA"){
            // deploy qa stuff
        } else {
            // deploy prod stuff
        }
        sh "kubectl apply -f k8s.yaml --record"
    }
}

由于Jenkinsfile和其他代码都在git hub源码库中,所以第一步就不需要再clone了,改为checkout scm

然后在jenkins配置任务界面选择:
流水线----pipeline脚本from SCM
SCM:git
Repository URL:https://github.com/dongyali521521/nginx-demo
Credentials:无
保存构建即可。

再测试一个项目
$ git clone https://github.com/dongyali521521/jenkins-demo.git
$ cd jenkins-demo
$ ls
Dockerfile k8s.yaml main.go README.md
$ cat Dockerfile

FROM golang:1.8.0-alpine

ADD . /go/src/app

WORKDIR /go/src/app

RUN GOOS=linux GOARCH=386 go build -v -o /go/src/app/jenkins-app

CMD ["./jenkins-app"]

$ cat main.go

package main

// Import the fmt for formatting strings
// Import os so we can read environment variables from the system
import (
    "fmt"
    "os"
)

func main() {
    fmt.Println("Hello, Kubernetes!I'm from Jenkins CI!4444")
    fmt.Println("BRANCH_NAME:", os.Getenv("branch"))
}

$ cat k8s.yaml

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: jenkins-demo
  namespace: default
spec:
  template:
    metadata:
      labels:
        app: jenkins-demo
    spec:
      containers:
      - image: dongyali521521/jenkins-demo:<BUILD_TAG>
        imagePullPolicy: IfNotPresent
        name: jenkins-demo
        env:
        - name: branch
          value: <BRANCH_NAME>

$ git add .
$ git commit -m "4444"
$ git push
新建任务----名字----流水线
在流水线脚本中输入如下内容----保存立即构建

node('dongyali-jnlp') {
    stage('Clone') {
        echo "1.Clone Stage"
        git url: "https://github.com/dongyali521521/jenkins-demo.git"
        script {
            build_tag = sh(returnStdout: true, script: 'git rev-parse --short HEAD').trim()
        }
    }
    stage('Test') {
      echo "2.Test Stage"
    }
    stage('Build') {
        echo "3.Build Docker Image Stage"
        sh "docker build -t dongyali521521/jenkins-demo:${build_tag} ."
    }
    stage('Push') {
        echo "4.Push Docker Image Stage"
        withCredentials([usernamePassword(credentialsId: 'dockerHub', passwordVariable: 'dockerHubPassword', usernameVariable: 'dockerHubUser')]) {
            sh "docker login -u ${dockerHubUser} -p ${dockerHubPassword}"
            sh "docker push dongyali521521/jenkins-demo:${build_tag}"
        }
    }
    stage('Deploy') {
        echo "5. Deploy Stage"
        def userInput = input(
            id: 'userInput',
            message: 'Choose a deploy environment',
            parameters: [
                [
                    $class: 'ChoiceParameterDefinition',
                    choices: "Dev\nQA\nProd",
                    name: 'Env'
                ]
            ]
        )
        echo "This is a deploy step to ${userInput}"
        sh "sed -i 's/<BUILD_TAG>/${build_tag}/' k8s.yaml"
        if (userInput == "Dev") {
            // deploy dev stuff
        } else if (userInput == "QA"){
            // deploy qa stuff
        } else {
            // deploy prod stuff
        }
        sh "kubectl apply -f k8s.yaml --record"
    }
}

$ kubectl get pod
jenkins-demo-67db45cf6c-mjdl4 0/1 CrashLoopBackOff 3 65s
$ kubectl logs jenkins-demo-67db45cf6c-mjdl4
Hello, Kubernetes!I'm from Jenkins CI!4444
BRANCH_NAME:

返回云计算教程...