cicd 07--构建前后端一体化自动化发布流程
1 介绍
cicd 04–构建自动化发布流程 介绍了一种通用的自动化发布流程，其适用于单个前端或者后端的服务，若某个服务同时包括前后端，且前后端代码分离，那么改方法就不适用了。为此需要设计一种同时包括前后端的自动化发布流程，其思路是分别先后clone 前端、后端代码，然后对前端编译|预处理后将其和后端一起打包为线上可用的镜像，最后发布到集群中。本文介绍 构建前后端一体化自动化发布流程， 贴在此处以便于自己和有需要的小伙伴查阅学习，后续会在此处更新相关优化方法和注意事项。
2 构建过程
2.1 功能说明
主要功能模块：
清理空间(Clean workspace)
下载代码(Clone frontend )
下载后端代码(Clone backend )
同步代码(Scp to sre-cicd)
build前端(generate frontend)
同步前端到后端(copy frontend to backend)
build镜像(Build docker images)
打tag(Set tag)
检查deploy(check deploy exists)
发布到k8s(deploy to k8s)
告警通知(Declarative: Post Actions)
2.2 jenkins & gitlab 配置
jenkins 基础配置：
GitLab Repository Name: 填写gitlab 的url
Build Trigger->
gitlab 配置：
分别在前后端 repo 中配置jenkins 的webhook：
在 settings->integrations 中新加Webhooks，
URL： 填写jenkins 中build Triggers的url
Secret Token：填写jenkins中生成的 Secret token
Trigger： Merge request events（可按需增加其它方式）
涉及的参数说明：
cluster = 'yourCluster'
namespace = 'sre-test'
deployment = 'xg-nginx-prod'
container_name = 'xg-nginx'
docker_repo = 'yourDockerRepoNamespace/yourDockerImageName'
branch = '${gitlabTargetBranch}'
repo_url_frontend = "git@yourGitlabUrl/test_flask_frontend.git"
repo_url_backend = "git@yourGitlabUrl/test_flask_backend.git"
work_dir_frontend = 'gitlab_test_flask_two_repo_frontend'
work_dir_backend = 'gitlab_test_flask_two_repo_backend'
build_front_docker = 'your_docker_image_repo/repo_name:ubuntu_20.04'
涉及的节点:
SRE_BUILD_NODE： 用于build镜像，并将其推送到阿里云镜像仓库
slave： 该节点用于clone git代码，并将其rsync到 docker build机器上
SRE_NODE： 用于执行kubectl或helm操作
jenkinsfile 配置：
cluster = 'yourCluster'
namespace = 'sre-test'
deployment = 'xg-nginx'
container_name = 'xg-nginx'
docker_repo = 'yourDockerRepoNamespace/yourDockerImageName'
print("branch=${gitlabTargetBranch}")
branch = '${gitlabTargetBranch}'
repo_url_frontend = "git@yourGitlabUrl/test_flask_frontend.git"
repo_url_backend = "gityourGitlabUrl/test_flask_backend.git"
work_dir_frontend = 'gitlab_test_flask_two_repo_frontend'
work_dir_backend = 'gitlab_test_flask_two_repo_backend'
build_front_docker = 'your_docker_image_repo/repo_name:ubuntu_20.04'
def get_tag_dir() {
echo "${gitlabSourceRepoURL}"
if ("${gitlabSourceRepoURL}" == "${repo_url_frontend}") {
return "${work_dir_frontend}"
}
return "${work_dir_backend}"
}
tag_dir = get_tag_dir() // tag_dir is work_dir_frontend or work_dir_backend according to the trigger repo
def date = new Date()
image_version = date.format('yy.w.yyMMddHHmm')
default_description = "${namespace}/${deployment}:${image_version}"
currentBuild.description = "${default_description}"
pipeline {
agent {
node {
label 'SRE_BUILD_NODE'
}
}
environment {
para = "para_just_for_test"
}
stages {
stage('Clean workspace') {
agent {
node {
label 'SRE-Build01-Server'
}
}
steps {
sh """
echo 'clean workspace'
rm -fr /data/nas-sre-prod/jenkins/apps/${work_dir_frontend}
rm -fr /data/nas-sre-prod/jenkins/apps/${work_dir_backend}
"""
}
}
stage("Clone frontend"){
agent {
node {
label 'slave'
}
}
steps {
sh """
echo "Clone frontend repo: ${repo_url_frontend} ${branch}"
pwd
"""
dir("${work_dir_frontend}"){
deleteDir()
git(
url: "${repo_url_frontend}",
credentialsId: '13*yourGitCredentialsId*14',
branch: "${branch}"
)
}
sh """
pwd
ls
"""
}
}
stage("Clone backend"){
agent {
node {
label 'slave'
}
}
steps {
sh """
echo "Clone frontend repo: ${repo_url_frontend} ${branch}"
pwd
"""
dir("${work_dir_backend}"){
deleteDir()
git(
url: "${repo_url_backend}",
credentialsId: '13*yourGitCredentialsId*14',
branch: "${branch}"
)
}
sh """
pwd
ls
"""
}
}
stage('Scp to sre-cicd') {
agent {
node {
label 'slave'
}
}
steps {
sh """
pwd
echo "rsync frontend"
rsync -Lrap ${work_dir_frontend} --exclude ${work_dir_frontend}/.git 106.yourSre-cicdMachine.148:/data/nas-sre-prod/jenkins/apps/
if [ \$? -ne 0 ]
then
echo "Found some error when copy the frontend repo"
exit 1
fi
echo "rsync backend"
rsync -Lrap ${work_dir_backend} --exclude ${work_dir_backend}/.git 106.yourSre-cicdMachine.148:/data/nas-sre-prod/jenkins/apps/
if [ \$? -ne 0 ]
then
echo "Found some error when copy the backend repo"
exit 1
fi
"""
}
}
stage('generate frontend') {
agent {
node {
label 'SRE_BUILD_NODE'
}
}
steps {
sh """
echo 'generate frontend files'
pwd
ls
docker rm -f ${work_dir_frontend} || pwd
docker run --name=${work_dir_frontend} -v /data/nas-sre-prod/jenkins/apps/${work_dir_frontend}:/home/ ${build_front_docker} bash /home/build.sh
docker rm -f ${work_dir_frontend} || pwd
ls
"""
}
}
stage('copy frontend to backend') {
agent {
node {
label 'SRE_BUILD_NODE'
}
}
steps {
sh """
echo 'copy frontend to backend'
pwd
ls
cp /data/nas-sre-prod/jenkins/apps/${work_dir_frontend}/src/get_item.html /data/nas-sre-prod/jenkins/apps/${work_dir_backend}/src/templates/
"""
}
}
stage('Build docker images') {
agent {
node {
label 'SRE_BUILD_NODE'
}
}
steps {
sh """
echo 'build docker image'
pwd
ls
cd /data/nas-sre-prod/jenkins/apps/${work_dir_backend}/
if [ ! -f Dockerfile ]
then
echo "No available dockerfile in workspace"
fi
if [ ! -f build_docker.sh ]
then
echo "No available build_docker.sh in workspace"
fi
bash build_docker.sh ${docker_repo} ${image_version}
pwd
ls
"""
}
}
stage('Set tag') {
agent {
node {
label 'slave'
}
}
steps {
dir("${tag_dir}"){
sh """
pwd
ls
echo "${branch} => ${image_version}"
git tag -m "Build docker image ${image_version} for ${branch}/${image_version}" ${branch}/${image_version} && git describe && git push origin ${branch}/${image_version}
ls
"""
}
}
}
stage('check deploy exists') {
agent {
node {
label 'SRE_NODE'
}
}
steps {
echo 'Hello, check deployment ${cluster} ${namespace}/${deployment} exists'
sh "/usr/local/bin/kubectl --kubeconfig /home/yourHome/.kube/config-${cluster} -n ${namespace} get deploy ${deployment}"
}
}
stage('deploy to k8s') {
agent {
node {
label 'SRE_NODE'
}
}
steps {
echo 'deploy to k8s'
sh """
/usr/local/bin/kubectl --kubeconfig /home/yourHome/.kube/config-${cluster} -n ${namespace} set image deployment/${deployment} ${container_name}=your_docker_image_repo/${docker_repo}:${image_version}
"""
}
}
}
post {
always {
echo 'I have finished'
}
success {
echo "all in one, succeed!"
sh """
curl -X POST -H "Content-Type: application/json" -d '{"msg_type":"text","content":{"text":"gitlab_test_flask_two_repo notify, ${namespace}/${deployment}:${image_version}, succeed!"}}' https://open.feishu.cn/open-apis/bot/v2/hook/46xxx-xxx-xxx05
"""
}
failure {
echo "all in one, failed!"
sh """
curl -X POST -H "Content-Type: application/json" -d '{"msg_type":"text","content":{"text":"gitlab_test_flask_two_repo notify, ${namespace}/${deployment}:${image_version}, failed!"}}' https://open.feishu.cn/open-apis/bot/v2/hook/46xxx-xxx-xxx05
"""
}
}
}
2.3 测试结果
前端修改代码:
后端代码修改:
jenkins 执行结果：
k8s 镜像效果：
3 注意事项
- 前后端一体化的cicd流程，可能由前端触发，也可能有后端触发，因此需要判断是哪个触发的，并在对应的git repo 中打上tag。
- 实际项目中，可能不同用户想通过不同分支触发并发布到不同的环境中，此时可以主动获取gitlab 触发分支，不同分支配置不同的命名空间即可。
- 本案例中的场景需要build 前端生成核心文件，此处使用ubuntu:20.04作为演示案例，实际中可能为node:13|14|15 等镜像。
4 说明
软件环境：
Docker: 20.10.*
K8s: 1.12
jenkins 版本:2.299
参考文档：
jenkins 官方文档
