helm模板部署java应用并添加Prometheus监控和ELK日志监控

转载

jiecho 2025-01-08 14:54:29

文章标签 GitOps ArgoCD Helm OpenShift git 文章分类 Java 后端开发

《OpenShift 4.x HOL教程汇总》
说明：本文已经在OpenShift 4.8环境中验证

文章目录

运行环境
用Helm创建样例Chart
根据Helm Chart安装OpenShift应用
从Helm Chat导出要部署的应用对象
将应用资源配置文件推送至Github的Repo
根据Github的配置创建OpenShift的应用资源
自动调整OpenShift的配置，以保持和Github中的配置同步
将Github中的新版配置同步更新至OpenShift

运行环境

安装ArgoCD服务器环境。
根据《OpenShift 4 - 使用Helm部署OpenShift应用》安装helm客户端即可。

用Helm创建样例Chart

创建名为myapp的helm chart，然后可以查看chart中的deployment.yaml。

$ mkdir helmstuff
$ cd helmstuff/
$ helm create myapp
Creating myapp
$ tree myapp/
myapp/
├── charts
├── Chart.yaml
├── templates
│   ├── deployment.yaml
│   ├── _helpers.tpl
│   ├── hpa.yaml
│   ├── ingress.yaml
│   ├── NOTES.txt
│   ├── serviceaccount.yaml
│   ├── service.yaml
│   └── tests
│       └── test-connection.yaml
└── values.yaml

根据Helm Chart安装OpenShift应用

创建OpenShift的helmstuff项目，然后通过helm安装应用。

$ oc new-project helmstuff
$ helm install adventure1 myapp/ -n helmstuff
NAME: adventure1	
LAST DEPLOYED: Sat Dec 26 08:13:17 2020
NAMESPACE: helmstuff
STATUS: deployed
REVISION: 1
NOTES:
1. Get the application URL by running these commands:
  export POD_NAME=$(kubectl get pods --namespace helmstuff -l "app.kubernetes.io/name=myapp,app.kubernetes.io/instance=adventure1" -o jsonpath="{.items[0].metadata.name}")
  echo "Visit http://127.0.0.1:8080 to use your application"
  kubectl --namespace helmstuff port-forward $POD_NAME 8080:80

查看Helm列表。

$ helm list -n helmstuff
NAME            NAMESPACE       REVISION        UPDATED                                 STATUS          CHART           APP VERSION
adventure1      helmstuff       1               2020-12-26 08:13:17.653701005 +0000 UTC deployed        myapp-0.1.0     1.16.0

查看pod状态为CrashLoopBackOff，然后helm中的adventure1，确认结果报错“Error: pod adventure1-myapp-test-connection failed”。

$ oc get pod -n helmstuff
NAME                                READY   STATUS             RESTARTS   AGE
adventure1-myapp-5b64cf64cb-r65fk   0/1     CrashLoopBackOff   1          32s
 
$ helm test adventure1 -n helmstuff
NAME: adventure1
LAST DEPLOYED: Sat Oct  2 08:36:28 2021
NAMESPACE: helmstuff
STATUS: deployed
REVISION: 1
TEST SUITE:     adventure1-myapp-test-connection
Last Started:   Sat Oct  2 08:39:18 2021
Last Completed: Sat Oct  2 08:41:41 2021
Phase:          Failed
NOTES:
1. Get the application URL by running these commands:
  export POD_NAME=$(kubectl get pods --namespace helmstuff -l "app.kubernetes.io/name=myapp,app.kubernetes.io/instance=adventure1" -o jsonpath="{.items[0].metadata.name}")
  export CONTAINER_PORT=$(kubectl get pod --namespace helmstuff $POD_NAME -o jsonpath="{.spec.containers[0].ports[0].containerPort}")
  echo "Visit http://127.0.0.1:8080 to use your application"
  kubectl --namespace helmstuff port-forward $POD_NAME 8080:$CONTAINER_PORT
Error: pod adventure1-myapp-test-connection failed

上面错误是由于标准的nginx容器是运行在80端口的，这在OpenShift默认是不被允许的。需要执行以下命令为名为“adventure1-myapp”的运行容器的 ServiceAccount 系统用户提权（将执行“anyuid”的权限赋给“helmstuff”项目中的“adventure1-myapp”系统用户）。

$ oc get sa adventure1-myapp -n helmstuff
NAME               SECRETS   AGE
adventure1-myapp   2         6m34s
 
$ oc adm policy add-scc-to-user anyuid system:serviceaccount:helmstuff:adventure1-myapp
clusterrole.rbac.authorization.k8s.io/system:openshift:scc:anyuid added: "adventure1-myapp"

先从helm中删除adventure1，然后重新创建helm的adventure1，最后再用helm测试adventure1，确认这次可测通过。

$ helm uninstall adventure1 -n helmstuff
release "adventure1" uninstalled
 
$ helm install adventure1 myapp/ -n helmstuff
NAME: adventure1
LAST DEPLOYED: Sat Oct  2 08:47:30 2021
NAMESPACE: helmstuff
STATUS: deployed
REVISION: 1
NOTES:
1. Get the application URL by running these commands:
  export POD_NAME=$(kubectl get pods --namespace helmstuff -l "app.kubernetes.io/name=myapp,app.kubernetes.io/instance=adventure1" -o jsonpath="{.items[0].metadata.name}")
  export CONTAINER_PORT=$(kubectl get pod --namespace helmstuff $POD_NAME -o jsonpath="{.spec.containers[0].ports[0].containerPort}")
  echo "Visit http://127.0.0.1:8080 to use your application"
  kubectl --namespace helmstuff port-forward $POD_NAME 8080:$CONTAINER_PORT
 
$ helm test adventure1
NAME: adventure1
LAST DEPLOYED: Sat Oct  2 08:47:30 2021
NAMESPACE: helmstuff
STATUS: deployed
REVISION: 1
TEST SUITE:     adventure1-myapp-test-connection
Last Started:   Sat Oct  2 08:48:44 2021
Last Completed: Sat Oct  2 08:48:51 2021
Phase:          Succeeded
NOTES:
1. Get the application URL by running these commands:
  export POD_NAME=$(kubectl get pods --namespace helmstuff -l "app.kubernetes.io/name=myapp,app.kubernetes.io/instance=adventure1" -o jsonpath="{.items[0].metadata.name}")
  export CONTAINER_PORT=$(kubectl get pod --namespace helmstuff $POD_NAME -o jsonpath="{.spec.containers[0].ports[0].containerPort}")
  echo "Visit http://127.0.0.1:8080 to use your application"
  kubectl --namespace helmstuff port-forward $POD_NAME 8080:$CONTAINER_PORT

此时应用已经部署好。

$ oc get deployment -n helmstuff
NAME               READY   UP-TO-DATE   AVAILABLE   AGE
adventure1-myapp   1/1     1            1           17m

从Helm Chat导出要部署的应用对象

创建manifest目录，让后将helm中名为adventure1的manifest导出到manifest/adventure1.yaml。

$ mkdir manifest
$ helm get manifest adventure1 > manifest/adventure1.yaml

查看导出的manifest/adventure1.yaml文件内容。

$ cat manifest/adventure1.yaml
---
# Source: myapp/templates/serviceaccount.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: adventure1-myapp
  labels:
    helm.sh/chart: myapp-0.1.0
    app.kubernetes.io/name: myapp
    app.kubernetes.io/instance: adventure1
    app.kubernetes.io/version: "1.16.0"
    app.kubernetes.io/managed-by: Helm
---
# Source: myapp/templates/service.yaml
apiVersion: v1
kind: Service
metadata:
  name: adventure1-myapp
  labels:
    helm.sh/chart: myapp-0.1.0
    app.kubernetes.io/name: myapp
    app.kubernetes.io/instance: adventure1
    app.kubernetes.io/version: "1.16.0"
    app.kubernetes.io/managed-by: Helm
spec:
  type: ClusterIP
  ports:
    - port: 80
      targetPort: http
      protocol: TCP
      name: http
  selector:
    app.kubernetes.io/name: myapp
    app.kubernetes.io/instance: adventure1
---
# Source: myapp/templates/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: adventure1-myapp
  labels:
    helm.sh/chart: myapp-0.1.0
    app.kubernetes.io/name: myapp
    app.kubernetes.io/instance: adventure1
    app.kubernetes.io/version: "1.16.0"
    app.kubernetes.io/managed-by: Helm
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: myapp
      app.kubernetes.io/instance: adventure1
  template:
    metadata:
      labels:
        app.kubernetes.io/name: myapp
        app.kubernetes.io/instance: adventure1
    spec:
      serviceAccountName: adventure1-myapp
      securityContext:
        {}
      containers:
        - name: myapp
          securityContext:
            {}
          image: "nginx:1.16.0"
          imagePullPolicy: IfNotPresent
          ports:
            - name: http
              containerPort: 80
              protocol: TCP
          livenessProbe:
            httpGet:
              path: /
              port: http
          readinessProbe:
            httpGet:
              path: /
              port: http
          resources:
            {}

将应用资源配置文件推送至Github的Repo

$ ls
manifest myapp

在你的Github账号上创建名为“gitops-helm-argocd”的Repository。
依次执行以下命令，将myapp应用资源推送的自己的Github账户中。

$ git init
Initialized empty Git repository in /home/xiaoyliu-redhat.com/helmstuff/.git/
$ git add *
$ git commit -m "initial commit of helm chart and working manifest"
[master (root-commit) c03cd60] initial commit of helm chart and working manifest
 Committer: GTPE Student <xiaoyliu-redhat.com@clientvm.beijing-b510.internal>
Your name and email address were configured automatically based
on your username and hostname. Please check that they are accurate.
You can suppress this message by setting them explicitly:
  
    git config --global user.name "Your Name"
    git config --global user.email you@example.com
 
After doing this, you may fix the identity used for this commit with:
  
    git commit --amend --reset-author
   
 11 files changed, 416 insertions(+)
 create mode 100644 manifest/adventure1.yaml
 create mode 100644 myapp/.helmignore
 create mode 100644 myapp/Chart.yaml
 create mode 100644 myapp/templates/NOTES.txt
 create mode 100644 myapp/templates/_helpers.tpl
 create mode 100644 myapp/templates/deployment.yaml
 create mode 100644 myapp/templates/ingress.yaml
 create mode 100644 myapp/templates/service.yaml
 create mode 100644 myapp/templates/serviceaccount.yaml
 create mode 100644 myapp/templates/tests/test-connection.yaml
 create mode 100644 myapp/values.yaml
 
$ git remote add origin https://github.com/YOUR-GITHUB/gitops-helm-argocd.git
$ git push -u origin master
Username for 'https://github.com': liuxiaoyu-git
Password for 'https://liuxiaoyu-git@github.com':
Counting objects: 17, done.
Delta compression using up to 2 threads.
Compressing objects: 100% (15/15), done.
Writing objects: 100% (17/17), 5.30 KiB | 0 bytes/s, done.
Total 17 (delta 0), reused 0 (delta 0)
To https://github.com/liuxiaoyu-git/gitops-helm-argocd.git
 * [new branch]      master -> master
Branch master set up to track remote branch master from origin.

确认Github中名为“gitops-helm-argocd”的Repository中已经有“myapp”和“** manifest**”了。

根据Github的配置创建OpenShift的应用资源

将Github资源加到ArgoCD中的Repo。

$ argocd repo add https://github.com/YOUR-GITHUB/gitops-helm-argocd.git --name gitops-helm-argocd
repository 'https://github.com/liuxiaoyu-git/gitops-helm-argocd.git' added
$ argocd repo list
TYPE  NAME  REPO                                                     INSECURE  LFS    CREDS  STATUS      MESSAGE
git         https://github.com/liuxiaoyu-git/gitops-helm-argocd.git  false     false  false  Successful

创建部署应用的项目，并增加“argocd.argoproj.io/managed-by=openshift-gitops”标签。

$ TARGET=adventure1-myapp
$ oc new-project ${TARGET}
$ oc label namespace ${TARGET} argocd.argoproj.io/managed-by=openshift-gitops

新建一个名为adventure1的ArgoCD应用，用它在github中的配置资源与OpenShift中的helmstuff项目建立关联。

$ argocd app create --name adventure1 --project default \
    --repo https://github.com/liuxiaoyu-git/gitops-helm-argocd.git --revision master --path manifest \
    --dest-server https://kubernetes.default.svc --dest-namespace ${TARGET} \
    --sync-policy automated --self-heal

同理，也需要为

$ oc adm policy add-scc-to-user anyuid system:serviceaccount:${TARGET}:adventure1-myapp

进入ArgoCD的控制台，查看adventure1应用。确认当前OpenShift的用资源和Github Repo中的资源是“Synced”的。

自动调整OpenShift的配置，以保持和Github中的配置同步

根据名为adventure1-myapp的OpenShift Service对象手动生成一个新的对象：adventure1-mybad。

$ oc get svc adventure1-myapp -n ${TARGET} -o json \
	| jq 'del(.spec.clusterIP)' \
	| jq 'del(.spec.clusterIPs)' \
	| sed "s/\"name\": \"adventure1-myapp\"/\"name\": \"adventure1-mybad\"/g" \
	| oc create -n ${TARGET} -f -
service/adventure1-mybad created
$ oc get svc -n ${TARGET}
NAME               TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)   AGE
adventure1-myapp   ClusterIP   172.30.12.170    <none>        80/TCP    3h38m
adventure1-mybad   ClusterIP   172.30.225.179   <none>        80/TCP    10s

此时可在ArgoCD控制台中的adventure1应用界面中看到已经是“OutOfSync”状态，且在名为adventure1-mybad的Service下方显示了黄色标记。
修改ArgoCD中adventure1应用的配置（“auto-prune”用来自动删除比ArgoCD多余的对象），然后通过ArgoCD控制台确认名为adventure1-mybad的Service已经被删除，而此时adventure1也转为Synced状态。

$ argocd app set adventure1 --sync-policy automated --auto-prune --self-heal
$ oc get svc -n ${TARGET}
NAME               TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
adventure1-myapp   ClusterIP   172.30.12.170   <none>        80/TCP    3h59m

helm模板部署java应用并添加Prometheus监控和ELK日志监控_Helm

4. 删除本项目中OpenShift的Deployment对象，确认ArgoCD会自动根据Github的配置重新创建一个新的Deployment对象。

$ oc delete deployment adventure1-myapp -n ${TARGET} && oc get pods -n ${TARGET} -w
deployment.extensions "adventure1-myapp" deleted
NAME                                READY   STATUS              RESTARTS   AGE
adventure1-myapp-5b64cf64cb-l9p5g   1/1     Terminating         1          3h47m
adventure1-myapp-5b64cf64cb-l9p5g   1/1     Terminating         1          3h47m
adventure1-myapp-5b64cf64cb-l9p5g   0/1     Terminating         1          3h47m
adventure1-myapp-5b64cf64cb-l9p5g   0/1     Terminating         1          3h47m
adventure1-myapp-5b64cf64cb-2kf2n   0/1     ContainerCreating   0          10s
adventure1-myapp-5b64cf64cb-2kf2n   0/1     ContainerCreating   0          18s
adventure1-myapp-5b64cf64cb-2kf2n   0/1     Running             0          18s
adventure1-myapp-5b64cf64cb-2kf2n   1/1     Running             0          19s

将Github中的新版配置同步更新至OpenShift

先设置ArogCD，关闭Github和OpenShift自动同步配置的功能。

$ argocd app set adventure1 --sync-policy none

修改现有Helm Chart，将version从“0.1.0”改为“0.1.1”，将appVersion从“1.16.0”改为“1.16.1”.

$ sed -i 's/1.16.0/1.16.1/g' myapp/Chart.yaml
$ sed -i 's/0.1.0/0.1.1/g' myapp/Chart.yaml
$ cat myapp/Chart.yaml
apiVersion: v2
name: myapp
description: A Helm chart for Kubernetes
 
# A chart can be either an 'application' or a 'library' chart.
#
# Application charts are a collection of templates that can be packaged into versioned archives
# to be deployed.
#
# Library charts provide useful utilities or functions for the chart developer. They're included as
# a dependency of application charts to inject those utilities and functions into the rendering
# pipeline. Library charts do not define any templates and therefore cannot be deployed.
type: application
  
# This is the chart version. This version number should be incremented each time you make changes
# to the chart and its templates, including the app version.
version: 0.1.1
 
# This is the version number of the application being deployed. This version number should be
# incremented each time you make changes to the application.
appVersion: 1.16.1

更新Helm中的adventure1配置，并查看改配置修改前后的变化。

$ helm list -n helmstuff
NAME            NAMESPACE       REVISION        UPDATED                                 STATUS          CHART           APP VERSION
adventure1      helmstuff       1               2020-12-26 08:18:17.200103853 +0000 UTC deployed        myapp-0.1.0     1.16.0
 
$ helm upgrade adventure1 myapp/ -n helmstuff
Release "adventure1" has been upgraded. Happy Helming!
NAME: adventure1
LAST DEPLOYED: Sat Dec 26 09:03:56 2020
NAMESPACE: helmstuff
STATUS: deployed
REVISION: 2
NOTES:
1. Get the application URL by running these commands:
  export POD_NAME=$(kubectl get pods --namespace helmstuff -l "app.kubernetes.io/name=myapp,app.kubernetes.io/instance=adventure1" -o jsonpath="{.items[0].metadata.name}")
  export CONTAINER_PORT=$(kubectl get pod --namespace helmstuff $POD_NAME -o jsonpath="{.spec.containers[0].ports[0].containerPort}")
  echo "Visit http://127.0.0.1:8080 to use your application"
  kubectl --namespace helmstuff port-forward $POD_NAME 8080:$CONTAINER_PORT

$ helm list -n helmstuff
NAME            NAMESPACE       REVISION        UPDATED                                 STATUS          CHART           APP VERSION
adventure1      helmstuff       2               2020-12-26 09:03:56.028781132 +0000 UTC deployed        myapp-0.1.1     1.16.1

用helm重新生成manifest/adventure1.yaml文件，然后可确认对象标签已经变为“myapp-0.1.1”和“1.16.1”。

$ helm get manifest adventure1 -n helmstuff > manifest/adventure1.yaml
$ more manifest/adventure1.yaml
# Source: myapp/templates/serviceaccount.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: adventure1-myapp
  labels:
    helm.sh/chart: myapp-0.1.1
    app.kubernetes.io/name: myapp
    app.kubernetes.io/instance: adventure1
    app.kubernetes.io/version: "1.16.1"
    app.kubernetes.io/managed-by: Helm

比较新旧版adventure1.yaml文件后，将本地变化的配置文件提交到Github。提交后可在Github上确认“manifest/adventure1.yaml”已经更新。

$ git diff manifest/adventure1.yaml
$ git add *
$ git commit -m "updated app version and manifests"
[master b844df1] updated app version and manifests
 Committer: GTPE Student <xiaoyliu-redhat.com@clientvm.beijing-b510.internal>
Your name and email address were configured automatically based
on your username and hostname. Please check that they are accurate.
You can suppress this message by setting them explicitly:
 
    git config --global user.name "Your Name"
    git config --global user.email you@example.com
 
After doing this, you may fix the identity used for this commit with:
 
    git commit --amend --reset-author
 
 2 files changed, 9 insertions(+), 9 deletions(-)
 
$ git push origin master
Username for 'https://github.com': liuxiaoyu-git
Password for 'https://liuxiaoyu-git@github.com':
Counting objects: 11, done.
Delta compression using up to 2 threads.
Compressing objects: 100% (5/5), done.
Writing objects: 100% (6/6), 534 bytes | 0 bytes/s, done.
Total 6 (delta 3), reused 0 (delta 0)
remote: Resolving deltas: 100% (3/3), completed with 3 local objects.
To https://github.com/liuxiaoyu-git/gitops-helm-argocd.git
   c03cd60..b844df1  master -> master

通过命令和控制台查看ArgoCD的adventure1应用的同步状态，发现此时对于变化的配置，OpenShift和Github是没有同步的。

$ argocd app get adventure1
Name:               adventure1
Project:            default
Server:             https://kubernetes.default.svc
Namespace:          helmstuff
URL:                https://argocd-server-argocd.apps.cluster-pek-99bc.pek-99bc.sandbox309.opentlc.com/applications/adventure1
Repo:               https://github.com/liuxiaoyu-git/gitops-helm-argocd.git
Target:             master
Path:               manifest
SyncWindow:         Sync Allowed
Sync Policy:        <none>
Sync Status:        OutOfSync from master (f294628)
Health Status:      Healthy

GROUP  KIND            NAMESPACE  NAME                              STATUS     HEALTH   HOOK  MESSAGE
apps   Deployment      helmstuff  adventure1-myapp                  OutOfSync  Healthy        deployment.apps/adventure1-myapp created
       Pod             helmstuff  adventure1-myapp-test-connection             Healthy
       Service         helmstuff  adventure1-myapp                  OutOfSync  Healthy
       ServiceAccount  helmstuff  adventure1-myapp                  OutOfSync

helm模板部署java应用并添加Prometheus监控和ELK日志监控_ArgoCD_02

在ArdoCD控制台中查看名为adventure1-myapp的Service的详细配置，其中在DIFF中显示了这个服务在OpenShift和Github的配置差异。
此时再次打开ArgoCD的同步选项。然后在ArgoCD控制台中确认adventure1应用的同步状态应已经变为“Synced”。

$ argocd app set adventure1 --sync-policy automated --auto-prune --self-heal

helm模板部署java应用并添加Prometheus监控和ELK日志监控_Helm_03

本文章为转载内容，我们尊重原作者对文章享有的著作权。如有内容错误或侵权问题，欢迎原作者联系我们进行内容更正或删除文章。

上一篇：网页接收render_templater传值

下一篇：java poi在word里生成饼状图

提问和评论都可以，用心的回复会被更多人看到评论

发布评论

相关文章

官方博客	全部文章	热门标签	班级博客
了解我们	网站地图	意见反馈

鸿蒙开发者社区	51CTO学堂
51CTO	软考资讯

helm模板部署java应用并添加Prometheus监控和ELK日志监控

helm模板部署java应用并添加Prometheus监控和ELK日志监控

文章目录

运行环境

用Helm创建样例Chart

根据Helm Chart安装OpenShift应用

从Helm Chat导出要部署的应用对象

将应用资源配置文件推送至Github的Repo

根据Github的配置创建OpenShift的应用资源

自动调整OpenShift的配置，以保持和Github中的配置同步

将Github中的新版配置同步更新至OpenShift

51CTO博客