gitlabl 如何生成runer token gitlab runner kubernetes

转载

mob64ca14048514 2024-04-17 09:32:12

文章标签 kubernetes gitlab docker git 命名空间 文章分类 运维

本文内容以语雀为准

资料

kubernetes

极狐 GitLab 中文文档

高级配置

极狐 GitLab 中文文档

添加额外主机别名

极狐 GitLab 中文文档

Pod 的 DNS 配置

极狐 GitLab 中文文档

GitLab Runner 的 Kubernetes 执行器

极狐 GitLab 中文文档

Docker 执行器

极狐 GitLab 中文文档

通过特权模式使用 Docker-in-Docker

极狐 GitLab 中文文档

说明

GitLab Runner 注册到 GitLab 的操作请参见上面章节中的(GitLab Runner 配置)[https://www.yuque.com/xuxiaowei-com-cn/gitlab-k8s/tutxiiri5t87m7oa]，只需要将流水线的执行器设置成kubernetes即可，然后执行流水线，会出现问题，按照下方内容去解决
本文采用遇见什么错误，增加对应的配置来介绍 GitLab Runner、kubernetes 的配置

配置

运行流水线，出现问题

Using Kubernetes namespace: default
ERROR: Preparation failed: getting Kubernetes config: 
invalid configuration: 
no configuration has been provided, 
try setting KUBERNETES_MASTER environment variable

原因：k8s地址未配置
修改文件

vim /etc/gitlab-runner/config.toml

修改对应流水线的配置内容如下

[runners.kubernetes]
    # k8s 地址
    host = "https://192.168.80.130:6443"

运行流水线，出现问题

ERROR: Job failed (system failure): 
prepare environment: setting up credentials: 
Post "https://192.168.61.139:6443/api/v1/namespaces/default/secrets": 
x509: certificate signed by unknown authority. 
Check https://docs.gitlab.com/runner/shells/index.html#shell-profile-loading for more information

原因：k8s证书未配置
修改文件

vim /etc/gitlab-runner/config.toml

修改对应流水线的配置内容如下

[runners.kubernetes]
    # k8s 证书
    ca_file = "/etc/kubernetes/pki/ca.crt"

运行流水线，出现问题

ERROR: Job failed (system failure): 
prepare environment: setting up credentials: secrets is forbidden: 
User "system:anonymous" cannot create resource "secrets" in API group "" in the namespace "default". 
Check https://docs.gitlab.com/runner/shells/index.html#shell-profile-loading for more information

原因：k8s账户未配置
修改文件

vim /etc/gitlab-runner/config.toml

修改对应流水线的配置内容如下

[runners.kubernetes]
    # service 账户配置
    # 设置 服务授权的名称
    service_account = "gitlab-runner"
    bearer_token = "先随便写一个"
    bearer_token_overwrite_allowed = true

运行流水线，出现问题

ERROR: Job failed (system failure): 
prepare environment: setting up credentials: Unauthorized. 
Check https://docs.gitlab.com/runner/shells/index.html#shell-profile-loading for more information

原因：k8s凭证不正确，需要：创建命名空间、创建角色、创建服务账户并授权命名空间、创建服务账户在命名空间的token
修改文件

vim /etc/gitlab-runner/config.toml

修改对应流水线的配置内容如下

# 创建命名空间
kubectl create namespace gitlab


# 创建角色 gitlab-runner 前，要求命名空间 gitlab 必须存在

vim role.yaml

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: gitlab-runner
  namespace: gitlab
rules:
  - apiGroups: ["*"]
    resources: ["pods"]
    verbs: ["list", "get", "watch", "create", "delete"]
  - apiGroups: ["*"]
    resources: ["pods/exec"]
    verbs: ["create"]
  - apiGroups: ["*"]
    resources: ["pods/log"]
    verbs: ["get"]
  - apiGroups: ["*"]
    resources: ["pods/attach"]
    verbs: ["list", "get", "create", "delete", "update"]
  - apiGroups: ["*"]
    resources: ["secrets"]
    verbs: ["list", "get", "create", "delete", "update"]      
  - apiGroups: ["*"]
    resources: ["configmaps"]
    verbs: ["list", "get", "create", "delete", "update"]

kubectl apply -f role.yaml

# 命名空间授权
kubectl create serviceaccount gitlab-runner -n gitlab

# 创建用户操作命名空间的Token，指定有效时间，单位是秒，315360000s代表10年
kubectl create token gitlab-runner -n gitlab --duration=315360000s

[runners.kubernetes]
    # service 账户配置
    # 设置 服务授权的名称
    service_account = "gitlab-runner"
    bearer_token = "填写上述生成的token"

运行流水线，出现问题

ERROR: Job failed (system failure): 
prepare environment: setting up credentials: 
secrets is forbidden: User "system:serviceaccount:gitlab:gitlab-runner" cannot create resource "secrets" in API group "" in the namespace "default". 
Check https://docs.gitlab.com/runner/shells/index.html#shell-profile-loading for more information

原因：要设置上述创建的命名空间
修改文件

vim /etc/gitlab-runner/config.toml

修改对应流水线的配置内容如下

[runners.kubernetes]
    namespace = "gitlab"

运行流水线，出现问题

ERROR: Job failed (system failure): 
prepare environment: setting up credentials: 
secrets is forbidden: User "system:serviceaccount:gitlab:gitlab-runner" cannot create resource "secrets" in API group "" in the namespace "gitlab". 
Check https://docs.gitlab.com/runner/shells/index.html#shell-profile-loading for more information

原因：创建角色绑定，将角色gitlab-runner、命名空间gitlab设置服务账户gitlab:gitlab-runner并命名为gitlab-runner

修改对应流水线的配置内容如下

kubectl create rolebinding gitlab-runner --namespace=gitlab --role=gitlab-runner --serviceaccount=gitlab:gitlab-runner

运行流水线，出现问题

WARNING: Failed to pull image with policy "": 
image pull failed: rpc error: 
code = Unknown 
desc = failed to pull and unpack image "registry.gitlab.com/gitlab-org/gitlab-runner/gitlab-runner-helper:x86_64-7178588d": 
failed to resolve reference "registry.gitlab.com/gitlab-org/gitlab-runner/gitlab-runner-helper:x86_64-7178588d": 
failed to do request: Head "https://registry.gitlab.com/v2/gitlab-org/gitlab-runner/gitlab-runner-helper/manifests/x86_64-7178588d": 
dial tcp 35.227.35.254:443: connect: connection refused
ERROR: Job failed: prepare environment: 
waiting for pod running: 
pulling image "registry.gitlab.com/gitlab-org/gitlab-runner/gitlab-runner-helper:x86_64-7178588d": 
image pull failed: rpc error: 
code = Unknown 
desc = failed to pull and unpack image "registry.gitlab.com/gitlab-org/gitlab-runner/gitlab-runner-helper:x86_64-7178588d": 
failed to resolve reference "registry.gitlab.com/gitlab-org/gitlab-runner/gitlab-runner-helper:x86_64-7178588d": 
failed to do request: Head "https://registry.gitlab.com/v2/gitlab-org/gitlab-runner/gitlab-runner-helper/manifests/x86_64-7178588d": 
dial tcp 35.227.35.254:443: connect: connection refused. 
Check https://docs.gitlab.com/runner/shells/index.html#shell-profile-loading for more information

原因：下载gitlab-runner-helper失败，需要手动上设置helper_image
修改文件

vim /etc/gitlab-runner/config.toml

修改对应流水线的配置内容如下

# 选择适合的gitlab-runner-helper版本

[runners.kubernetes]
    # helper_image="gitlab/gitlab-runner-helper:x86_64-${CI_RUNNER_REVISION}"
  	# 由于 gitlab 将 gitlab-runner-helper 发布到 hub.docker.com 的时间较慢，可以会用 bitnami/gitlab-runner-helper
  	# 也可以使用 xuxiaoweicomcn/gitlab-runner-helper：所有镜像均为 registry.gitlab.com/gitlab-org/gitlab-runner/gitlab-runner-helper 中拉取并上传的，未做任何修改
		# bitnami/gitlab-runner-helper:15.6.1
    helper_image = "gitlab/gitlab-runner-helper:x86_64-v14.10.2"

运行流水线，出现问题

ERROR: Job failed (system failure): 
prepare environment: waiting for pod running: 
timed out waiting for pod to start. 
Check https://docs.gitlab.com/runner/shells/index.html#shell-profile-loading for more information

原因：创建 pod 时需要 helper_image，但是拉取超时，可手动拉取 helper_image；拉取流水线所用的镜像超时，可手动拉取

# 执行过程可使用 kubectl -n gitlab describe pod pod的名称，查看状态，pod的名称可在流水线中看到
ctr -n=k8s.io image pull docker.io/gitlab/gitlab-runner-helper:x86_64-v14.10.2
# ctr -n=k8s.io image pull docker.io/bitnami/gitlab-runner-helper:15.6.1
# 也可以使用 xuxiaoweicomcn/gitlab-runner-helper：所有镜像均为 registry.gitlab.com/gitlab-org/gitlab-runner/gitlab-runner-helper 中拉取并上传的，未做任何修改

# 假如流水线使用的镜像是 node:16.0.0
ctr -n=k8s.io image pull docker.io/node:16.0.0

ctr -n=k8s.io image list

如需在 GitLab Runner 中使用执行器 kubernetes 构建 Docker 镜像，需要配置下列内容，并且主机的 docker.socket 处于运行状态（并设置开机自启）

流水线设置参见：https://gitlab.com/xuxiaowei-com-cn/dragonwell8

systemctl start docker.socket
systemctl enable docker.socket

[[runners]]
  ...
  [runners.kubernetes]
    ...
    [runners.kubernetes.volumes]
    [[runners.kubernetes.volumes.host_path]]
      name = "docker"
      mount_path = "/var/run/docker.sock"
      host_path = "/var/run/docker.sock"

问题

如果克隆镜像时无法解析 GitLab 的域名，可以在 GitLab Runner 中自定义域名的IP（其他自定义域名同理）

vim /etc/gitlab-runner/config.toml

[[runners]]
  ...
  [runners.kubernetes]
    [[runners.kubernetes.host_aliases]]
      # 自定义 GitLab 的 IP
			ip = "192.168.80.14"
      hostnames = ["gitlab.example.com"]
    [[runners.kubernetes.host_aliases]]
			# 自定义 Docker host 的 IP
      ip = "192.168.80.33"
      hostnames = ["host.docker.example.xuxiaowei.cloud"]
    [[runners.kubernetes.host_aliases]]
		  # 自定义 Docker 私库的 IP
      ip = "192.168.80.45"
      hostnames = ["registry.docker.example.xuxiaowei.cloud"]

本文章为转载内容，我们尊重原作者对文章享有的著作权。如有内容错误或侵权问题，欢迎原作者联系我们进行内容更正或删除文章。