集群外是可以读取kubeconfig文件，集群内读不到这个文件，如果想从 Kubernetes 集群内运行的应用程序向 Kubernetes API 进行身份验证，怎么操作？
list查询pod
测试环境：Mac+minikube
源码参考：github源码地址
package main
import (
"context"
"fmt"
"time"
"k8s.io/apimachinery/pkg/api/errors"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/client-go/kubernetes"
"k8s.io/client-go/rest"
)
func main() {
// 创建 in-cluster config
// 这里的rest.InClusterConfig实际会调用client-go的服务账户信息,每创建pod的时候都会使用此认证
// https://github.com/kubernetes/kubernetes/blob/master/staging/src/k8s.io/client-go/rest/config.go#L512
// tokenFile = "/var/run/secrets/kubernetes.io/serviceaccount/token"
// rootCAFile = "/var/run/secrets/kubernetes.io/serviceaccount/ca.crt"
// 登录容器内可以查看到token和ca信息
// kubectl exec -it test-incluster-client-go bash
// root@test-incluster-client-go:/# ls /var/run/secrets/kubernetes.io/serviceaccount/
// ca.crt namespace token
config, err := rest.InClusterConfig()
if err != nil {
panic(err.Error())
}
// 创建 clientset
clientset, err := kubernetes.NewForConfig(config)
if err != nil {
panic(err.Error())
}
for {
//通过省略命名空间获取所有命名空间中的pod
//或者指定namespace来获取特定命名空间中的pod
pods, err := clientset.CoreV1().Pods("").List(context.TODO(), metav1.ListOptions{})
if err != nil {
panic(err.Error())
}
fmt.Printf("There are %d pods in the cluster\n", len(pods.Items))
//错误处理的例子:
// -使用helper函数，例如:errors.IsNotFound()
// -和/或转换为StatusError，并使用它的属性，如ErrStatus。消息
_, err = clientset.CoreV1().Pods("default").Get(context.TODO(), "test-node-local-dns", metav1.GetOptions{})
if errors.IsNotFound(err) {
fmt.Printf("Pod test-node-local-dns not found in default namespace\n")
} else if statusError, isStatus := err.(*errors.StatusError); isStatus {
fmt.Printf("Error getting pod %v\n", statusError.ErrStatus.Message)
} else if err != nil {
panic(err.Error())
} else {
fmt.Printf("Found test-node-local-dns pod in default namespace\n")
}
time.Sleep(10 * time.Second)
}
}
关于使用镜像运行的pod连接k8s集群时可以使用InClusterConfig,即使用令牌连接，github源码地址
func InClusterConfig() (*Config, error) {
const (
tokenFile = "/var/run/secrets/kubernetes.io/serviceaccount/token"
rootCAFile = "/var/run/secrets/kubernetes.io/serviceaccount/ca.crt"
)代码只保留这三个文件：
➜ client-go-incluster-k8s-api git:(master) ✗ ls
Dockerfile main.go pod.yaml
初始化项目：
➜ client-go-incluster-k8s-api git:(master) ✗ go mod init app
go: creating new go.mod: module app
go: to add module requirements and sums:
go mod tidy
➜ client-go-incluster-k8s-api git:(master) ✗ cat go.mod
module app
go 1.17
设置代理：
➜ client-go-incluster-k8s-api git:(master) ✗ export GOPROXY=https://goproxy.cn
自动获取依赖：
➜ client-go-incluster-k8s-api git:(master) ✗ go mod tidy
go: finding module for package k8s.io/client-go/rest
go: finding module for package k8s.io/apimachinery/pkg/api/errors
go: finding module for package k8s.io/client-go/kubernetes
go: finding module for package k8s.io/apimachinery/pkg/apis/meta/v1
go: found k8s.io/apimachinery/pkg/api/errors in k8s.io/apimachinery v0.23.5
go: found k8s.io/apimachinery/pkg/apis/meta/v1 in k8s.io/apimachinery v0.23.5
go: found k8s.io/client-go/kubernetes in k8s.io/client-go v0.23.5
go: found k8s.io/client-go/rest in k8s.io/client-go v0.23.5
➜ client-go-incluster-k8s-api git:(master) ✗ cat go.mod
module app
go 1.17
require (
k8s.io/apimachinery v0.23.5
k8s.io/client-go v0.23.5
)
require (
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/go-logr/logr v1.2.0 // indirect
github.com/gogo/protobuf v1.3.2 // indirect
github.com/golang/protobuf v1.5.2 // indirect
github.com/google/go-cmp v0.5.5 // indirect
github.com/google/gofuzz v1.1.0 // indirect
github.com/googleapis/gnostic v0.5.5 // indirect
github.com/json-iterator/go v1.1.12 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.2 // indirect
golang.org/x/net v0.0.0-20211209124913-491a49abca63 // indirect
golang.org/x/oauth2 v0.0.0-20210819190943-2bc19b11175f // indirect
golang.org/x/sys v0.0.0-20210831042530-f4d43177bf5e // indirect
golang.org/x/term v0.0.0-20210615171337-6886f2dfbf5b // indirect
golang.org/x/text v0.3.7 // indirect
golang.org/x/time v0.0.0-20210723032227-1f47c861a9ac // indirect
google.golang.org/appengine v1.6.7 // indirect
google.golang.org/protobuf v1.27.1 // indirect
gopkg.in/inf.v0 v0.9.1 // indirect
gopkg.in/yaml.v2 v2.4.0 // indirect
gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b // indirect
k8s.io/api v0.23.5 // indirect
k8s.io/klog/v2 v2.30.0 // indirect
k8s.io/kube-openapi v0.0.0-20211115234752-e816edb12b65 // indirect
k8s.io/utils v0.0.0-20211116205334-6203023598ed // indirect
sigs.k8s.io/json v0.0.0-20211020170558-c049b76a60c6 // indirect
sigs.k8s.io/structured-merge-diff/v4 v4.2.1 // indirect
sigs.k8s.io/yaml v1.2.0 // indirect
)
编译当前项目，注意：在mac上编译linux可以执行的二进制，采用交差编译，否则无法执行
➜ client-go-incluster-k8s-api git:(master) ✗ GOOS=linux GOARCH=amd64 go build -o ./app .
➜ client-go-incluster-k8s-api git:(master) ✗ ls app
app
然后使用提供的 Dockerfile 将其打包到一个 docker 镜像中，以便在 Kubernetes 上运行它
➜ client-go-incluster-k8s-api git:(master) ✗ docker build -t in-cluster .
[+] Building 441.7s (8/8) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 36B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker.io/library/debian:latest 3.7s
=> [auth] library/debian:pull token for registry-1.docker.io 0.0s
=> [internal] load build context 0.0s
=> => transferring context: 27B 0.0s
=> [1/2] FROM docker.io/library/debian@sha256:87eefc7c15610cca61db5c0fd280911c6a737c0680d807432c0bd80cd0cca39b 437.5s
=> => resolve docker.io/library/debian@sha256:87eefc7c15610cca61db5c0fd280911c6a737c0680d807432c0bd80cd0cca39b 0.0s
=> => sha256:87eefc7c15610cca61db5c0fd280911c6a737c0680d807432c0bd80cd0cca39b 1.85kB / 1.85kB 0.0s
=> => sha256:0040bafca14127bdc2dcb0d9897a26f5d44136e89c02046710de6ea01a227278 529B / 529B 0.0s
=> => sha256:d69c6cd3a20d21ec91b677c3bcd10d9975f4fe67eff81afb5a09bdef5134afeb 1.46kB / 1.46kB 0.0s
=> => sha256:dbba69284b2786013fe94fefe0c2e66a7d3cecbb20f6d691d71dac891ee37be5 54.94MB / 54.94MB 434.9s
=> => extracting sha256:dbba69284b2786013fe94fefe0c2e66a7d3cecbb20f6d691d71dac891ee37be5 2.3s
=> [2/2] COPY ./app /app 0.2s
=> exporting to image 0.2s
=> => exporting layers 0.2s
=> => writing image sha256:8118b96d6c8af0ec3ba87fdd54da38beb457283e7f58fe3043f0b343609da7d0 0.0s
=> => naming to docker.io/library/in-cluster 0.0s
Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix them
查看当前镜像：
➜ client-go-incluster-k8s-api git:(master) ✗ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
in-cluster latest 8118b96d6c8a 2 minutes ago 163MB
把mac上的in-cluster:latest镜像拷贝到minikube中，否则pod启动失败（因为minikube里面还没有这个镜像），登录minikube后确认已经成功
➜ client-go-incluster-k8s-api git:(master) ✗ minikube image load in-cluster:latest
➜ client-go-incluster-k8s-api git:(master) ✗ minikube ssh
_ _
_ _ ( ) ( )
___ ___ (_) ___ (_)| |/') _ _ | |_ __
/' _ ` _ `\| |/' _ `\| || , < ( ) ( )| '_`\ /'__`\
| ( ) ( ) || || ( ) || || |\`\ | (_) || |_) )( ___/
(_) (_) (_)(_)(_) (_)(_)(_) (_)`\___/'(_,__/'`\____)
$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
in-cluster latest 8118b96d6c8a 13 minutes ago 163MB
如果您在集群上启用了 RBAC，请使用以下代码段创建角色绑定，这将授予默认服务帐户查看权限
kubectl create clusterrolebinding default-view --clusterrole=view --serviceaccount=default:default
minikube dashboard，打开链接可以查看创建的Cluster Role Bindings：view
出现报错：error: no matches for kind "ClusterRoleBinding" in version "rbac.authorization.k8s.io/v1beta1"，排查可能是kubectl的版本（v1.16.2）和minikube中k8s版本不匹配，尝试升级kubectl，创建成功：
➜ client-go-incluster-k8s-api git:(master) ✗ kubectl create clusterrolebinding default-view --clusterrole=view --serviceaccount=default:default
error: no matches for kind "ClusterRoleBinding" in version "rbac.authorization.k8s.io/v1beta1"
➜ client-go-incluster-k8s-api git:(master) ✗ kubectl version
Client Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.2", GitCommit:"c97fe5036ef3df2967d086711e6c0c405941e14b", GitTreeState:"clean", BuildDate:"2019-10-15T23:41:55Z", GoVersion:"go1.12.10", Compiler:"gc", Platform:"darwin/amd64"}
➜ client-go-incluster-k8s-api git:(master) ✗ minikube version
minikube version: v1.25.2
commit: 362d5fdc0a3dbee389b3d3f1034e8023e72bd3a7
➜ client-go-incluster-k8s-api git:(master) ✗ brew upgrade kubectl
➜ client-go-incluster-k8s-api git:(master) ✗ kubectl create clusterrolebinding default-view --clusterrole=view --serviceaccount=default:default
clusterrolebinding.rbac.authorization.k8s.io/default-view created
create yaml配置文件：
➜ client-go-incluster-k8s-api git:(master) ✗ kubectl apply -f pod.yaml
pod/test-incluster-client-go configured
➜ client-go-incluster-k8s-api git:(master) ✗ cat pod.yaml
apiVersion: v1
kind: Pod
metadata:
name: test-incluster-client-go
spec:
containers:
- name: incluster
image: in-cluster:latest
imagePullPolicy: IfNotPresent
查看日志输出，有15个pod，删除一个deployment的2pod，还剩下13：
There are 15 pods in the cluster
Pod test-node-local-dns not found in default namespace
There are 15 pods in the cluster
Pod test-node-local-dns not found in default namespace
There are 15 pods in the cluster
Pod test-node-local-dns not found in default namespace
There are 15 pods in the cluster
Pod test-node-local-dns not found in default namespace
There are 13 pods in the cluster
Pod test-node-local-dns not found in default namespace
➜ client-go-incluster-k8s-api git:(master) ✗ kubectl get pod -A | grep -vi NAMESPACE
default nginx-deployment-9456bbbf9-78z87 1/1 Running 0 10d
default nginx-deployment-9456bbbf9-7tgnm 1/1 Running 0 10d
default nginx-deployment-9456bbbf9-n4mch 1/1 Running 0 10d
default test-incluster-client-go 1/1 Running 0 10m
kube-system coredns-64897985d-jq6tk 1/1 Running 1 (27d ago) 27d
kube-system etcd-minikube 1/1 Running 1 (27d ago) 27d
kube-system kube-apiserver-minikube 1/1 Running 1 (27d ago) 27d
kube-system kube-controller-manager-minikube 1/1 Running 1 (27d ago) 27d
kube-system kube-proxy-fbd4n 1/1 Running 1 (27d ago) 27d
kube-system kube-scheduler-minikube 1/1 Running 1 (27d ago) 27d
kube-system storage-provisioner 1/1 Running 2 (16d ago) 27d
kubernetes-dashboard dashboard-metrics-scraper-58549894f-bl7rz 1/1 Running 1 (27d ago) 27d
kubernetes-dashboard kubernetes-dashboard-ccd587f44-6nz6f 1/1 Running 2 (16d ago) 27d
test nginx-test-785479479f-bfl46 1/1 Running 0 2d6h
test nginx-test-785479479f-zrbfb 1/1 Running 0 2d6h
➜ client-go-incluster-k8s-api git:(master) ✗ kubectl get pod -A | grep -vi NAMESPACE | wc -l
15
➜ client-go-incluster-k8s-api git:(master) ✗ kubectl get deployments -A
NAMESPACE NAME READY UP-TO-DATE AVAILABLE AGE
default nginx-deployment 3/3 3 3 10d
kube-system coredns 1/1 1 1 27d
kubernetes-dashboard dashboard-metrics-scraper 1/1 1 1 27d
kubernetes-dashboard kubernetes-dashboard 1/1 1 1 27d
test nginx-test 2/2 2 2 2d6h
➜ client-go-incluster-k8s-api git:(master) ✗ kubectl delete deployments nginx-test -n test
deployment.apps "nginx-test" deleted
➜ client-go-incluster-k8s-api git:(master) ✗ kubectl get pod -A | grep -vi NAMESPACE
default nginx-deployment-9456bbbf9-78z87 1/1 Running 0 10d
default nginx-deployment-9456bbbf9-7tgnm 1/1 Running 0 10d
default nginx-deployment-9456bbbf9-n4mch 1/1 Running 0 10d
default test-incluster-client-go 1/1 Running 0 11m
kube-system coredns-64897985d-jq6tk 1/1 Running 1 (27d ago) 27d
kube-system etcd-minikube 1/1 Running 1 (27d ago) 27d
kube-system kube-apiserver-minikube 1/1 Running 1 (27d ago) 27d
kube-system kube-controller-manager-minikube 1/1 Running 1 (27d ago) 27d
kube-system kube-proxy-fbd4n 1/1 Running 1 (27d ago) 27d
kube-system kube-scheduler-minikube 1/1 Running 1 (27d ago) 27d
kube-system storage-provisioner 1/1 Running 2 (16d ago) 27d
kubernetes-dashboard dashboard-metrics-scraper-58549894f-bl7rz 1/1 Running 1 (27d ago) 27d
kubernetes-dashboard kubernetes-dashboard-ccd587f44-6nz6f 1/1 Running 2 (16d ago) 27d
➜ client-go-incluster-k8s-api git:(master) ✗ kubectl get pod -A | grep -vi NAMESPACE | wc -l
13
创建job pod list查询
测试环境：Mac+minikube
源码参考：github源码地址
初始化项目：
✗ ls
Dockerfile README.md main.go
✗ go mod init app
go: creating new go.mod: module app
go: to add module requirements and sums:
go mod tidy
✗ go mod tidy
go: finding module for package k8s.io/client-go/kubernetes
go: finding module for package k8s.io/apimachinery/pkg/apis/meta/v1
go: finding module for package k8s.io/client-go/rest
go: found k8s.io/apimachinery/pkg/apis/meta/v1 in k8s.io/apimachinery v0.23.5
go: found k8s.io/client-go/kubernetes in k8s.io/client-go v0.23.5
go: found k8s.io/client-go/rest in k8s.io/client-go v0.23.5
跨环境交差编译
✗ GOOS=linux GOARCH=amd64 go build -o ./listing .
✗ ls
Dockerfile README.md go.mod go.sum listing main.go
打包镜像
✗ cat Dockerfile
FROM centos
COPY ./listing /listing
ENTRYPOINT ["./listing"]
✗ docker build -t listing .
✗ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
listing latest 0cfd37e7083d 3 minutes ago 271MB
生成job
kubectl create job list --image=list --dry-run -o yaml > listing.yaml
//把creationTimestamp: null 这些无用的信息删除，添加：
✗ vim listing.yaml
apiVersion: batch/v1
kind: Job
metadata:
name: list
spec:
template:
metadata:
spec:
containers:
- image: listing:latest
imagePullPolicy: IfNotPresent
name: list
resources: {}
restartPolicy: Never
status: {}
✗ kubectl create -f listing.yaml
job.batch/listing created
报错处理：
✗ kubectl get pod
NAME READY STATUS RESTARTS AGE
listing-v4cmc 0/1 ErrImagePull 0 11s
//需要把镜像同步到minikube上
✗ kubectl describe pod listing-v4cmc
Normal Scheduled 18s default-scheduler Successfully assigned default/listing-v4cmc to minikube
Normal Pulling 18s kubelet Pulling image "listing:latest"
Warning Failed 14s kubelet Failed to pull image "listing:latest": rpc error: code = Unknown desc = Error response from daemon: pull access denied for listing, repository does not exist or may require 'docker login': denied: requested access to the resource is denied
Warning Failed 14s kubelet Error: ErrImagePull
Normal BackOff 13s kubelet Back-off pulling image "listing:latest"
Warning Failed 13s kubelet Error: ImagePullBackOff
✗ minikube image load listing:latest
➜ client-go-list-容器化k8s原生Go程序,完成集群内调用k8s-api git:(master) ✗ minikube ssh
_ _
_ _ ( ) ( )
___ ___ (_) ___ (_)| |/') _ _ | |_ __
/' _ ` _ `\| |/' _ `\| || , < ( ) ( )| '_`\ /'__`\
| ( ) ( ) || || ( ) || || |\`\ | (_) || |_) )( ___/
(_) (_) (_)(_)(_) (_)(_)(_) (_)`\___/'(_,__/'`\____)
$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
listing latest 0cfd37e7083d 9 minutes ago 271MB
job创建的listing pod已经Completed了，看看日志
//job创建的listing pod已经Completed了，看看日志
✗ kubectl get pod
NAME READY STATUS RESTARTS AGE
listing-v4cmc 0/1 Completed 0 3m22s
nginx-deployment-9456bbbf9-78z87 1/1 Running 0 12d
nginx-deployment-9456bbbf9-7tgnm 1/1 Running 0 12d
nginx-deployment-9456bbbf9-n4mch 1/1 Running 0 12d
test-incluster-client-go 1/1 Running 0 2d
➜ client-go-list-容器化k8s原生Go程序,完成集群内调用k8s-api git:(master) ✗ kubectl logs -f listing-v4cmc
获取default namespace下的pod
listing-v4cmc
nginx-deployment-9456bbbf9-78z87
nginx-deployment-9456bbbf9-7tgnm
nginx-deployment-9456bbbf9-n4mch
test-incluster-client-go
获取default namespace下的deployment的名字
nginx-deployment
获取kube-system namespace下的daemonset的名字
kube-proxy
获取kube-system svc
kube-dns
