ApiServer认证,认证一共有三种方式:
1)Https双向认证,是双向认证啊,不是单向认证(最安全)。
2)Http Token认证
3)Http Base认证,用户名和密码
我们之前介绍http方式是没有任何认证措施的,也就是说只要能访问master的主机都可以与其进行通信。特别说明:kubectl命令行工具既同时支持CA双向认证也支持简单认证(http base或者token)两种模式与apiserver进行通信,但其他组件只能配置成一种模式。
下面开始进行各类证书的生成以及kubeconfig文件的生成。
证书生成,可采用openssl,也可以采用CFSSL工具。下面这篇博客,采用的是CFSSL工具:
我比较熟悉openssl,因此介绍openssl使用方式
一、 生成各类证书
0)环境配置
[root@localhost ~]# mkdir kube-ca
[root@localhost kube-ca]#
[root@localhost kube-ca]# mkdir -p ./{certs,private,newcerts}
[root@localhost kube-ca]# touch ./index.txt
[root@localhost kube-ca]# echo 01 > ./serial
[root@localhost kube-ca]#
修改openssl配置文件,主要是扩展x509,设置多ip。[root@localhost kube-ca]# vi /etc/pki/tls/openssl.cnf
[ CA_default ]
#dir = /etc/pki/CA # Where everything is kept
dir = /etc/kubernetes/kube-ca # 重新指定目录
certs = $dir/certs # Where the issued certs are kept
crl_dir = $dir/crl # Where the issued crl are kept
database = $dir/index.txt # database index file.
#unique_subject = no # Set to 'no' to allow creation of
# several ctificates with same subject.
new_certs_dir = $dir/newcerts # default place for new certs.
certificate = $dir/cacert.pem # The CA certificate
serial = $dir/serial # The current serial number
crlnumber = $dir/crlnumber # the current crl number
# must be commented out to leave a V1 CRL
crl = $dir/crl.pem # The current CRL
private_key = $dir/private/cakey.pem # The private key
RANDFILE = $dir/private/.rand # private random number file
x509_extensions = usr_cert # 扩展x509_extension是usr_cert项
[usr_cer]
subjectAltName = @alt_names
#扩展多个IP
[alt_names]
IP.1 = 127.0.0.1
IP.2 = 192.168.1.105
IP.3 = 192.63.63.1
IP.4 = 192.63.63.20
1)生成CA根证书生成https私钥
[root@localhost kube-ca]#
[root@localhost kube-ca]# openssl genrsa -out private/ca.key 2048
Generating RSA private key, 2048 bit long modulus
..................................+++
.......................................................................+++
e is 65537 (0x10001)
[root@localhost kube-ca]#
生成https证书[root@localhost kube-ca]# openssl req -new -x509 -key private/ca.key -out certs/ca.crt
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [XX]:CN
State or Province Name (full name) []:BeiJing
Locality Name (eg, city) [Default City]:BeiJing
Organization Name (eg, company) [Default Company Ltd]:
Organizational Unit Name (eg, section) []:
Common Name (eg, your name or your server's hostname) []:mykubeca.io
Email Address []:abcd@abc.com
[root@localhost kube-ca]#
其中Common Name 是随意指定mykubeca.io。2)生成apiserver证书
生成服务端私钥
[root@localhost kube-ca]# mkdir apiserver
[root@localhost kube-ca]# openssl genrsa -out apiserver/apiserver.key 2048
Generating RSA private key, 2048 bit long modulus
...............+++
................+++
e is 65537 (0x10001)
[root@localhost kube-ca]#
生成服务端https证书,其中CommonName可以和CA中的CommonName不同,一般情况下,CommonName为服务的域名(也可以是ip,hostname)。[root@localhost kube-ca]# openssl req -new -key apiserver/apiserver.key -out apiserver/apiserver.csr
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [XX]:CN
State or Province Name (full name) []:BeiJing
Locality Name (eg, city) [Default City]:BeiJing
Organization Name (eg, company) [Default Company Ltd]:
Organizational Unit Name (eg, section) []:
Common Name (eg, your name or your server's hostname) []:mykube.io
Email Address []:abc@abc.com
Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:
An optional company name []:
[root@localhost kube-ca]#
[root@localhost kube-ca]#
CA证书进行签名,这一步非常重要,一定要执行。[root@localhost kube-ca]# openssl ca -in apiserver/apiserver.csr -keyfile ./private/ca.key -cert ./certs/ca.crt -out apiserver/apiserver.crt
Using configuration from /etc/pki/tls/openssl.cnf
Check that the request matches the signature
Signature ok
Certificate Details:
Serial Number: 1 (0x1)
Validity
Not Before: Mar 4 05:04:56 2018 GMT
Not After : Mar 4 05:04:56 2019 GMT
Subject:
countryName = CN
stateOrProvinceName = BeiJing
organizationName = Default Company Ltd
commonName = mykube.io
emailAddress = abc@abc.com
X509v3 extensions:
X509v3 Basic Constraints:
CA:FALSE
Netscape Comment:
OpenSSL Generated Certificate
X509v3 Subject Key Identifier:
FC:5C:32:61:B3:A1:0C:F8:94:FE:D0:C1:4C:56:D2:C6:39:61:00:B5
X509v3 Authority Key Identifier:
keyid:9A:FA:EE:26:A6:59:D6:F8:01:52:2C:15:17:63:A6:85:8F:88:DE:11
X509v3 Subject Alternative Name:
IP Address:127.0.0.1, IP Address:192.168.1.105, IP Address:192.169.122.215, IP Address:192.169.122.1
Certificate is to be certified until Mar 4 05:04:56 2019 GMT (365 days)
Sign the certificate? [y/n]:y
1 out of 1 certificate requests certified, commit? [y/n]y
Write out database with 1 new entries
Data Base Updated
[root@localhost kube-ca]#
3)生成客户端证书 客户端的证书生成和服务端类似,只需要保证客户端的CommonName与服务端的一致即可。当然也是需要进行CA签名,否则后面会报错的。此处不再描述生成过程。
1.2 部署证书以及kubeconfig文件
1.2.1 修改master节点
1)拷贝根证书、服务端秘钥和证书
[root@localhost kube-ca]# mkdir /etc/kubernetes/ca
[root@localhost kube-ca]# cp certs/ca.crt apiserver/apiserver.key apiserver/apiserver.crt /etc/kubernetes/ca
[root@localhost kube-ca]#
2)修改/etc/kubernetes/apiserver配置文件,在KUBE_API_ARGS中增加如下配置:--client-ca-file=/etc/kubernetes/ca/ca.crt --tls-private-key-file=/etc/kubernetes/ca/apiserver.key --tls-cert-file=/etc/kubernetes/ca/apiserver.crt
重启apiserver,apiserver默认监听端口是6443端口,通过curl进行校验:[root@localhost kube-ca] curl https://192.63.63.1:6443/api/v1/nodes --cert /etc/kubernetes/kube-ca/client/client.crt --key /etc/kubernetes/kube-ca/client/client.key --cacert /etc/kubernetes/kube-ca/certs/ca.crt -v
如果能正常显示数据,则认为证书配置成功,否则证书配置失败。1.2.2 修改node节点
1)Node节点,一般运行kubelet、kube-proxy两个组件,为了方便二者使用同一份客户端证书。将ca根证书,client私钥和证书拷贝到node2中/etc/kubernets/ca目录中。
2)创建kebeconfig文件,依次执行一下命令,会生成两个文件kubelet.kubeconfig和kube-proxy.kubeconfig
export KUBE_APISERVER="https://192.63.63.1:6443"
kubectl config set-cluster kubernetes \
--certificate-authority=/etc/kubernetes/ca/ca.crt \
--server=${KUBE_APISERVER} \
--kubeconfig=kubelet.kubeconfig
kubectl config set-credentials kubelet \
--client-certificate=/etc/kubernetes/ca/client.crt \
--client-key=/etc/kubernetes/ca/client.key \
--kubeconfig=kubelet.kubeconfig
kubectl config set-context default \
--cluster=kubernetes \
--user=kubelet \
--kubeconfig=kubelet.kubeconfig
kubectl config use-context default --kubeconfig=kubelet.kubeconfig
kubectl config set-cluster kubernetes \
--certificate-authority=/etc/kubernetes/ca/ca.crt \
--server=${KUBE_APISERVER} \
--kubeconfig=kube-proxy.kubeconfig
kubectl config set-credentials kube-proxy \
--client-certificate=/etc/kubernetes/ca/client.crt \
--client-key=/etc/kubernetes/ca/client.key \
--kubeconfig=kube-proxy.kubeconfig
kubectl config set-context default \
--cluster=kubernetes \
--user=kube-proxy \
--kubeconfig=kube-proxy.kubeconfig
kubectl config use-context default --kubeconfig=kube-proxy.kubeconfig
将文件kubele.kubeconfig拷贝到/var/lib/kubelet以及文件kube-proxy.kubeconfig拷贝到/var/lib/kube-proxy中,如果没有上述目录则创建。3)修改配置文件,
修改/etc/kubernetes/kubelet文件,在KUBELET_ARGS中增加--kubeconfig=/var/lib/kubelet/kubelet.kubeconfig
修改修改/etc/kubernetes/proxy文件,在KUBELET_ARGS中增加--kubeconfig=/var/lib/kubelet/kube-proxy.kubeconfig
重新启动kubelet和kube-proxy服务,然后在master节点中,查看nodes信息:
[root@localhost ~]#
[root@localhost ~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
127.0.0.1 Ready <none> 17d v1.9.1
node1 Ready <none> 5m v1.9.1
node2 Ready <none> 1m v1.9.1
[root@localhost ~]#
下面是kubelet.kubeconfig和kube-proxy.kubeconfig文件内容如下:[root@localhost kube-proxy]# cat /var/lib/kubelet/kubelet.kubeconfig
apiVersion: v1
clusters:
- cluster:
certificate-authority: /etc/kubernetes/ca/ca.crt
server: https://192.169.122.1:6443
name: kubernetes
contexts:
- context:
cluster: kubernetes
user: kubelet
name: default
current-context: default
kind: Config
preferences: {}
users:
- name: kubelet
user:
as-user-extra: {}
client-certificate: /etc/kubernetes/ca/client.crt
client-key: /etc/kubernetes/ca/client.key
[root@localhost kube-proxy]#
[root@localhost kube-proxy]# cat /var/lib/kube-proxy/kube-proxy.kubeconfig
apiVersion: v1
clusters:
- cluster:
certificate-authority: /etc/kubernetes/ca/ca.crt
server: https://192.169.122.1:6443
name: kubernetes
contexts:
- context:
cluster: kubernetes
user: kube-proxy
name: default
current-context: default
kind: Config
preferences: {}
users:
- name: kube-proxy
user:
as-user-extra: {}
client-certificate: /etc/kubernetes/ca/client.crt
client-key: /etc/kubernetes/ca/client.key
[root@localhost kube-proxy]#
二、遇到问题问题1:
[root@localhost controller]# curl https://127.0.0.1:443/api/v1/nodes --cert /etc/kubernetes/kube-ca/client/client.crt --key /etc/kubernetes/kube-ca/client/client.key --cacert /etc/kubernetes/kube-ca/certs/ca.crt -v
* About to connect() to 127.0.0.1 port 443 (#0)
* Trying 127.0.0.1...
* Connected to 127.0.0.1 (127.0.0.1) port 443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* CAfile: /etc/kubernetes/kube-ca/certs/ca.crt
CApath: none
* NSS error -12190 (SSL_ERROR_PROTOCOL_VERSION_ALERT)
* Peer reports incompatible or unsupported protocol version.
* Closing connection 0curl: (35) Peer reports incompatible or unsupported protocol version.
解决方法:
更新以下软件:
yum update nss nss-util nspryum update curl
问题2:之前通过kubectl get nodes提示无法找到master之类错误(具体是啥错误不清楚了)
解决方式1:kubectl get nodes --kubeconfig=XXX 指定kubeconfig文件,可以参考kubelet的文件
解决方式2:kubectl默认读取~/.kube/config,这个config文件里面有设置server地址。其实这个config文件就是kubeconfig文件。
至此,https方式访问介绍完成,后面介绍token和basic方式。
---------------------