Ingress 对象

Ingress 介绍

在Kuberbetes中除了使用NodePort，LoadBalancer, Port Proxy（hostPort）等实现外部访问入口之外，还可以使用Kubernetes 自带的Ingress来实现服务的负载均衡和策略路由的工作，其本质相当于一个Nginx代理服务器，可以对请求（http）实现精细的控制。

Ingress 是可以利用 Nginx、Haproxy 等负载均衡器暴露集群内服务的工具，

使用Ingress可以配置一个用于外部访问的URL地址，同时实现流量负载，SSL，基于名称的虚拟主机（例如Nginx的upstream）等。用户通过发送POST请求到API server,来申请Ingress资源， Ingress控制器负责完成Ingress转发工作，通常会使用负载均衡，也可以配置边缘路由器或其他前端以帮助以HA方式处理流量。

在当前版本中,Ingress处于测试版本，在使用Ingress时，需要先创建Ingress Contronller.

创建Ingree Controller

参考链接

在Kubernetes中，Ingress Controller将以Pod的形式运行，监控apiserver的 ingress接口后端的backend services,如果service发生变化，则Ingress Controller 应自动更新其转发规则。 Ingress Controller 需要实现基于不同HTTP URL向后转发的负载均衡分发规则，并可灵活设置7层的负载分发策略。如果公有云服务商能提供该类型的HTTP路由LoadBlanacer，也可以设置其为Ingress Controller。

在新的版本中Ingress Controller和具有负载均衡的软件已经捆绑到一起，目前支持多种类型， 如nginx ,haproxy 等。

创建Ingree Contoller和backend

使用如下的定义的文件，可以直接从官方下载，需要修改谷歌的镜像地址（https://kubernetes.github.io/ingress-nginx/deploy/）：

---

apiVersion: v1
kind: Namespace
metadata:
  name: ingress-nginx
---

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: default-http-backend
  labels:
    app: default-http-backend
  namespace: ingress-nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app: default-http-backend
  template:
    metadata:
      labels:
        app: default-http-backend
    spec:
      terminationGracePeriodSeconds: 60
      containers:
      - name: default-http-backend
        # Any image is permissible as long as:
        # 1. It serves a 404 page at /
        # 2. It serves 200 on a /healthz endpoint
        image: mirrorgooglecontainers/defaultbackend:1.4
        livenessProbe:
          httpGet:
            path: /healthz
            port: 8080
            scheme: HTTP
          initialDelaySeconds: 30
          timeoutSeconds: 5
        ports:
        - containerPort: 8080
        resources:
          limits:
            cpu: 10m
            memory: 20Mi
          requests:
            cpu: 10m
            memory: 20Mi
---

apiVersion: v1
kind: Service
metadata:
  name: default-http-backend
  namespace: ingress-nginx
  labels:
    app: default-http-backend
spec:
  ports:
  - port: 80
    targetPort: 8080
  selector:
    app: default-http-backend
---

kind: ConfigMap
apiVersion: v1
metadata:
  name: nginx-configuration
  namespace: ingress-nginx
  labels:
    app: ingress-nginx
---

kind: ConfigMap
apiVersion: v1
metadata:
  name: tcp-services
  namespace: ingress-nginx
---

kind: ConfigMap
apiVersion: v1
metadata:
  name: udp-services
  namespace: ingress-nginx
---

apiVersion: v1
kind: ServiceAccount
metadata:
  name: nginx-ingress-serviceaccount
  namespace: ingress-nginx

---

apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
  name: nginx-ingress-clusterrole
rules:
  - apiGroups:
      - ""
    resources:
      - configmaps
      - endpoints
      - nodes
      - pods
      - secrets
    verbs:
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - nodes
    verbs:
      - get
  - apiGroups:
      - ""
    resources:
      - services
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - "extensions"
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - ""
    resources:
        - events
    verbs:
        - create
        - patch
  - apiGroups:
      - "extensions"
    resources:
      - ingresses/status
    verbs:
      - update

---

apiVersion: rbac.authorization.k8s.io/v1beta1
kind: Role
metadata:
  name: nginx-ingress-role
  namespace: ingress-nginx
rules:
  - apiGroups:
      - ""
    resources:
      - configmaps
      - pods
      - secrets
      - namespaces
    verbs:
      - get
  - apiGroups:
      - ""
    resources:
      - configmaps
    resourceNames:
      # Defaults to "<election-id>-<ingress-class>"
      # Here: "<ingress-controller-leader>-<nginx>"
      # This has to be adapted if you change either parameter
      # when launching the nginx-ingress-controller.
      - "ingress-controller-leader-nginx"
    verbs:
      - get
      - update
  - apiGroups:
      - ""
    resources:
      - configmaps
    verbs:
      - create
  - apiGroups:
      - ""
    resources:
      - endpoints
    verbs:
      - get

---

apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
  name: nginx-ingress-role-nisa-binding
  namespace: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: nginx-ingress-role
subjects:
  - kind: ServiceAccount
    name: nginx-ingress-serviceaccount
    namespace: ingress-nginx

---

apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: nginx-ingress-clusterrole-nisa-binding
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: nginx-ingress-clusterrole
subjects:
  - kind: ServiceAccount
    name: nginx-ingress-serviceaccount
    namespace: ingress-nginx
---

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: nginx-ingress-controller
  namespace: ingress-nginx 
spec:
  replicas: 1
  selector:
    matchLabels:
      app: ingress-nginx
  template:
    metadata:
      labels:
        app: ingress-nginx
      annotations:
        prometheus.io/port: '10254'
        prometheus.io/scrape: 'true'
    spec:
      serviceAccountName: nginx-ingress-serviceaccount
      containers:
        - name: nginx-ingress-controller
          image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.16.2
          args:
            - /nginx-ingress-controller
            - --default-backend-service=$(POD_NAMESPACE)/default-http-backend
            - --configmap=$(POD_NAMESPACE)/nginx-configuration
            - --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services
            - --udp-services-configmap=$(POD_NAMESPACE)/udp-services
            - --publish-service=$(POD_NAMESPACE)/ingress-nginx
            - --annotations-prefix=nginx.ingress.kubernetes.io
    #       - --report-node-internal-ip-address=true
          securityContext:
            capabilities:
                drop:
                - ALL
                add:
                - NET_BIND_SERVICE
            # www-data -> 33
            runAsUser: 33
          env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
          ports:
          - name: http
            containerPort: 80
#            hostPort: 80       如果不使用NodePort，可以使用hostPort的方式直接映射到宿主端口
          - name: https
            containerPort: 443
#            hostPort: 443
          livenessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            initialDelaySeconds: 10
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 1
          readinessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 1

使用如下文件创建 Ingress的services:

apiVersion: v1
kind: Service
metadata:
  name: ingress-nginx
  namespace: ingress-nginx
spec:
  type: NodePort
  ports:
  - name: http
    port: 80
    targetPort: 80
    protocol: TCP
  - name: https
    port: 443
    targetPort: 443
    protocol: TCP
  selector:
    app: ingress-nginx

这里使用的nodeport的方式，对于其他云平台可以使用官方提供的对应yaml文件创建。

查看配置是否成功：

# kubectl get pods --all-namespaces -l app=ingress-nginx --watch

NAMESPACE       NAME                                        READY     STATUS    RESTARTS   AGE
ingress-nginx   nginx-ingress-controller-6c9fcdf8d9-fvn57   1/1       Running   0          38m


# kubectl get pods -n ingress-nginx
NAME                                        READY     STATUS    RESTARTS   AGE
default-http-backend-7fb45cbc-kv24l         1/1       Running   0          39m
nginx-ingress-controller-6c9fcdf8d9-fvn57   1/1       Running   0          39m


# kubectl get svc -n ingress-nginx
NAME                   TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)                      AGE
default-http-backend   ClusterIP   10.1.74.56    <none>        80/TCP                       40m
ingress-nginx          NodePort    10.1.28.135   <none>        80:34596/TCP,443:28411/TCP   24m

在安装有kube-proxy的节点上会显示Nodeport映射的端口：

[root@node-2 ~]# netstat -lntp|egrep  "28411|34596"
tcp6       0      0 :::28411                :::*                    LISTEN      1164/kube-proxy     
tcp6       0      0 :::34596                :::*                    LISTEN      1164/kube-proxy  

[root@node-3 ~]# netstat -lntp|egrep  "28411|34596"
tcp6       0      0 :::28411                :::*                    LISTEN      1164/kube-proxy     
tcp6       0      0 :::34596                :::*                    LISTEN      1164/kube-proxy 

访问任意kube-proxy节点的端口，会返回404（还未配置Ingress）说明Ingress安装完成：

# curl 10.0.0.3:34596

default backend - 404

当然，在指定NodePort时，也可以指定端口：

apiVersion: v1
kind: Service
metadata:
  name: ingress-nginx
  namespace: ingress-nginx
spec:
  type: NodePort
  ports:
  - name: http
    port: 80
    targetPort: 80
    nodePort: 20080
    protocol: TCP
  - name: https
    port: 443
    targetPort: 443
    nodePort: 20443
    protocol: TCP
  selector:
    app: ingress-nginx

定义Ingress的访问策略

使用Ingress可以实现多种访问策略：

• 转发到单个后端服务
• 同一域名，不同的URL路径被转发到不同服务器上
• 不同域名（虚拟主机名）转发到不同服务器上
• 不使用域名的转发规则

转发到单个后端服务上

定义如下Ingress配置文件:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: test1
spec:
  rules:
  - host: test.com
    http:
      paths:
      - path: /demo
        backend:
          serviceName: php-apache
          servicePort: 80



# curl  test.com:20080/demo/index.html
demo page! 

同一域名，不同的URL路径被转发到不同服务器上

定义如下文件：

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: test2
spec:
  rules:
  - host: test.com
    http:
      paths:
      - path: /tomcat
        backend:
          serviceName: tomcat-service
          servicePort: 8080
      - path: /apache
        backend:
          serviceName: php-apache
          servicePort: 80

这里会将路径带入到后端服务中，如果后端的服务没有此路径，将会找不到服务，从而返回404。

不同域名（虚拟主机名）转发到不同服务器上

这里使用示例定义一个Ingress 用于访问集群中的 tomcat和apache,当前的服务如下：

# kubectl  get svc
NAME             TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)    AGE
php-apache       ClusterIP   10.1.175.166   <none>        80/TCP     4d
tomcat-service   ClusterIP   10.1.228.9     <none>        8080/TCP   1h

# kubectl  get pod  -o wide
NAME                                 READY     STATUS    RESTARTS   AGE       IP          NODE
php-apache-56b5765b95-rhvg2          1/1       Running   2          5d        10.2.15.6   10.0.0.2
tomcat-deployment-65799d5fc4-5dx4h   1/1       Running   0          3h        10.2.22.7   10.0.0.3

创建一个Ingress:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: test
spec:
  rules:
  - host: tomcat.com
    http:
      paths:
      - backend:
          serviceName: tomcat-service
          servicePort: 8080
  - host: apache.com
    http:
      paths:
      - path: /demo/index.html
        backend:
          serviceName: php-apache
          servicePort: 80

创建成功后，会显示如下信息：

# kubectl  get  ing -o wide
NAME      HOSTS                   ADDRESS   PORTS     AGE
test      tomcat.com,apache.com             80        2h

在不使用kubernetes支持的公有云平台下，默认的 ADDRESS地址会显示为空，如果需要显示节点的IP，可以添加--report-node-internal-ip-address=true的参数，或者修改flags.go z中的源码，对应的useNodeInternalIP = flags.Bool("report-node-internal-ip-address", false, 中 false 修改为 true。

在本主机host 文件中配置上域名和映射20080端口的主机节点，通过不同的域名访问此节点：

[root@node-1 ingress]# kubectl  get nodes
NAME       STATUS    ROLES     AGE       VERSION
10.0.0.2   Ready     <none>    25d       v1.10.4
10.0.0.3   Ready     <none>    24d       v1.10.4

[root@node-1 ingress]# echo "10.0.0.2 tomcat.com apache.com" >> /etc/hosts

[root@node-1 ~]# curl apache.com:20080
OK!

[root@node-1 ~]# curl tomcat.com:20080
tomcat demo!

[root@node-1 ~]# curl 10.0.0.2:20080
default backend - 404      # 当找不到对应的后端Service时，会去访问指定的默认backend。

查看代理信息：

[root@node-1 ~]# curl --head tomcat.com:20080
HTTP/1.1 200 
Server: nginx/1.13.12
Date: Tue, 03 Jul 2018 11:47:31 GMT
Content-Type: text/html;charset=UTF-8
Connection: keep-alive
Vary: Accept-Encoding

[root@node-1 ~]# curl --head apache.com:20080
HTTP/1.1 200 OK
Server: nginx/1.13.12
Date: Tue, 03 Jul 2018 11:47:41 GMT
Content-Type: text/html; charset=UTF-8
Connection: keep-alive
Vary: Accept-Encoding
X-Powered-By: PHP/5.6.14

登录Ingress的Pod,我们可以看到已经添加了相应的转发规则：

# kubectl  exec -it nginx-ingress-controller-6c9fcdf8d9-fvn57 -n ingress-nginx -- sh

$ cat nginx.conf

...
	upstream default-php-apache-80 {
		least_conn;
		
		keepalive 32;
		
		server 10.2.15.6:80 max_fails=0 fail_timeout=0;
		
	}
	
	upstream default-tomcat-service-8080 {
		least_conn;
		
		keepalive 32;
		
		server 10.2.22.7:8080 max_fails=0 fail_timeout=0;
		
	}

...
	server {
		server_name apache.com ;
		
		listen 80;
		
		listen [::]:80;
		
		set $proxy_upstream_name "-";

...

	## start server tomcat.com
	server {
		server_name tomcat.com ;
		
		listen 80;
		
		listen [::]:80;
		
		set $proxy_upstream_name "-";
		

Ingress的TLS安全设置

对于使用https设置TLS的安全证书方面，Ingress 也可以支持。 通过以下步骤进行设置：

• 创建自签名的密钥和SSL证书文件
• 将证书保存到Kubernetes中的一个Secret资源对象上。
• 将该Secret对象设置到Ingress中。

目前，Ingress仅支持单个TLS端口443，并假定TLS termination。 如果Ingress中的TLS配置部分指定了不同的主机，则它们将根据通过SNI TLS扩展指定的主机名（假如Ingress controller支持SNI）在多个相同端口上进行复用。TLS secret中必须包含名为tls.crt和tls.key的密钥，这里面包含了用于TLS的证书和私钥，其格式如下：

apiVersion: v1
data:
  tls.crt: base64 encoded cert   # cert 文件内容
  tls.key: base64 encoded key    # key 文件内容
kind: Secret
metadata:
  name: testsecret
  namespace: default
type: Opaque

在Ingress中引用 Secret:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: no-rules-map
spec:
  tls:
    - secretName: testsecret
  backend:
    serviceName: s1
    servicePort: 80

创建自签名的密钥和SSL证书文件并生成Secret

首先生成证书：

# openssl  req -x509 -nodes -days 5000 -newkey rsa:2048 -keyout tls.key -out tls.crt -subj "/CN=myweb.com" 

利用生成的key 和 crt文件创建Secret:

# kubectl  create secret tls myweb-ingress-secret --key tls.key --cert tls.crt
secret "myweb-ingress-secret" created

# kubectl  get secret
NAME                   TYPE                                  DATA      AGE
default-token-hmvnc    kubernetes.io/service-account-token   3         26d
myweb-ingress-secret   kubernetes.io/tls                     2         13s

我们也可以使用YAML文件的方式创建，其格式和下面的类似：

# kubectl  get secret myweb-ingress-secret -o yaml

apiVersion: v1
data:
  tls.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUMr
  ...
  Gh0UlJJUkV5bUJpZjZNdmRlOERmUVRiT0x5OUF5Y0xVb2gyL2RlRnhOST0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=
  
  tls.key: LS0tLS1CRUdJTiBQUklWQVRFIEtFWS0tLS0tCk1JSUV2d0lCQURBTkJna3Foa2lHOXcwQ
  ...
  MzekE9PQotLS0tLUVORCBQUklWQVRFIEtFWS0tLS0tCg==
kind: Secret
metadata:
  name: myweb-ingress-secret
  namespace: default
type: kubernetes.io/tls

创建Ingress对象

TLS的证书创建到Secret之后，就可以利用他来创建Ingress对象了，使用如下文件：

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: myweb-ingress
spec:
  tls:
  - secretName: myweb-ingress-secret
  rules:
  - host: myweb.com
    http:
      paths:
      - path: /demo
        backend:
          serviceName: php-apache
          servicePort: 80

使用默认的443端口访问（本地使用的NodePort，映射到Ingress的443端口）：

# curl -k https://myweb.com:20443/demo/index.html
demo test https page!

使用网页可以访问网页内容： https://myweb.com:20443/demo