背景: 在一个新的项目中,采用了kubernetes的微服务架构进行部署项目,考虑到网关及其他服务的外部暴露问题,摒弃了NodePort方式,采用了ingress-controller-nginx的模式进行暴露内部服务。实际ingress规则示例:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: nginx
name: gateway-ingress
namespace: je-dev
spec:
rules:
- host: dev-gateway.xxx.com
http:
paths:
- backend:
service:
name: dev-jecloud-gateway
port:
number: 9090
path: /
pathType: Prefix
问题: 前端项目独立采用Nginx进行部署,并配置虚拟主机规则,在验证文件上传时,出现了"413 Request Entity Too Large",第一反应是去修改前端nginx配置参数:client_max_body_size和client_body_buffer_size,但是配置完成后,重启后问题依旧提示不能上传超过1M的文件大小,等同于配置修改无效。思来想去,后端的网关采用的控制器是ingress-nginx-controller,于是去查找资料发现默认的ingress-nginx-controller也会带来这样的上传限制问题。 解决: 对于 Nginx Ingress Controller,可以通过在 Ingress 资源上设置 nginx.ingress.kubernetes.io/proxy-body-size 注解来限制上传大小。例如:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
nginx.ingress.kubernetes.io/proxy-body-size: "100m" # 设置上传大小限制为 50 MB
name: example-ingress
namespace: example
spec:
# Ingress 规则...
这是针对于单个ingress的规则进行配置,解决如下:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
nginx.ingress.kubernetes.io/proxy-body-size: "100M" #配置限制上传100M
kubernetes.io/ingress.class: nginx
name: gateway-ingress
namespace: je-dev
spec:
rules:
- host: dev-gateway.xxx.com
http:
paths:
- backend:
service:
name: dev-gateway
port:
number: 9090
path: /
pathType: Prefix
但是修改完,验证上传还是会有这样的问题,由于系统交互存在多个ingress规则,所以这里通过ingress-controller的configmap进行全局ingress上传大小配置,修改如下:
[root@k8s-master-130 ~]# kubectl get cm -n ingress-nginx
NAME DATA AGE
ingress-nginx-controller 2 166d
kube-root-ca.crt 1 166d
[root@k8s-master-130 ~]# kubectl edit cm ingress-nginx-controller -n ingress-nginx
# Please edit the object below. Lines beginning with a '#' will be ignored,
# and an empty file will abort the edit. If an error occurs while saving this file will be
# reopened with the relevant failures.
#
apiVersion: v1
data:
allow-snippet-annotations: "true"
proxy-body-size: 100m #增加这行
kind: ConfigMap
metadata:
annotations:
meta.helm.sh/release-name: ingress-nginx
meta.helm.sh/release-namespace: ingress-nginx
creationTimestamp: "2023-09-15T09:00:12Z"
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.5.1
helm.sh/chart: ingress-nginx-4.4.0
name: ingress-nginx-controller
namespace: ingress-nginx
resourceVersion: "196125506"
selfLink: /api/v1/namespaces/ingress-nginx/configmaps/ingress-nginx-controller
uid: 92765cbb-42c8-4e64-985a-1ee05060231a
保存,退出
[root@k8s-master-130 ~]# kubectl describe cm ingress-nginx-controller -n ingress-nginx #查看配置
Name: ingress-nginx-controller
Namespace: ingress-nginx
Labels: app.kubernetes.io/component=controller
app.kubernetes.io/instance=ingress-nginx
app.kubernetes.io/managed-by=Helm
app.kubernetes.io/name=ingress-nginx
app.kubernetes.io/part-of=ingress-nginx
app.kubernetes.io/version=1.5.1
helm.sh/chart=ingress-nginx-4.4.0
Annotations: meta.helm.sh/release-name: ingress-nginx
meta.helm.sh/release-namespace: ingress-nginx
Data
====
proxy-body-size:
----
100m
allow-snippet-annotations:
----
true
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal UPDATE 49m (x2 over 142d) nginx-ingress-controller ConfigMap ingress-nginx/ingress-nginx-controller
Normal UPDATE 49m nginx-ingress-controller ConfigMap ingress-nginx/ingress-nginx-controller
最终,修改完ingress-nginx-controller的参数配置,解决了前端上传的413错误! 对于 Traefik Ingress Controller,可以使用标签或注解来设置上传和下载大小限制。以下是设置上传和下载大小的示例:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
traefik.ingress.kubernetes.io/max-body-size: "100m" # 设置上传大小限制为 50 MB
name: example-ingress
namespace: example
spec:
# Ingress 规则...