在k8s中部署rabbitmq集群+nfs

原创

风山渐1 2023-03-23 09:14:59 博主文章分类：rabbitmq ©著作权

文章标签 rabbitmq k8s 文章分类 运维

©著作权归作者所有：来自51CTO博客作者风山渐1的原创作品，请联系作者获取转载授权，否则将追究法律责任

在k8s中部署rabbitmq集群

rabbitmq官网地址 https://www.rabbitmq.com/

官网推荐使用mq管理工具Kubernetes Operator 来安装管理mq集群

安装Kubernetes Operator

官网的文档 https://www.rabbitmq.com/kubernetes/operator/operator-overview.html

# Operator的yaml文件地址
https://github.com/rabbitmq/cluster-operator/releases/latest/download/cluster-operator.yml

# 安装
kubectl apply -f "https://github.com/rabbitmq/cluster-operator/releases/latest/download/cluster-operator.yml"

安装完之后查看Operator是否启动正常, 默认pod在 rabbitmq-system命名空间下

# kubectl get pod -n rabbitmq-system
NAME                                         READY   STATUS    RESTARTS      AGE
rabbitmq-cluster-operator-6779b65b88-p4hg9   1/1     Running   1 (90m ago)   16h

如果启动失败可以使用describe 和 logs 查看详情。

Operator安装完成之后，就可以安装rabbitmq了

安装rabbitmq

安装mq时，会使用storageClass创建pvc，所以k8s需要提前配置好storageClass。

安装rabbitmq的文档https://www.rabbitmq.com/kubernetes/operator/using-operator.html

rabbitmq配置文件的文档https://www.rabbitmq.com/configure.html

创建一个最简单的mq配置文件，只定义集群名称，其他都使用默认值

apiVersion: rabbitmq.com/v1beta1
kind: RabbitmqCluster
metadata:
  name: rabbitmq-cluster

执行，会调用Operator去创建mq

kubectl apply -f rabbitmq-cluster.yaml

自定义mq的yaml文件，配置项可以参考官网文档

rabbitmq-cluster.yaml

apiVersion: rabbitmq.com/v1beta1
kind: RabbitmqCluster
metadata:
  name: rabbitmq-cluster
  labels:
    app: rabbitmq-cluster
spec:
  replicas: 3
  image: registry.cn-beijing.aliyuncs.com/qf_imqges/rabbitmq:3.9
  persistence:
    storageClassName: nfs-client
    storage: 5Gi
  resources:
    limits:
      cpu: 1
      memory: 500Mi
    requests:
      cpu: 100m
      memory: 500Mi
  service:
    type: NodePort
  override:
    service:
      spec:
        ports:
          - name: amqp
            protocol: TCP
            nodePort: 30427
            port: 5672
            targetPort: 5672
          - name: management
            nodePort: 31629
            port: 15672
            protocol: TCP
            targetPort: 15672
          - name: prometheus
            nodePort: 30322
            port: 15692
            protocol: TCP
            targetPort: 15692
  rabbitmq:
    additionalPlugins:
      - rabbitmq_delayed_message_exchange

执行

kubectl apply -f rabbitmq-cluster.yaml

查看statefulset，默认在default命名空间下创建一个statefulset

[root@master01 nfsclass2]# kubectl get statefulset
NAME                      READY   AGE
rabbitmq-cluster-server   3/3     55m

查看pod

[root@master01 nfsclass2]# kubectl get pod
NAME                                      READY   STATUS    RESTARTS       AGE
rabbitmq-cluster-server-0                 1/1     Running   0              52m
rabbitmq-cluster-server-1                 1/1     Running   0              52m
rabbitmq-cluster-server-2                 1/1     Running   0              52m

查看service

[root@master01 nfsclass2]# kubectl get svc
NAME                     TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)                                          AGE
rabbitmq-cluster         NodePort    10.1.126.172   <none>        5672:30427/TCP,15672:31629/TCP,15692:30322/TCP   64m
rabbitmq-cluster-nodes   ClusterIP   None           <none>        4369/TCP,25672/TCP                               64m

查看pvc

[root@master01 nfsclass2]# kubectl get pvc
NAME                                    STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
persistence-rabbitmq-cluster-server-0   Bound    pvc-b88b10f0-6167-458e-8928-9fec0f2bfb00   5Gi        RWO            nfs-client     56m
persistence-rabbitmq-cluster-server-1   Bound    pvc-e21f3a5d-7830-42ff-8f71-e2c3fa5312bc   5Gi        RWO            nfs-client     56m
persistence-rabbitmq-cluster-server-2   Bound    pvc-d3ba10f2-5f12-4cca-a804-8b812a45d506   5Gi        RWO            nfs-client     56m

查看secret，存放了mq的默认用户和密码

[root@master01 nfsclass2]# kubectl get secret
NAME                             TYPE     DATA   AGE
rabbitmq-cluster-default-user    Opaque   7      62m
rabbitmq-cluster-erlang-cookie   Opaque   1      62m

mq集群安装完成

注意事项

Operator升级版本的时候要注意升级提示信息：有的Operator版本升级需要重启rabbitmq集群，Operator对rabbitmq集群的版本也有要求。

如果在升级Operator时不想重启rabbitmq集群可以通过标签暂停对mq集群的核对检查

# 查看mq集群名称
# kubectl get rabbitmqclusters
NAME               ALLREPLICASREADY   RECONCILESUCCESS   AGE
rabbitmq-cluster   True               True               76m

# 暂停
kubectl label rabbitmqclusters rabbitmq-cluster rabbitmq.com/pauseReconciliation=true

恢复

kubectl label rabbitmqclusters rabbitmq-cluster rabbitmq.com/pauseReconciliation-

rabbitmq集群也可以单独升级，修改image版本号，apply既可

exchange 、queue 在创建的时候需要指定持久化存储，在发送消息时也要指定消息持久化存储。

python测试脚本，发送消息

import pika
 
# ######################### producer #########################
 
account = pika.PlainCredentials("mqadmin", "123456")
connection = pika.BlockingConnection(pika.ConnectionParameters("11.0.1.11", 30427, '/', credentials=account))

channel = connection.channel()
 
#create queue named hello
#channel.queue_declare(queue='hello')

channel.basic_publish(exchange='test_exchanges',
                      routing_key = 'test_queue',
                      body='111')
print("[x] Sent 'Hello World!'")
connection.close()