消息队列开启注解

RabbitMQ Work Queues

简介

Hello Word中只是简单介绍，下面开始重点分析
例如：当我们有复杂需求，我们需要提升效率，毕竟只有一个消费者难免处理不过来，就如官网中所提到的一样——在这篇教程中，将创建一个工作队列（Work Queue），它会发送一些耗时的任务给多个工作者（Worker）。

工作队列（又称：任务队列——Task Queues）是为了避免等待一些占用大量资源、时间的操作。当我们把任务（Task）当作消息发送到队列中，一个运行在后台的工作者（worker）进程就会取出任务然后处理。当你运行多个工作者（workers），任务就会在它们之间共享。

循环调度

使用工作队列的一个好处就是它能够并行的处理队列。如果堆积了很多任务，我们只需要添加更多的工作者（workers）就可以，然后扩展也很简单。

默认来说，RabbitMQ会按顺序得把消息发送给每个消费者（consumer）。平均每个消费者都会收到同等数量得消息。这种发送消息得方式叫做——轮询（round-robin）。

round-robin 一定要细细体会。

Send.py
import pika
import time
#  
credentials = pika.PlainCredentials('xiaoxia', 'xiaoxia')
connection = pika.BlockingConnection(pika.ConnectionParameters(
    '47.244.*.*', 5672, '/', credentials))
# 声明queue
channel = connection.channel()
# 声明queue
channel.queue_declare(queue='round_robin_queue')
# n RabbitMQ a message can never be sent directly to the queue, it always needs to go through an exchange.
n = 20
sum = 0
counter = 1
while counter <= n:
    channel.basic_publish(exchange='',                 #Producer只能发送到exchange，它是不能直接发送到queue的，发送到默认exchange
                          routing_key='round_robin_queue',         #路由key发送指定队列
                          body='Hello World:'+str(counter)) #发送的消息
    time.sleep(1)
    counter += 1
print(" [x] Sent 'Hello World!'")
connection.close()

Receive.py

import pika
credentials = pika.PlainCredentials('xiaoxia', 'xiaoxia')
#建立连接
connection = pika.BlockingConnection(pika.ConnectionParameters(
    '47.244.*.93', 5672, '/', credentials))
channel = connection.channel()
# You may ask why we declare the queue again ‒ we have already declared it in our previous code.
# We could avoid that if we were sure that the queue already exists. For example if send.py program
# was run before. But we're not yet sure which program to run first. In such cases it's a good
# practice to repeat declaring the queue in both programs.
#如果生产者先运行并创建了队列这里就可以不用声明，但是有可能消费者先运行 下面的basic_consume就会因为没有队列报错。
channel.queue_declare(queue='round_robin_queue')

def callback(ch, method, properties, body):  #定义回调函数用于取出队列中的数据
    print(" [x] Received %r" % body)


channel.basic_consume(queue='round_robin_queue',
                      on_message_callback=callback,
                      auto_ack=False)          #不确认消息
print(' [*] Waiting for messages. To exit press CTRL+C')
channel.start_consuming()                      #监听数据

实际测试场景：先执行5个Receiver.py 后再执行Send.py,实际结果如下

[x] Received b'Hello World:1'
 [x] Received b'Hello World:6'
 [x] Received b'Hello World:11'
 [x] Received b'Hello World:16'

[*] Waiting for messages. To exit press CTRL+C
 [x] Received b'Hello World:2'
 [x] Received b'Hello World:7'
 [x] Received b'Hello World:12'
 [x] Received b'Hello World:17'

[*] Waiting for messages. To exit press CTRL+C
 [x] Received b'Hello World:3'
 [x] Received b'Hello World:8'
 [x] Received b'Hello World:13'
 [x] Received b'Hello World:18'

[*] Waiting for messages. To exit press CTRL+C
 [x] Received b'Hello World:4'
 [x] Received b'Hello World:9'
 [x] Received b'Hello World:14'
 [x] Received b'Hello World:19'

[x] Received b'Hello World:5'
 [x] Received b'Hello World:10'
 [x] Received b'Hello World:15'
 [x] Received b'Hello World:20'

你发现了什么？ 队列依次轮训发送给5个消费者【Receiver.py】
通过上面的要理解Work Queue,轮训round-robin【把消息依次分发】

消息确认

理清楚两个参数:

• auto_ack
• ch.basic_ack(delivery_tag=method.delivery_tag)

先看auto_act,源码解释如下：

bool auto_ack: if set to True, automatic acknowledgement mode will be used
                              (see http://www.rabbitmq.com/confirms.html). This corresponds
                              with the 'no_ack' parameter in the basic.consume AMQP 0.9.1
                              method

也就是说，如果设置True,自动确认机制 模式将被使用。对应的是两种确认模式

• 自动确认模式（automatic acknowledgement
  model）：当RabbbitMQ将消息发送给应用后，消费者端自动回送一个确认消息，此时RabbitMQ删除此消息。
• 显式确认模式（explicit acknowledgement
  model）：消费者收到消息后，可以在执行一些逻辑后，消费者自己决定什么时候发送确认回执（acknowledgement），RabbitMQ收到回执后才删除消息，这样就保证消费端不会丢失消息
  通过如下代码和Rabbit 的操作后台看下实际效果。

WorkQueue_Receiver4.py
import pika
credentials = pika.PlainCredentials('xiaoxia', 'xiaoxia')
#建立连接
connection = pika.BlockingConnection(pika.ConnectionParameters(
    '47.244.28.93', 5672, '/', credentials))
channel = connection.channel()
# You may ask why we declare the queue again ‒ we have already declared it in our previous code.
# We could avoid that if we were sure that the queue already exists. For example if send.py program
# was run before. But we're not yet sure which program to run first. In such cases it's a good
# practice to repeat declaring the queue in both programs.
#如果生产者先运行并创建了队列这里就可以不用声明，但是有可能消费者先运行 下面的basic_consume就会因为没有队列报错。
channel.queue_declare(queue='round_robin_queue_tt5)')

def callback(ch, method, properties, body):  #定义回调函数用于取出队列中的数据
    print(" [x] Received %r" % body)


channel.basic_consume(queue='round_robin_queue_tt5',
                      on_message_callback=callback,
                      auto_ack=True)          #不确认消息
print(' [*] Waiting for messages. To exit press CTRL+C')
channel.start_consuming()                      #监听数据




WorkQueue_Send4.py
import time
import pika
credentials = pika.PlainCredentials('xiaoxia', 'xiaoxia')
connection = pika.BlockingConnection(pika.ConnectionParameters(
    '47.244.*.*', 5672, '/', credentials))
# 声明queue
channel = connection.channel()
# 声明queue
channel.queue_declare(queue='round_robin_queue_test4')
# n RabbitMQ a message can never be sent directly to the queue, it always needs to go through an exchange.
n = 20
sum = 0
counter = 1
while counter <= n:
    channel.basic_publish(exchange='',  # Producer只能发送到exchange，它是不能直接发送到queue的，发送到默认exchange
                          routing_key='round_robin_queue_test4',  # 路由key发送指定队列
                          body='Hello World:' + str(counter))  # 发送的消息
    time.sleep(1)
    counter += 1
print(" [x] Sent 'Hello World!'")
connection.close()

主要看WorkQueue_Receiver4.py代码，auto_ack 变量，auto_ack=True，也就是自动确认模式

当运行WorkQueue_Send4.py 后【测试时候运行了两次，所以有40条消息】，后面显示当前Queues 状态running Ready 和Total消息不断增加，最终后台有40条未处理的消息

运行WorkQueue_Receiver4.py

/usr/local/Cellar/python3/3.6.2/Frameworks/Python.framework/Versions/3.6/bin/python3.6 /Users/user/Workspaces/git/pythonjichu/jichu/com/wfc/python3.5学习/rabitmq/WorkQueue_Receiver4.py
 [*] Waiting for messages. To exit press CTRL+C
 [x] Received b'Hello World:1'
 [x] Received b'Hello World:2'
 [x] Received b'Hello World:3'
 [x] Received b'Hello World:4'
 [x] Received b'Hello World:5'
 [x] Received b'Hello World:6'
 [x] Received b'Hello World:7'
 [x] Received b'Hello World:8'
 [x] Received b'Hello World:9'
 [x] Received b'Hello World:10'
 [x] Received b'Hello World:11'
 [x] Received b'Hello World:12'
 [x] Received b'Hello World:13'
 [x] Received b'Hello World:14'
 [x] Received b'Hello World:15'
 [x] Received b'Hello World:16'
 [x] Received b'Hello World:17'
 [x] Received b'Hello World:18'
 [x] Received b'Hello World:19'
 [x] Received b'Hello World:20'
 [x] Received b'Hello World:1'
 [x] Received b'Hello World:2'
 [x] Received b'Hello World:3'
 [x] Received b'Hello World:4'
 [x] Received b'Hello World:5'
 [x] Received b'Hello World:6'
 [x] Received b'Hello World:7'
 [x] Received b'Hello World:8'
 [x] Received b'Hello World:9'
 [x] Received b'Hello World:10'
 [x] Received b'Hello World:11'
 [x] Received b'Hello World:12'
 [x] Received b'Hello World:13'
 [x] Received b'Hello World:14'
 [x] Received b'Hello World:15'
 [x] Received b'Hello World:16'
 [x] Received b'Hello World:17'
 [x] Received b'Hello World:18'
 [x] Received b'Hello World:19'
 [x] Received b'Hello World:20'

再看看控制面板，MesseageQueue 情况，MQ消息队列已经全部清空了

如果设置了参数auto_ack=True,那么在回调中函数就没有必要向服务端确认了，如果写了就会直接报错的

auto_ack=True

Traceback (most recent call last):
  File "/Users/user/Workspaces/git/pythonjichu/jichu/com/wfc/python3.5学习/rabitmq/WorkQueue_Receiver4.py", line 30, in <module>
    channel.start_consuming()  # 监听数据
  File "/usr/local/Cellar/python3/3.6.2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pika/adapters/blocking_connection.py", line 1857, in start_consuming
    self._process_data_events(time_limit=None)
  File "/usr/local/Cellar/python3/3.6.2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pika/adapters/blocking_connection.py", line 2023, in _process_data_events
    raise self._closing_reason  # pylint: disable=E0702
pika.exceptions.ChannelClosedByBroker: (406, 'PRECONDITION_FAILED - unknown delivery tag 1')

直接设置 auto_ack=False,或者不设置，默认就是False，先不给RabbitMQTT确认消息接收到了。不回馈服务端【屏蔽掉 ch.basic_ack(delivery_tag=method.delivery_tag) 】

def callback(ch, method, properties, body):  # 定义回调函数用于取出队列中的数据
    print(" [x] Received %r" % body)
    # ch.basic_ack(delivery_tag=method.delivery_tag)  # 发送ack消息

那么 WorkQueue_Receiver5.py 代码如下。

import time
import pika
#  
credentials = pika.PlainCredentials('xiaoxia', 'xiaoxia')
connection = pika.BlockingConnection(pika.ConnectionParameters(
    '47.244.*.*', 5672, '/', credentials))
# 声明queue
channel = connection.channel()
# 声明queue
channel.queue_declare(queue='round_robin_queue_test5')
# n RabbitMQ a message can never be sent directly to the queue, it always needs to go through an exchange.
n = 20
sum = 0
counter = 1
while counter <= n:
    channel.basic_publish(exchange='',  # Producer只能发送到exchange，它是不能直接发送到queue的，发送到默认exchange
                          routing_key='round_robin_queue_test5',  # 路由key发送指定队列
                          body='Hello World:' + str(counter))  # 发送的消息
    time.sleep(1)
    counter += 1
print(" [x] Sent 'Hello World!'")
connection.close()

WorkQueue_Receiver5.py 代码

import pika
credentials = pika.PlainCredentials('xiaoxia', 'xiaoxia')
# 建立连接
connection = pika.BlockingConnection(pika.ConnectionParameters(
    '47.244.*.*', 5672, '/', credentials))
channel = connection.channel() 
channel.queue_declare(queue='round_robin_queue_test5')
def callback(ch, method, properties, body):  # 定义回调函数用于取出队列中的数据
    print(" [x] Received %r" % body)
    # ch.basic_ack(delivery_tag=method.delivery_tag)  # 发送ack消息
channel.basic_consume(queue='round_robin_queue_test5',
                      on_message_callback=callback,
                      auto_ack=False)
print(' [*] Waiting for messages. To exit press CTRL+C')
channel.start_consuming()  # 监听数据

运行send和receive代码后，看下RabbitMQTT状态如何,和consol得到消息

/usr/local/Cellar/python3/3.6.2/Frameworks/Python.framework/Versions/3.6/bin/python3.6 /Users/user/Workspaces/git/pythonjichu/jichu/com/wfc/python3.5学习/rabitmq/WorkQueue_Receiver5.py
 [*] Waiting for messages. To exit press CTRL+C
 [x] Received b'Hello World:1'
 [x] Received b'Hello World:2'
 [x] Received b'Hello World:3'
 [x] Received b'Hello World:4'
 [x] Received b'Hello World:5'
 [x] Received b'Hello World:6'
 [x] Received b'Hello World:7'
 [x] Received b'Hello World:8'
 [x] Received b'Hello World:9'
 [x] Received b'Hello World:10'
 [x] Received b'Hello World:11'
 [x] Received b'Hello World:12'
 [x] Received b'Hello World:13'
 [x] Received b'Hello World:14'
 [x] Received b'Hello World:15'
 [x] Received b'Hello World:16'
 [x] Received b'Hello World:17'
 [x] Received b'Hello World:18'
 [x] Received b'Hello World:19'
 [x] Received b'Hello World:20'

所有说，即使消费者 消费了 消息，但是RabbitMQTT的MQ上面消息并没有清除。
那么消息队列上面还是有消息的，这样会造成，如果再次运行 WorkQueue_Receiver5.py ，会造成消息重复发送的情况。为什么会重复发送，因为RabbitMQTT中的MessageQueue上面的消息是Unacked 状态，消息并没有从Queue上面删除，这样就会造成MQ上面的消息越来越多。

直接设置 auto_ack=False,或者不设置，默认就是False，给RabbitMQTT确认消息接收到了。回馈MQ端【 ch.basic_ack(delivery_tag=method.delivery_tag) 】
发送和接收的代码如下

WorkQueue_Receiver6.py

import pika
credentials = pika.PlainCredentials('xiaoxia', 'xiaoxia')
# 建立连接
connection = pika.BlockingConnection(pika.ConnectionParameters(
    '47.244.*.*', 5672, '/', credentials))
channel = connection.channel()
channel.queue_declare(queue='round_robin_queue_test6')
def callback(ch, method, properties, body):  # 定义回调函数用于取出队列中的数据
    print(" [x] Received %r" % body)
    ch.basic_ack(delivery_tag=method.delivery_tag)  # 发送ack消息
channel.basic_consume(queue='round_robin_queue_test6',
                      on_message_callback=callback,
                      auto_ack=False)
print(' [*] Waiting for messages. To exit press CTRL+C')
channel.start_consuming()  # 监听数据


WorkQueue_Send6.py

import time
import pika
#  
credentials = pika.PlainCredentials('xiaoxia', 'xiaoxia')
connection = pika.BlockingConnection(pika.ConnectionParameters(
    '47.244.*.*', 5672, '/', credentials))
# 声明queue
channel = connection.channel()
# 声明queue
channel.queue_declare(queue='round_robin_queue_test6')
# n RabbitMQ a message can never be sent directly to the queue, it always needs to go through an exchange.
n = 20
sum = 0
counter = 1
while counter <= n:
    channel.basic_publish(exchange='',  # Producer只能发送到exchange，它是不能直接发送到queue的，发送到默认exchange
                          routing_key='round_robin_queue_test6',  # 路由key发送指定队列
                          body='Hello World:' + str(counter))  # 发送的消息
    time.sleep(1)
    counter += 1
print(" [x] Sent 'Hello World!'")
connection.close()

先运行Send后，后台MQ状态，和运行Receive后，后台状态

由此可见，在消费者【Receive端】确认消息回馈到RabbitMQ后，RabbitMQ 在接收到消息后，立刻将消息清除掉。

如果有多个客户端，由于消费者消费过程中遇到异常导致无法回馈给RabbitMQ情况下，RabbitMQ消息队列会分发给其它消费者，避免消息丢失！