上一篇文章（19. consuemr消费失败后，消息如何处理？）分析到了，需要重新消费的消息，会被转存在延时队列中，这一篇文章，咱们就来深入分析一下延时队列
在rocketmq中，延时消息不支持自定义延时，只支持特定的延时时间级别
MessageStoreConfig
private String messageDelayLevel = "1s 5s 10s 30s 1m 2m 3m 4m 5m 6m 7m 8m 9m 10m 20m 30m 1h 2h";
供18个延时时间，从1开始供18个级别，delayTimeLevel=0时，表示不延时。
在生产消息的时候，可以同个message.setDelayTimeLevel()来设置延时级别。
broker在收到消息之后，在commitlog.asyncPutMessage()方法里，会将原始的topic和queueId使用属性PROPERTY_REAL_TOPIC，PROPERTY_REAL_QUEUE_ID保存下来
然后将topic设置为SCHEDULE_TOPIC_XXXX，将queueId设置为delayTimeLevel-1，然后存储在commitlog中
等到reputService的时候，消息会被分到SCHEDULE_TOPIC_XXXX的consumeQueue中，并不会分配到目标topic的consumeQueue中，
接下来会通过ScheduleMessageService来调度延时消息
在DefaultMessageStore初始化的时候
public DefaultMessageStore(final MessageStoreConfig messageStoreConfig, final BrokerStatsManager brokerStatsManager,
final MessageArrivingListener messageArrivingListener, final BrokerConfig brokerConfig) throws IOException {
...
this.scheduleMessageService = new ScheduleMessageService(this);
...
}DefaultMessageStore.start()
enableDLegerCommitLog默认是false
public void start() throws Exception {
...
if (!messageStoreConfig.isEnableDLegerCommitLog()) {
this.haService.start();
this.handleScheduleMessageService(messageStoreConfig.getBrokerRole());
}
...
}public void handleScheduleMessageService(final BrokerRole brokerRole) {
if (this.scheduleMessageService != null) {
if (brokerRole == BrokerRole.SLAVE) {
this.scheduleMessageService.shutdown();
} else {
this.scheduleMessageService.start();
}
}
}ScheduleMessageService.start()
public void start() {
// cas 设置started状态
if (started.compareAndSet(false, true)) {
super.load();
// 初始化一个timer
this.timer = new Timer("ScheduleMessageTimerThread", true);
for (Map.Entry<Integer, Long> entry : this.delayLevelTable.entrySet()) {
Integer level = entry.getKey();
Long timeDelay = entry.getValue();
// 初始化消费进度
Long offset = this.offsetTable.get(level);
if (null == offset) {
offset = 0L;
}
// 为每个延时级别都初始化一个timerTask
if (timeDelay != null) {
this.timer.schedule(new DeliverDelayedMessageTimerTask(level, offset), FIRST_DELAY_TIME);
}
}
// 定时任务，默认每隔10s持久化一下延时队列的消费进度
this.timer.scheduleAtFixedRate(new TimerTask() {
@Override
public void run() {
try {
if (started.get()) ScheduleMessageService.this.persist();
} catch (Throwable e) {
log.error("scheduleAtFixedRate flush exception", e);
}
}
}, 10000, this.defaultMessageStore.getMessageStoreConfig().getFlushDelayOffsetInterval());
}
}delayLevelTable就是18个定时级别了，先初始化一个timer,然后为每个delayLevel都会初始化一个延时任务(DeliverDelayedMessageTimerTask)，延时1s,
然后有初始化了一个定时任务，定期持久化一下延时队列的消费进度
org.apache.rocketmq.store.schedule.ScheduleMessageService.DeliverDelayedMessageTimerTask#run
public void run() {
try {
if (isStarted()) {
this.executeOnTimeup();
}
} catch (Exception e) {
// XXX: warn and notify me
log.error("ScheduleMessageService, executeOnTimeup exception", e);
ScheduleMessageService.this.timer.schedule(new DeliverDelayedMessageTimerTask(
this.delayLevel, this.offset), DELAY_FOR_A_PERIOD);
}
}public void executeOnTimeup() {
// 获取到consumeQueue
ConsumeQueue cq =
ScheduleMessageService.this.defaultMessageStore.findConsumeQueue(TopicValidator.RMQ_SYS_SCHEDULE_TOPIC,
delayLevel2QueueId(delayLevel));
long failScheduleOffset = offset;
if (cq != null) {
//读取到buffer
SelectMappedBufferResult bufferCQ = cq.getIndexBuffer(this.offset);
if (bufferCQ != null) {
try {
long nextOffset = offset;
int i = 0;
ConsumeQueueExt.CqExtUnit cqExtUnit = new ConsumeQueueExt.CqExtUnit();
for (; i < bufferCQ.getSize(); i += ConsumeQueue.CQ_STORE_UNIT_SIZE) {
// 每次读取一个消息
// commitlog offset
long offsetPy = bufferCQ.getByteBuffer().getLong();
// msg size
int sizePy = bufferCQ.getByteBuffer().getInt();
// tagscode
long tagsCode = bufferCQ.getByteBuffer().getLong();
if (cq.isExtAddr(tagsCode)) {
if (cq.getExt(tagsCode, cqExtUnit)) {
tagsCode = cqExtUnit.getTagsCode();
} else {
//can't find ext content.So re compute tags code.
log.error("[BUG] can't find consume queue extend file content!addr={}, offsetPy={}, sizePy={}",
tagsCode, offsetPy, sizePy);
long msgStoreTime = defaultMessageStore.getCommitLog().pickupStoreTimestamp(offsetPy, sizePy);
tagsCode = computeDeliverTimestamp(delayLevel, msgStoreTime);
}
}
long now = System.currentTimeMillis();
//送达时间？
long deliverTimestamp = this.correctDeliverTimestamp(now, tagsCode);
// 计算下一个offset
nextOffset = offset + (i / ConsumeQueue.CQ_STORE_UNIT_SIZE);
long countdown = deliverTimestamp - now;
// 判断消息到期没有，到期了就读取消息的完整内容
if (countdown <= 0) {
// 读取消息的完整内容
MessageExt msgExt =
ScheduleMessageService.this.defaultMessageStore.lookMessageByOffset(
offsetPy, sizePy);
if (msgExt != null) {
try {
// 重新构建消息
// 如果是普通的消息，会恢复成原来的topic和queueId
// 如果是重试消息，就恢复成重试的topic和queueId,在消费之前才会恢复成最原始的topic和queueId
MessageExtBrokerInner msgInner = this.messageTimeup(msgExt);
if (TopicValidator.RMQ_SYS_TRANS_HALF_TOPIC.equals(msgInner.getTopic())) {
log.error("[BUG] the real topic of schedule msg is {}, discard the msg. msg={}",
msgInner.getTopic(), msgInner);
continue;
}
// 将消息写入到commitlog
PutMessageResult putMessageResult =
ScheduleMessageService.this.writeMessageStore
.putMessage(msgInner);
// 写入成功
if (putMessageResult != null
&& putMessageResult.getPutMessageStatus() == PutMessageStatus.PUT_OK) {
if (ScheduleMessageService.this.defaultMessageStore.getMessageStoreConfig().isEnableScheduleMessageStats()) {
ScheduleMessageService.this.defaultMessageStore.getBrokerStatsManager().incTopicPutNums(msgInner.getTopic(), putMessageResult.getAppendMessageResult().getMsgNum(), 1);
ScheduleMessageService.this.defaultMessageStore.getBrokerStatsManager().incTopicPutSize(msgInner.getTopic(),
putMessageResult.getAppendMessageResult().getWroteBytes());
ScheduleMessageService.this.defaultMessageStore.getBrokerStatsManager().incBrokerPutNums(putMessageResult.getAppendMessageResult().getMsgNum());
}
continue;
} else {
// XXX: warn and notify me
log.error(
"ScheduleMessageService, a message time up, but reput it failed, topic: {} msgId {}",
msgExt.getTopic(), msgExt.getMsgId());
// 写入commitlog失败，为下一个offset生成一个timertask，并更新消费进度
ScheduleMessageService.this.timer.schedule(
new DeliverDelayedMessageTimerTask(this.delayLevel,
nextOffset), DELAY_FOR_A_PERIOD);
ScheduleMessageService.this.updateOffset(this.delayLevel,
nextOffset);
return;
}
} catch (Exception e) {
/*
* XXX: warn and notify me
*/
log.error(
"ScheduleMessageService, messageTimeup execute error, drop it. msgExt="
+ msgExt + ", nextOffset=" + nextOffset + ",offsetPy="
+ offsetPy + ",sizePy=" + sizePy, e);
}
}
} else {
// 为下一个offset生成1个timertask,并更新进度
ScheduleMessageService.this.timer.schedule(
new DeliverDelayedMessageTimerTask(this.delayLevel, nextOffset),
countdown);
ScheduleMessageService.this.updateOffset(this.delayLevel, nextOffset);
return;
}
} // end of for
nextOffset = offset + (i / ConsumeQueue.CQ_STORE_UNIT_SIZE);
ScheduleMessageService.this.timer.schedule(new DeliverDelayedMessageTimerTask(
this.delayLevel, nextOffset), DELAY_FOR_A_WHILE);
ScheduleMessageService.this.updateOffset(this.delayLevel, nextOffset);
return;
} finally {
bufferCQ.release();
}
} // end of if (bufferCQ != null)
else {
long cqMinOffset = cq.getMinOffsetInQueue();
if (offset < cqMinOffset) {
failScheduleOffset = cqMinOffset;
log.error("schedule CQ offset invalid. offset=" + offset + ", cqMinOffset="
+ cqMinOffset + ", queueId=" + cq.getQueueId());
}
}
} // end of if (cq != null)
// 没有找到消费队列，就表示还没有重试消息或者延时消息，创建timertask, 100ms后重新调度
ScheduleMessageService.this.timer.schedule(new DeliverDelayedMessageTimerTask(this.delayLevel,
failScheduleOffset), DELAY_FOR_A_WHILE);
}先根据topic:SCHEDULE_TOPIC_XXXX、queueId=delayLevel,定位动consumeQueue, 再根据消费offset从consumeQueue中读取未消费的消息，这里读取的消息只是consumeQueue中的消息（commitlog offset, msg size, tagccode），并不是完整的消息
接着校正消息的实际送达时间，然后判断消息是否到期了，到期了就从commitlog中读取完整的消息。
然后将消息重新封装，恢复原来的topic和queueId,重新投递到commitlog中，在经reputService分配到相应的consumeQueue中,这样消费者就可以消费啦
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
接着分析上篇文章（19. consuemr消费失败后，消息如何处理？）中的，消息消费失败之后，会生成重试消息发送到broker中，然后broker将其转成延时消息（延时级别=3+重试次数）存储在commitlog中，reputservice会将其分配到topic:SCHEDULE_TOPIC_XXXX、queueId=delayLevel的consumeQueue中，这时consumer就可以再次消费消息了。
