redisson sentinel 配置 redis.sentinel.master

前言

本文为记述工作中用到redis读写分离时怎样确保高效可靠性等问题的demo演示过程，描述中有不恰当地方，欢迎各位不吝赐教。

主从介绍

主从模式（master-slaves）主要实现数据的读写分离，master负责处理写入请求，读取请求交给slaves处理，slaves同时也处理数据dump等操作，这样可以提升服务器吞吐量，但再高可用上表现不佳，其原因在于当master down下线后，slaves无法成为主。关于主从间数据一致性问题本文不详细赘述，感兴趣的同学可以查阅了解一下。主从结构图如下

---

哨兵sentinel模式

Redis主从模式这个弊端即slave不能提升为主，在sentinel出现后，有效的解决了这个问题。sentinel相当于是一个投票者或者哨兵，它时刻监视着redis集群的各个服务器，当主master挂了之后，它将进行投票进行新master的选举。下图

Master OK

Master NG

Sentinel的工作方式

• 每个Sentinel以每秒钟一次的频率向它所知的Master，Slave以及其他 Sentinel 实例发送一个 PING 命令
• 如果一个实例（instance）距离最后一次有效回复 PING 命令的时间超过 down-after-milliseconds 选项所指定的值， 则这个实例会被 Sentinel 标记为主观下线。
• 如果一个Master被标记为主观下线，则正在监视这个Master的所有 Sentinel 要以每秒一次的频率确认Master的确进入了主观下线状态。
• 当有足够数量的 Sentinel（大于等于配置文件指定的值）在指定的时间范围内确认Master的确进入了主观下线状态， 则Master会被标记为客观下线
• 在一般情况下， 每个 Sentinel 会以每 10 秒一次的频率向它已知的所有Master，Slave发送 INFO 命令
• 当Master被 Sentinel 标记为客观下线时，Sentinel 向下线的 Master 的所有 Slave 发送 INFO 命令的频率会从 10 秒一次改为每秒一次
• 若没有足够数量的 Sentinel 同意 Master 已经下线， Master 的客观下线状态就会被移除。
  若 Master 重新向 Sentinel 的 PING 命令返回有效回复， Master 的主观下线状态就会被移除。

主观下线和客观下线

• 主观下线（Subjectively Down， 简称 SDOWN）指的是单个 Sentinel 实例对服务器做出的下线判断。
• 客观下线（Objectively Down， 简称 ODOWN）指的是多个 Sentinel 实例在对同一个服务器做出 SDOWN 判断， 并且通过 SENTINEL is-master-down-by-addr 命令互相交流之后， 得出的服务器下线判断。
  客观下线条件只适用于主服务器： 对于任何其他类型的 Redis 实例， Sentinel 在将它们判断为下线前不需要进行协商， 所以从服务器或者其他 Sentinel 永远不会达到客观下线条件。
  只要一个 Sentinel 发现某个主服务器进入了客观下线状态， 这个 Sentinel 就可能会被其他 Sentinel 推选出， 并对失效的主服务器执行自动故障迁移操作。

环境准备

• Redis 3.2.100 (GitHub release)
• StackExchange.redis 1.2.6 (1.2.6)
• .Net framework 4.5
• vs2019
• VM工作机一台

环境搭建&执行

本环境将创建1个master,2个slaves,3个sentinels

2. 安装redis
   根据上面链接，下载widowns版本64位redis 3.2.100，默认安装即可（C:\Program Files\Redis）



3. 配置config文件
   配置说明 本文使用的配置文件，，由于是在一台机器上测试，为了方便区分，给每个config所在folder里都copy了一份redis程序（步骤1安装除 *.conf的所有文件）。

Role	IP	Port	Used Configuration File
Master	127.0.0.1	11111	redis.windows.conf
Slave1	127.0.0.1	12111	redis.windows.conf
Slave2	127.0.0.1	13111	redis.windows.conf
Sentinel1	127.0.0.1	11121	sentinel.conf
Sentinel2	127.0.0.1	11131	sentinel.conf
Sentinel3	127.0.0.1	11141	sentinel.conf
master和slave配置文件除了port不同外，slave中包含slaveof，其他项目设置相同。

• 配置主redis

• bind 127.0.0.1
port 11111
requirepass 123456
masterauth 123456           这个也是需要的，主变成从会用到

• 配置从redis

• bind 127.0.0.1
port 12111
requirepass 123456
slaveof 17.0.0.1 11111
masterauth 123456

配置哨兵也是通过配安文件配置的。但是redis安装包里面没有配置文件项，需要自己创建或者在源码项目中找到这个配置文件。

sentinel3.2

# Example sentinel.conf

# *** IMPORTANT ***
#
# By default Sentinel will not be reachable from interfaces different than
# localhost, either use the 'bind' directive to bind to a list of network
# interfaces, or disable protected mode with "protected-mode no" by
# adding it to this configuration file.
#
# Before doing that MAKE SURE the instance is protected from the outside
# world via firewalling or other means.
#
# For example you may use one of the following:
#
# bind 127.0.0.1 192.168.1.1
#
# protected-mode no

# port <sentinel-port>
# The port that this sentinel instance will run on
port 26379

# sentinel announce-ip <ip>
# sentinel announce-port <port>
#
# The above two configuration directives are useful in environments where,
# because of NAT, Sentinel is reachable from outside via a non-local address.
#
# When announce-ip is provided, the Sentinel will claim the specified IP address
# in HELLO messages used to gossip its presence, instead of auto-detecting the
# local address as it usually does.
#
# Similarly when announce-port is provided and is valid and non-zero, Sentinel
# will announce the specified TCP port.
#
# The two options don't need to be used together, if only announce-ip is
# provided, the Sentinel will announce the specified IP and the server port
# as specified by the "port" option. If only announce-port is provided, the
# Sentinel will announce the auto-detected local IP and the specified port.
#
# Example:
#
# sentinel announce-ip 1.2.3.4

# dir <working-directory>
# Every long running process should have a well-defined working directory.
# For Redis Sentinel to chdir to /tmp at startup is the simplest thing
# for the process to don't interfere with administrative tasks such as
# unmounting filesystems.
dir /tmp

# sentinel monitor <master-name> <ip> <redis-port> <quorum>
#
# Tells Sentinel to monitor this master, and to consider it in O_DOWN
# (Objectively Down) state only if at least <quorum> sentinels agree.
#
# Note that whatever is the ODOWN quorum, a Sentinel will require to
# be elected by the majority of the known Sentinels in order to
# start a failover, so no failover can be performed in minority.
#
# Slaves are auto-discovered, so you don't need to specify slaves in
# any way. Sentinel itself will rewrite this configuration file adding
# the slaves using additional configuration options.
# Also note that the configuration file is rewritten when a
# slave is promoted to master.
#
# Note: master name should not include special characters or spaces.
# The valid charset is A-z 0-9 and the three characters ".-_".
sentinel monitor mymaster 127.0.0.1 6379 2

# sentinel auth-pass <master-name> <password>
#
# Set the password to use to authenticate with the master and slaves.
# Useful if there is a password set in the Redis instances to monitor.
#
# Note that the master password is also used for slaves, so it is not
# possible to set a different password in masters and slaves instances
# if you want to be able to monitor these instances with Sentinel.
#
# However you can have Redis instances without the authentication enabled
# mixed with Redis instances requiring the authentication (as long as the
# password set is the same for all the instances requiring the password) as
# the AUTH command will have no effect in Redis instances with authentication
# switched off.
#
# Example:
#
# sentinel auth-pass mymaster MySUPER--secret-0123passw0rd

# sentinel down-after-milliseconds <master-name> <milliseconds>
#
# Number of milliseconds the master (or any attached slave or sentinel) should
# be unreachable (as in, not acceptable reply to PING, continuously, for the
# specified period) in order to consider it in S_DOWN state (Subjectively
# Down).
#
# Default is 30 seconds.
sentinel down-after-milliseconds mymaster 30000

# sentinel parallel-syncs <master-name> <numslaves>
#
# How many slaves we can reconfigure to point to the new slave simultaneously
# during the failover. Use a low number if you use the slaves to serve query
# to avoid that all the slaves will be unreachable at about the same
# time while performing the synchronization with the master.
sentinel parallel-syncs mymaster 1

# sentinel failover-timeout <master-name> <milliseconds>
#
# Specifies the failover timeout in milliseconds. It is used in many ways:
#
# - The time needed to re-start a failover after a previous failover was
#   already tried against the same master by a given Sentinel, is two
#   times the failover timeout.
#
# - The time needed for a slave replicating to a wrong master according
#   to a Sentinel current configuration, to be forced to replicate
#   with the right master, is exactly the failover timeout (counting since
#   the moment a Sentinel detected the misconfiguration).
#
# - The time needed to cancel a failover that is already in progress but
#   did not produced any configuration change (SLAVEOF NO ONE yet not
#   acknowledged by the promoted slave).
#
# - The maximum time a failover in progress waits for all the slaves to be
#   reconfigured as slaves of the new master. However even after this time
#   the slaves will be reconfigured by the Sentinels anyway, but not with
#   the exact parallel-syncs progression as specified.
#
# Default is 3 minutes.
sentinel failover-timeout mymaster 180000

# SCRIPTS EXECUTION
#
# sentinel notification-script and sentinel reconfig-script are used in order
# to configure scripts that are called to notify the system administrator
# or to reconfigure clients after a failover. The scripts are executed
# with the following rules for error handling:
#
# If script exits with "1" the execution is retried later (up to a maximum
# number of times currently set to 10).
#
# If script exits with "2" (or an higher value) the script execution is
# not retried.
#
# If script terminates because it receives a signal the behavior is the same
# as exit code 1.
#
# A script has a maximum running time of 60 seconds. After this limit is
# reached the script is terminated with a SIGKILL and the execution retried.

# NOTIFICATION SCRIPT
#
# sentinel notification-script <master-name> <script-path>
# 
# Call the specified notification script for any sentinel event that is
# generated in the WARNING level (for instance -sdown, -odown, and so forth).
# This script should notify the system administrator via email, SMS, or any
# other messaging system, that there is something wrong with the monitored
# Redis systems.
#
# The script is called with just two arguments: the first is the event type
# and the second the event description.
#
# The script must exist and be executable in order for sentinel to start if
# this option is provided.
#
# Example:
#
# sentinel notification-script mymaster /var/redis/notify.sh

# CLIENTS RECONFIGURATION SCRIPT
#
# sentinel client-reconfig-script <master-name> <script-path>
#
# When the master changed because of a failover a script can be called in
# order to perform application-specific tasks to notify the clients that the
# configuration has changed and the master is at a different address.
# 
# The following arguments are passed to the script:
#
# <master-name> <role> <state> <from-ip> <from-port> <to-ip> <to-port>
#
# <state> is currently always "failover"
# <role> is either "leader" or "observer"
# 
# The arguments from-ip, from-port, to-ip, to-port are used to communicate
# the old address of the master and the new address of the elected slave
# (now a master).
#
# This script should be resistant to multiple invocations.
#
# Example:
#
# sentinel client-reconfig-script mymaster /var/redis/reconfig.sh

sentinel3.2

1. 启动master-slave-sentinel
   启动顺序依次为 master -> slave -> sentinel，这里我们都以service的形式启动。
   双击运行install.bat脚本，将会创建对应的service，其中哨兵以sentinel模式启动，启动log可以通过设置logfile指定文件名。

logfile "server.log"

1. 我们可以在运行窗口打开以下程序查看service是否创建成功。

services.msc

2. 如果启动没有问题，将会创建6个networkservice，如下图



3. 验证

1. Master11111文件夹下，双击运行client.bat启动client窗口连接master，输入info replication查看master状态

我们可以看到角色为master，已连接两个slaves(port分别为12111,13111)

1. Slave12111文件夹下，双击运行client.bat启动client窗口连接slave1，输入info replication查看slave1状态

role:slave

master_* 为连接master信息及状态

1. Slave13111文件夹下，双击运行client.bat启动client窗口连接slave2，输入info replication查看slave2状态

role:slave

master_* 为连接master信息及状态如果设定了logfile "server.log"，在对应folder下创建的server.log文件中，也能够确认连接状态，形如

1. C# Demo 解压后打开工程，packages.config中依赖StackExchange.Redis版本为1.2.6，如果工程提示没找到该包，可以利用Nuget 在packages folder下寻找包文件，执行Install-Package <package file path>。
   Demo里主要干了三件事

• 连接master-slave
• 连接sentinels
• 写入当前时间到指定key，并读取

1. 这里列出连接sentinel模式的代码，该代码监听master change事件

//connnet sentinel code
	ConfigurationOptions sentineloption = new ConfigurationOptions();
        sentineloption.TieBreaker = "";//sentinel模式一定要写

        //三个哨兵
        sentineloption.EndPoints.Add("127.0.0.1", 11121);
        sentineloption.EndPoints.Add("127.0.0.1", 11131);
        sentineloption.EndPoints.Add("127.0.0.1", 11141);

        //哨兵连接模式
        sentineloption.CommandMap = CommandMap.Sentinel; // 需要指定为sentinel
        sentineloption.ServiceName = "mymaster";
        //我们可以成功的连接一个sentinel服务器，对这个连接的实际意义在于：当一个主从进行切换后，如果它外层有Twemproxy代理，我们可以在这个时机（+switch-master事件)通知你的Twemproxy代理服务器，并更新它的配置文件里的master服务器的地址，然后从起你的Twemproxy服务，这样你的主从切换才算真正完成。
        //一般没有代理服务器，直接更改从数据库配置文件，将其升级为主数据库。
        _sentinel = ConnectionMultiplexer.Connect(sentineloption);
        sentinelsub = _sentinel.GetSubscriber();

        sentinelsub.Subscribe("+switch-master", (ch, mg) =>
        {
            Console.WriteLine((string)mg);
        });

① -> ② -> ③顺序点击，得到右边console内容，表明代码连接操作redis成功

接下来，我们模拟master宕机（停止当前master对应的service即可），看一下sentinel从主切换结果

等待几秒左右（等待过程中，如果有写入请求，master还没选出来，会有数据遗漏？），console里输出一行信息mymaster 127.0.0.1 11111 127.0.0.1 13111，代表master已经由11111切换到13111上，这是我们监听"+switch-master"得到的。
至此，demo演示完毕。