系统版本:

[root@node1 ~]# cat /etc/redhat-release

CentOS Linux release 7.4.1708 (Core)

Redis下载：

https://download.redis.io/releases/

redis-4.0.14.tar.gz

twemproxy官网：

https://github.com/twitter/twemproxy

twemproxy软件包下(可能需要外网):

wget https://github.com/twitter/twemproxy/releases/download/0.5.0/twemproxy-0.5.0.tar.gz

解压安装twemproxy:

tar zxf twemproxy-0.5.0.tar.gz

编译报错，没有安装gcc:

./configure

报错:

configure: error: in `/root/twemproxy/twemproxy-0.5.0':

configure: error: no acceptable C compiler found in $PATH

报错处理:

yum install gcc make -y

Redis集群(twemproxy+haproxy)架构_twemproxy

再次编译安装:

mkdir /usr/local/twemproxy

./configure --prefix=/usr/local/twemproxy/

Redis集群(twemproxy+haproxy)架构_twemproxy_02

编译完成，直接安装:

# make -j 4 && make install

Redis集群(twemproxy+haproxy)架构_twemproxy_03

Tar包下复制nutcracker.yml文件到安装目录下:

[root@master conf]# pwd

/root/twemproxy/twemproxy-0.5.0/conf

cp nutcracker.yml /usr/local/twemproxy/

安装redis:

安装步骤忽略,太简单。

redis编译报错:

In file included from adlist.c:34:0:

zmalloc.h:50:31: fatal error: jemalloc/jemalloc.h: No such file or directory

 #include <jemalloc/jemalloc.h>

错误的本质是我们在开始执行make 时遇到了错误gcc未安装,然后我们安装好了gcc 后，我们再执行make ,这时就出现了jemalloc/jemalloc.h: No such file or directory 编译失败，有残留的文件，我们需要清理下，然后重新编译就可以了.

解决办法两种:

第一种:

make MALLOC=libc && make install

第二种:

make distclean
make && make install

修改nutcracker.yml配置文件:

[root@master twemproxy]# cat nutcracker.yml

redis_test:

  listen: 0.0.0.0:22121

  hash: fnv1a_64

  distribution: ketama

  auto_eject_hosts: true

  redis: true

  #redis_auth: mldnjava

  server_retry_timeout: 2000

  server_failure_limit: 1

  servers:

    - 10.40.42.105:6379:1

    - 10.40.42.127:6379:1

启动twemproxy服务前，校验文件配置:

# /usr/local/twemproxy/sbin/nutcracker -t /usr/local/twemproxy/nutcracker.yml

Redis集群(twemproxy+haproxy)架构_Redis集群twemproxy_04

查看twemproxy启动命令并启动服务:

[root@master sbin]# /usr/local/twemproxy/sbin/nutcracker -c /usr/local/twemproxy/nutcracker.yml -o /usr/local/twemproxy/twemproxy.log -d

Redis集群(twemproxy+haproxy)架构_Redis集群twemproxy_05

查看nutcracker服务进程:

[root@master sbin]# ps -ef | grep nutc

root     114525      1  0 00:03 ?        00:00:00 /usr/local/twemproxy/sbin/nutcracker -c /usr/local/twemproxy/nutcracker.yml -o /usr/local/twemproxy/twemproxy.log -d

root     114704  91662  0 00:04 pts/0    00:00:00 grep --color=auto nutc

查看nutcracker服务端口:

[root@master twemproxy]# netstat -antlp | grep 22121

tcp        0      0 0.0.0.0:22121           0.0.0.0:*               LISTEN      114525/nutcracker

测试

连接Twemproxy新建一个key,node1 redis客户端直接查询：

Twemproxy新建一个key:

10.40.42.103:22121> set hello world

OK

Redis集群(twemproxy+haproxy)架构_Redis集群twemproxy_06

 

Haproxy+ twemproxy架构:

部署2台twemproxy，代理相同的redis服务：

Node2上的twemproxy服务:

[root@node2 twemproxy]# cat nutcracker.yml

redis_test:

  listen: 0.0.0.0:22121

  hash: fnv1a_64

  distribution: ketama

  auto_eject_hosts: true

  redis: true

  #redis_auth: redhat

  server_retry_timeout: 2000

  server_failure_limit: 1

  servers:

    - 10.40.42.105:6379:1

    - 10.40.42.127:6379:1

 

Master上的twemproxy服务:

[root@master twemproxy]# cat nutcracker.yml

redis_test:

  listen: 0.0.0.0:22121

  hash: fnv1a_64

  distribution: ketama

  auto_eject_hosts: true

  redis: true

  #redis_auth: redhat

  server_retry_timeout: 2000

  server_failure_limit: 1

  servers:

    - 10.40.42.105:6379:1

    - 10.40.42.127:6379:1

安装haproxy服务:

[root@node1 ~]# yum install haproxy -y

修改haproxy.cfg配置文件:

[root@node1 ~]# cat /etc/haproxy/haproxy.cfg | grep -v "^#" | grep -v "^[ ]\+#"
global
    log         127.0.0.1 local2
    chroot      /var/lib/haproxy
    pidfile     /var/run/haproxy.pid
    maxconn     4000
    user        haproxy
    group       haproxy
    daemon

    stats socket /var/lib/haproxy/stats

defaults
    mode                    http
    log                     global
    option                  httplog
    option                  dontlognull
    option http-server-close
    option forwardfor       except 127.0.0.0/8
    option                  redispatch
    retries                 3
    timeout http-request    10s
    timeout queue           1m
    timeout connect         10s
    timeout client          1m
    timeout server          1m
    timeout http-keep-alive 10s
    timeout check           10s
    maxconn                 3000

listen  twemproxy
    bind 0.0.0.0:5666
    option tcplog
    mode tcp
    balance     roundrobin
    server  app1 10.40.42.103:22121  check inter 10s
    server  app2 10.40.42.127:22121  check inter 10s

listen admin_stats
    bind *:11001
    mode http
    stats refresh 30s
    stats uri /admin
    stats realm welcome login\ Haproxy
    stats auth admin:admin123

    stats admin if TRUE # 通过设置此选项，可以在监控页面上手工启用或禁用后端真实服务器

Redis集群(twemproxy+haproxy)架构_Redis集群twemproxy_07

启动haproxy服务，并查看进程:

[root@node1 ~]# systemctl start haproxy
[root@node1 ~]# ps -ef | grep hapro
root      79089      1  0 05:05 ?        00:00:00 /usr/sbin/haproxy-systemd-wrapper -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid
haproxy   79090  79089  0 05:05 ?        00:00:00 /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid -Ds
haproxy   79091  79090  0 05:05 ?        00:00:00 /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid -Ds
root      79102  49336  0 05:05 pts/1    00:00:00 grep --color=auto hapro

浏览器访问:

http://10.40.42.105:11001/admin

Redis集群(twemproxy+haproxy)架构_redis_08

测试:

通过haproxy代理连接新建一个key:

[root@node1 bin]# ./redis-cli -h 10.40.42.105 -p 5666
10.40.42.105:5666> set mew happy
OK

Redis集群(twemproxy+haproxy)架构_redis_09

Node2本地redis客户端连接获取key:

[root@node2 bin]# ./redis-cli 
127.0.0.1:6379> get mew
"happy"

Redis集群(twemproxy+haproxy)架构_haproxy_10