实验

架构图

keepalived多脚本检测配置 keepalived监控nginx脚本不执行_nginx

内容

Keepalived+Nginx实现Nginx负载均衡器的高可用，Nginx反向代理后端的web服务。但是Keepalived对Nginx的健康检查需要自定义vrrp脚本。

vrrp脚本的定义

vrrp_script <SCRIPT_NAME> {
   script <STRING>|<QUOTED-STRING> #要执行的命令或脚本路径
   interval <INTEGER>              #脚本调用时间间隔，默认1秒
   timeout <INTEGER>               #脚本执行等待超时时长，超过该时长，表示脚本执行失败
   weight <INTEGER:-254..254>      #根据该权重值调整vrrp实例优先级值，默认值为0
   rise <INTEGER>                  #需要成功多少次，vrrp才进行角色状态切换
   fall <INTEGER>                  #需要失败多少次，vrrp才进行角色状态切换
}

vrrp监控

#在vrrp_instance内设置
#监控接口，当某接口down掉时，切换为fault状态
track_interface {
    ens33
}
#监控脚本，即执行vrrp_script定义的脚本
track_script {
    <SCRIPT_NAME>
}
环境

ip

系统

软件

角色

192.168.253.128

CentOS7

Keepalived,Nginx

MASTER节点，web服务器

192.168.253.158

CentOS7

Keepalived,Nginx

BACKUP节点，web服务器

第一步：192.168.253.128和192.168.253.158上准备好nginx的web服务以及负载均衡
192.168.253.128和192.168.253.158上都执行下列操作

systemctl stop firewalld 
setenforce  0
#wget -O /etc/yum.repos.d/epel.repo http://mirrors.aliyun.com/repo/epel-7.repo如果已经有epel源不需要执行
yum install nginx  -y  #nginx需要epel源才可以yum安装

修改配置文件/etc/nginx/nginx.conf

worker_processes  1;
events {
    worker_connections  1024;
}
http {
    include       mime.types;
    default_type  application/octet-stream;
    sendfile        on;
    keepalive_timeout  65;
    upstream backend {
         server 192.168.253.128:8080 weight=1;
         server 192.168.253.158:8080 weight=3;
    }
    server {
        listen 80;
        location / {
           proxy_pass http://backend;
        }
    }
    server {
        listen       8080;
        server_name  localhost;
        location / {
            root   html;
            index  index.html index.htm;
        }
        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }
    }
}

第二步：设置各自的首页，方便区分
192.168.253.128主机上

echo "192.168.253.128:8080" > /usr/share/nginx/html/index.html

192.168.253.158主机上

echo "192.168.253.158:8080" > /usr/share/nginx/html/index.html

第三步：测试两台负载均衡器是否正常
启动nginx

systemctl start nginx

keepalived多脚本检测配置 keepalived监控nginx脚本不执行_nginx_02


192.168.253.128这台负载均衡器没问题，权重也是1:3

keepalived多脚本检测配置 keepalived监控nginx脚本不执行_keepalived多脚本检测配置_03


192.168.253.158这台负载均衡器没问题，权重也是1:3

第三步：192.168.253.128上安装keepalived和修改配置文件

yum install keepalived psmisc -y

修改配置文件 /etc/keepalived/keepalived.conf

! Configuration File for keepalived
global_defs {
    notification_email {
         root@localhost
    }
    notification_email_from keepalived@localhost
    smtp_server 127.0.0.1
    smtp_connect_timeout 30
    router_id 192.168.253.128
}

vrrp_script chk_nginx {
    script "killall -0 nginx && exit 0 || exit 1"  #如果存在nginx进程就正常退出，否则返回1（只有返回0才是成功）
    interval 1      #脚本调用时间间隔，1s
    weight -10      #权重降低10
    fall 2          #需要失败2次，vrrp才进行角色状态切换
    rise 2          #需要成功2次，vrrp才进行角色状态切换
}


vrrp_instance VI_1 {
    state MASTER
    interface ens33
    virtual_router_id 15
    priority 95
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1234
    }
    virtual_ipaddress {
        192.168.253.190/32 dev ens33
    }
   track_script {
       chk_nginx  ##监控chk_nginx 
   }
   notify_master "systemctl start nginx" #状态变为MASTER时，触发的脚本
   notify_backup "systemctl start nginx" #状态变为MASTER时，触发的脚本
   notify_fault "systemctl  start nginx" #状态变为fault，触发的脚本
}

第四步：在192.168.253.158上安装keepalived和修改配置文件

yum install keepalived psmisc -y

修改配置文件/etc/keepalived/keepalived.conf

! Configuration File for keepalived
global_defs {
    notification_email {
         root@localhost
    }
    notification_email_from keepalived@localhost
    smtp_server 127.0.0.1
    smtp_connect_timeout 30
    router_id 192.168.253.158
}

vrrp_script chk_nginx {
    script "killall -0 nginx && exit 0 || exit 1"
    interval 1      
    weight -10
    fall 2
    rise 2
}


vrrp_instance VI_1 {
    state BACKUP
    interface ens33
    virtual_router_id 15
    priority 90
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1234
    }
    virtual_ipaddress {
        192.168.253.190/32 dev ens33
    }
   track_script {
       chk_nginx
   }
   notify_master "systemctl start nginx"
   notify_backup "systemctl start nginx"
   notify_fault "systemctl  start nginx"
}

第五步：测试keepalived的健康检查
两台主机运行keepalived

systemctl  start  keepalived

抓包分析

tcpdump -i ens33 -nn -p vrrp

keepalived多脚本检测配置 keepalived监控nginx脚本不执行_nginx_04


这里可以看到192.168.253.128是MASTER节点

这时候我们down掉nginx，看192.168.253.128是否会优先级降低和重启nginx

keepalived多脚本检测配置 keepalived监控nginx脚本不执行_Keepalived+Nginx_05


这里可以看到192.168.253.128的优先级已经下降了。而且192.168.253.158也成为了MASTER节点

keepalived多脚本检测配置 keepalived监控nginx脚本不执行_html_06


而且nginx也重新启动成功了

keepalived多脚本检测配置 keepalived监控nginx脚本不执行_keepalived多脚本检测配置_07


修复成功之后，192.168.253.128的优先级就上升了于是主节点从192.168.253.158变成192.168.253.128第六步：测试nginx的负载均衡

for i in {1..15} ; do curl 192.168.253.190 ; done

keepalived多脚本检测配置 keepalived监控nginx脚本不执行_nginx_08

这时候我们down掉192.168.253.128的keepalived和nginx的负载均衡功能

systemctl stop keepalived 
iptables -A INPUT -p tcp --dport 80 -j REJECT #访问80端口的tcp协议全部拒绝，也就是阻断了192.168.253.128的负载均衡功能

keepalived多脚本检测配置 keepalived监控nginx脚本不执行_keepalived多脚本检测配置_09


然后再访问vip看是否正常

keepalived多脚本检测配置 keepalived监控nginx脚本不执行_nginx_10


可以看到我们down了一台nginx负载均衡器和一台keepalived，但是后端的web服务没有受到影响。