使用nginx+keepalived实现高可用的存储方案

nginx集群心跳检测配置_nginx

前言

需求与目的

  • 通过keepalived来映射虚拟ip(vip),监测nginx心跳活动完成热备
  • 当主机(192.168.4.136)的nginx服务宕机后keepalived则会执行nginx心跳监测脚本,如果发现nginx服务宕机则尝试重新启动,若无法正常重启,则会停止keepalived服务并将vip交给从机(192.168.4.137)
  • 使用nginx做负载均衡以及反向代理

一、安装环境

Centos7.5 :
192.168.4.136 nginx1 主
192.168.4.137 nginx2 从
192.168.232.161 vip(虚拟ip)

Nginx版本: 1.20.2
Keepalived版本: 2.2.7

二、安装nginx

1. 安装编译工具及库文件

[root@pt49 /]# yum -y install make zlib zlib-devel gcc-c++ libtool openssl openssl-devel

2.关闭防火墙

systemctl stop firewalld #关闭防火墙
systemctl disable firewalld # 永久关闭防火墙
systemctl status firewalld #查看防火墙状态

安装nginx

首先进入nginx官网下载目前的稳定版
或者进入链接下载我目前使用的nginx1.20.2版本(蓝奏云网盘)

https://wwn.lanzoul.com/iXly2009g5qh
密码:46ns

nginx集群心跳检测配置_nginx_02

1. 下载nginx并将nginx放在opt下进行解压

[root@pt49 /]# cd opt/
[root@pt49 opt]# tar -zxvf nginx-1.20.2.tar.gz

2.进入安装目录

[root@pt49 opt]# cd nginx-1.20.2/

3.编译安装

[root@pt49 nginx-1.20.2]# ./configure
[root@pt49 nginx-1.20.2]# make & make install

4.查看nginx版本

[root@pt49 nginx-1.20.2]# /usr/local/nginx/sbin/nginx -v

nginx集群心跳检测配置_负载均衡_03


5.将nginx配置为服务

创建nginx.service文件

vim /usr/lib/systemd/system/nginx.service

#nginx.service内容

[Unit]
Description=nginx - high performance web server
Documentation=http://nginx.org/en/docs/
After=network.target remote-fs.target nss-lookup.target
 
[Service]
Type=forking
PIDFile=/usr/local/nginx/logs/nginx.pid
ExecStartPre=/usr/local/nginx/sbin/nginx -t -c /usr/local/nginx/conf/nginx.conf
ExecStart=/usr/local/nginx/sbin/nginx -c /usr/local/nginx/conf/nginx.conf
ExecReload=/bin/kill -s HUP $MAINPID
ExecStop=/bin/kill -s QUIT $MAINPID
PrivateTmp=true
 
[Install]
WantedBy=multi-user.target
systemctl enable nginx.service #开启服务
systemctl daemon-reload #初始化
chkconfig --add nginx  #将nginx服务添加到开机自启动
chkconfig nginx  on  #打开nginx自启动

以上就是nginx的安装
7.启动命令

systemctl start nginx #启动nginx
systemctl stop nginx #停止nginx
systemctl status nginx #查看nginx状态
systemctl restart nginx #重启nginx

三、配置nginx

nginx的配置文件为解压目录下的conf/nginx.conf
由于只用测试主从服务器之间的跳转所以针对应用的负载均衡不用做调整,所以nginx不需要配置

四、安装keepalived

进入官网下载keepalived

或者下载我正在使用的keepalived2.2.7版本(蓝奏云网盘)

https://wwn.lanzoul.com/ioEmP009g5md
密码:hcfk

nginx集群心跳检测配置_linux_04

1. 下载keepalived并将keepalived放在opt下进行解压

[root@pt49 /]# cd opt/
[root@pt49 opt]# tar -zxvf keepalived-2.2.7.tar.gz

2.进入安装目录

[root@pt49 opt]# cd keepalived-2.2.7/

3.配置keepalived，使用如下命令

[root@pt49 opt]# ./configure --prefix=/usr/local/keepalived  --sysconf=/etc

命令解释

  • —prefix ：keepalived安装目录
  • –sysconf ：keepalived的核心配置文件，必须要在 /etc 目录下面，固定位置，改为其他
    位置会导致启动不了不在该目录下的话，会在启动keepalived的时候，会在日志文件里面抱一个错误，说找不到/etc这个文件夹。

如果在过程中出现如下错误

nginx集群心跳检测配置_nginx集群心跳检测配置_05


执行以下命令

[root@pt49 opt]# yum -y install libnl libnl-devel

然后再执行

[root@pt49 opt]# ./configure --prefix=/usr/local/keepalived  --sysconf=/etc

4.编译和安装keepalived

[root@pt49 opt]# make && make install

5.使用如下命令可查看keepalived安装在那儿

[root@pt49 opt]# whereis keepalived

nginx集群心跳检测配置_linux_06


/usr/local/keepalived ：是之前指定的安装目录位置

五、配置keepalived

1.进入核心配置下

[root@pt49 opt]# cd /etc/keepalived

2.创建keepalived配置文件

[root@pt49 opt]# vim keepalived.conf

!!!keepalived.conf文件具体内容图文解释!!

nginx集群心跳检测配置_负载均衡_07

keepalived.conf文件具体内容 (主机配置)

! Configuration File for keepalived

global_defs {
   # ---路由ID全局唯一
   router_id Keep1
}

vrrp_script chk_nginx {
   # ---nginx状态监测脚本路径
    script "/etc/keepalived/check_nginx.sh"
    interval 5
    weight -20
}

vrrp_instance VI_1 {
    # ---主服务器配置:MASTER，备服务器:BACKUP
    state MASTER
    # ---当前实例绑定的网卡 可以根据ifconfig命令进行查看
    interface ens32
    # ---主备服务器VRID号必须一致
    virtual_router_id 51
    # ---服务器优先级,备服务器填小
    priority 150
    # ---主备之间同步检查心跳的间隔时间默认为一秒
    advert_int 1
    # ---主备服务器密码必须一致
    authentication {
        auth_type PASS
        auth_pass a1b2c3d4
    }
    # ---谁是主服务器谁获得该VIP
    virtual_ipaddress {
        192.168.232.161
    }
	track_script {
       chk_nginx
    }
}

keepalived.conf文件具体内容 (从机配置)

! Configuration File for keepalived

global_defs {
   router_id Keep2
}

vrrp_script chk_nginx {
    script "/etc/keepalived/check_nginx.sh"
    interval 5
    weight -20
}

vrrp_instance VI_1 {
    state BACKUP
    interface ens32
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass a1b2c3d4
    }
    virtual_ipaddress {
        192.168.232.161
    }
	track_script {
       chk_nginx
    }
}

nginx心跳监测脚本
在/etc/keepalived/目录下创建check_nginx.sh
check_nginx.sh文件内容如下

#!/bin/bash
A=`ps -C nginx --no-header | wc -l`
if [ $A -eq 0 ];then
    /usr/local/nginx/sbin/nginx
    sleep 2
    if [ `ps -C nginx --no-header | wc -l` -eq 0 ];then
        pkill keepalived
    fi
fi

！！！需要注意的是如果在运行check_nginx.sh时报错“坏的解释器：没有那个文件和目录则执行以下命令”

错误原因：这个文件在Windows 下编辑过，在Windows下每一行结尾是\n\r，而Linux下则是\n，所以才会有 多出来的\r。

sed -i 's/\r$//' check_nginx.sh

记得给脚本文件赋予权限

chmod 777 check_nginx.sh

3.将keepalived配置为系统服务
首先进入到原本的解压目录，我之前是在opt下的所以进入/opt/keepalived-2.2.7/keepalived/etc

[root@pt49 /]# cd /opt/keepalived-2.2.7/keepalived/etc/

拷贝两个文件到目标路径,如果已经存在则覆盖

[root@pt49 etc]# cp init.d/keepalived /etc/init.d/
[root@pt49 etc]# cp sysconfig/keepalived /etc/sysconfig/

4.初始化相关配置

systemctl enable keepalived.service #开启服务
systemctl daemon-reload #初始化
chkconfig --add keepalived #将keepalived服务添加到开机自启动
chkconfig keepalived on  #打开keepalived自启动

5.相关启动命令

systemctl start keepalived #启动keepalived 
systemctl stop keepalived #停止keepalived 
systemctl status keepalived #查看keepalived 状态
systemctl restart keepalived #重启keepalived