使用Nginx+Lua实现Web项目的灰度发布

Nginx编译安装Lua模块

一、安装LUA环境及相关库
官方网站:https://github.com/openresty/lua-nginx-module
1、LuaJIT
wget http://luajit.org/download/LuaJIT-2.0.2.tar.gz
make && make install PREFIX=/usr/local/LuaJIT

# vim /etc/profile

export LUAJIT_LIB=/usr/local/LuaJIT/lib
export LUAJIT_INC=/usr/local/LuaJIT/include/luajit-2.0

# source /etc/profile

2、下载解压ngx_devel_kit和lua-nginx-module
cd /usr/local/src
wget https://github.com/simpl/ngx_devel_kit/archive/v0.3.0.tar.gz
wget https://github.com/openresty/lua-nginx-module/archive/v0.10.9rc7.tar.gz
分别解压

tar xf v0.10.9rc7.tar.gz 
tar xf v0.3.0.tar.gz 

3、重新编译编译Nginx
cd /usr/local/src/
wget http://nginx.org/download/nginx-1.12.1.tar.gz

[root@node1 src]# tar xf nginx-1.12.1.tar.gz 
[root@node1 src]# cd nginx-1.12.1
预编译
./configure --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib64/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx --with-compat --with-file-aio --with-threads --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --with-http_flv_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_mp4_module --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-mail --with-mail_ssl_module --with-stream --with-stream_realip_module --with-stream_ssl_module --with-stream_ssl_preread_module --with-cc-opt='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -fPIC' --with-ld-opt='-Wl,-z,relro -Wl,-z,now -pie' --add-module=/usr/local/src/ngx_devel_kit-0.3.0 --add-module=/usr/local/src/lua-nginx-module-0.10.9rc7

# 并行编译
make -j 4 && make install

4、加载lua库,加入到ld.so.conf文件
echo "/usr/local/LuaJIT/lib"/etc/ld.so.conf
ldconfig


报错:
[root@node1 nginx-1.12.1]# nginx -v
nginx: error while loading shared libraries: libluajit-5.1.so.2: cannot open shared object file: No such file or directory

解决:
# ln -s /usr/local/LuaJIT/lib/libluajit-5.1.so /usr/lib64/libluajit-5.1.so.2


二、使用Nginx+Lua实现Web项目的灰度发布

灰度发布概念:
按照一定的关系区别,分部分的代码进行上线,使代码的发布能平滑过渡上线
1.使用用户的信息cookie等信息区别
2.根据用户的ip地址区分

灰度发布在百度百科中解释:

灰度发布(又名金丝雀发布)是指在黑与白之间,能够平滑过渡的一种发布方式。在其上可以进行A/B testing,即让一部分用户继续用产品特性A,一部分用户开始用产品特性B,如果用户对B没有什么反对意见,那么逐步扩大范围,把所有用户都迁移到B上面来。灰度发布可以保证整体系统的稳定,在初始灰度的时候就可以发现、调整问题,以保证其影响度。

灰度期:灰度发布开始到结束期间的这一段时间,称为灰度期。

这里用于WEB系统新代码的测试发布,让一部分(IP)用户访问新版本,一部分用户仍然访问正常版本,其原理如图:


具体实现步骤:
1.部署两个tomcat实例:
分别监听 8080和9090 端口

tomcat8080作为灰度环境(升级以后的环境)
tomcat9090作为生产环境(升级之前的环境)

tomcat8080的测试页面代码:
[root@node1 tomcat8080]# cat /data/tomcat8080/webapps/ROOT/java_test.jsp 
<%@ page language="java" import="java.util.*" pageEncoding="utf-8"%>
<HTML>
    <HEAD>
        <TITLE>jsp Test Page</TITLE>
    </HEAD>
    <BODY>
        <%
            Random rand = new Random();
            out.println("<h1>Random number:</h1>");
            out.println("<h1>gray server:</h1>");
            out.println(rand.nextInt(99)+100);
            
        %>
    </BODY>
</HTML>


[root@node1 tomcat8080]# cat /data/tomcat9090/webapps/ROOT/java_test.jsp 
<%@ page language="java" import="java.util.*" pageEncoding="utf-8"%>
<HTML>
    <HEAD>
        <TITLE>jsp Test Page</TITLE>
    </HEAD>
    <BODY>
        <%
            Random rand = new Random();
            out.println("<h1>shengchang xianshang server</h1>");
            out.println("<h1>Random number:</h1>");
            out.println(rand.nextInt(99)+100);
            
        %>
    </BODY>
</HTML>


测试tomcat是否部署正常
2.安装memcached
[root@node1 ~]# yum install -y memcached

[root@node1 conf.d]# memcached -p11211 -u nobody -d
[root@node1 conf.d]# ps -ef|grep memcached
nobody    17177      1  0 18:12 ?        00:00:00 memcached -p11211 -u nobody -d
root      17213  17150  0 18:12 pts/1    00:00:00 grep --color=auto memcached
[root@node1 conf.d]# netstat -tnlp|grep 11211
tcp        0      0 0.0.0.0:11211           0.0.0.0:*               LISTEN      17177/memcached     
tcp6       0      0 :::11211                :::*                    LISTEN      17177/memcached


修改nginx配置文件
[root@node1 conf.d]# pwd
/etc/nginx/conf.d


修改nginx配置,引入lua脚本
经过测试,通过命令引入脚本文件的方式测试无法通过content_by_lua_file /opt/app/dep.lua;
①合并nginx写法,都写入nginx主配置文件:
[root@node1 nginx]# cat nginx.conf
worker_processes  1;

events {

    worker_connections  1024;

}

http {

    include       mime.types;

    default_type  application/octet-stream;

    sendfile        on;

    keepalive_timeout  65;

    proxy_next_upstream     error timeout;

    proxy_redirect          off;

    proxy_set_header        Host $host;

    proxy_set_header        X-Real-IP $http_x_forwarded_for;

    proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;

    client_max_body_size    100m;

    client_body_buffer_size 256k;

    proxy_connect_timeout   180;

    proxy_send_timeout      180;

    proxy_read_timeout      180;

    proxy_buffer_size       8k;

    proxy_buffers           8 64k;

    proxy_busy_buffers_size 128k;

    proxy_temp_file_write_size 128k;

     upstream sc_server {

        server 192.168.3.177:9090;

    }

    upstream gray_server {

        server 192.168.3.177:8080;

    }
    lua_package_path "/usr/local/share/lua/5.1/memcached.lua";

    server {

        listen       80;

        server_name  localhost;

       location / {

       content_by_lua '

            clientIP = ngx.req.get_headers()["X-Real-IP"]

            if clientIP == nil then

                clientIP = ngx.req.get_headers()["x_forwarded_for"]

            end

            if clientIP == nil then

                clientIP = ngx.var.remote_addr

            end

                local memcached = require "resty.memcached"

                local memc, err = memcached:new()

                if not memc then

                    ngx.say("failed to instantiate memc: ", err)

                    return

                end

                local ok, err = memc:connect("127.0.0.1", 11211)

                if not ok then

                    ngx.say("failed to connect: ", err)

                    return

                end

                local res, flags, err = memc:get(clientIP)

                if err then

                    ngx.say("failed to get clientIP ", err)

                    return

                end

                if  res == "1" then

                    ngx.exec("@gray_server")

                    return

                end

                 ngx.exec("@sc_server")

              
               ';

       }

       location @sc_server{

           proxy_pass http://sc_server;

       }

       location @gray_server{

           proxy_pass http://gray_server;

       }

    location /hello {

        default_type 'text/plain';

        content_by_lua 'ngx.say("hello, lua")';

    }

    location /myip {
    default_type 'text/palin';
    content_by_lua '
        clientIP = ngx.req.get_headers()["x_forwarded_for"]
        if clientIP == nil then
            clientIP = ngx.var.remote_addr
        end    
        ngx.say("IP:",clientIP)
        ';
    }

    location = /50x.html {

        root   html;

    }

   }

}


②主配置和lua配置文件分开写
主配置文件
[root@node1 conf.d]# cat /etc/nginx/nginx.conf

user  nginx;
worker_processes  2;

error_log  /var/log/nginx/error.log warn;
pid        /var/run/nginx.pid;


events {
    worker_connections  1024;
}


http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for" "$request_uri"';

    access_log  /var/log/nginx/access.log  main;

    sendfile        on;
    #tcp_nopush     on;

    keepalive_timeout  65;

    #gzip  on;

    include /etc/nginx/conf.d/*.conf;
}

lua的配置文件
[root@node1 conf.d]# cat /etc/nginx/conf.d/gray_lua.conf 

upstream sc_server {
    server 192.168.3.177:9090;
}

upstream gray_server{
    server 192.168.3.177:8080;
}

lua_package_path "/usr/local/share/lua/5.1/memcached.lua";

    server {

        listen       80;

        server_name  localhost;

       location / {

       content_by_lua '

            clientIP = ngx.req.get_headers()["X-Real-IP"]

            if clientIP == nil then

                clientIP = ngx.req.get_headers()["x_forwarded_for"]

            end

            if clientIP == nil then

                clientIP = ngx.var.remote_addr

            end

                local memcached = require "resty.memcached"

                local memc, err = memcached:new()

                if not memc then

                    ngx.say("failed to instantiate memc: ", err)

                    return

                end

                local ok, err = memc:connect("127.0.0.1", 11211)

                if not ok then

                    ngx.say("failed to connect: ", err)

                    return

                end

                local res, flags, err = memc:get(clientIP)

                if err then

                    ngx.say("failed to get clientIP ", err)

                    return

                end

                if  res == "1" then

                    ngx.exec("@gray_server")

                    return

                end

                 ngx.exec("@sc_server")
               ';

       }

       location @sc_server{

           proxy_pass http://sc_server;

       }

       location @gray_server{

           proxy_pass http://gray_server;

       }

    location /hello {

        default_type 'text/plain';

        content_by_lua 'ngx.say("hello, lua")';

    }

    location /myip {
    default_type 'text/palin';
    content_by_lua '
        clientIP = ngx.req.get_headers()["x_forwarded_for"]
        if clientIP == nil then
            clientIP = ngx.var.remote_addr
        end    
        ngx.say("IP:",clientIP)
        ';
    }


    error_page   500 502 503 504  /50x.html;
    location = /50x.html {
        root   /usr/share/nginx/html;
    }

}


3.安装lua_memcached插件
wget https://github.com/agentzh/lua-resty-memcached/archive/v0.11.tar.gz
tar xf v0.11.tar.gz 
cp -r lua-resty-memcached-0.11/lib/resty /usr/share/lua/5.1/

向memcached中插入本机ip数据,作为可以访问灰度环境的IP
[root@node1 conf.d]# telnet 127.0.0.1 11211
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.
set 192.168.3.84 0 0 1
1
STORED
get 192.168.3.84
VALUE 192.168.3.84 0 1
1
END


测试是否可以分离:

不在memcached中的IP访问java_test.jsp

[root@node1 tomcat8080]# ifconfig eth0
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.3.177  netmask 255.255.255.0  broadcast 192.168.3.255
        inet6 fe80::3c78:50fd:7203:2f0e  prefixlen 64  scopeid 0x20<link>
        ether 00:50:56:3b:dc:7e  txqueuelen 1000  (Ethernet)
        RX packets 495041  bytes 239594322 (228.4 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 54942  bytes 10493549 (10.0 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

[root@node1 tomcat8080]# curl http://192.168.3.177/java_test.jsp

<HTML>
    <HEAD>
        <TITLE>jsp Test Page</TITLE>
    </HEAD>
    <BODY>
        <h1>shengchang xianshang server</h1>
<h1>Random number:</h1>
114

    </BODY>
</HTML>

在memcached环境中的IP访问:




lua脚本的基础知识

[root@node1 ~]# yum install -y lua

基本语法:
1运行方式
交互模式运行:
[root@node1 ~]# lua
Lua 5.1.4  Copyright (C) 1994-2008 Lua.org, PUC-Rio
> print("hello lua")
hello lua

写入脚本运行:
[root@node1 ~]# cat test.lua 
#!/usr/bin/lua

print("hello lua")
[root@node1 ~]# lua test.lua 
hello lua

2.注释
--行注释
--[[
块注释
--]]

3.变量

> a = 'aio\n123"'
> print(a)
aio
123"
> a="aio\n123\""
> print(a)
aio
123"
> a='\97io\10\04923"'
> print(a)
aio
123"
> a=[[aio
>> 123"]]
> print(a)
aio
123"

布尔类型只有nil和false是false,数字0 空字符串(' \0' )都是true
lua中的变量如果没有特殊说明,全是全局变量
如果是局部变量前面加Local

4.循环

while循环语句
[root@node1 ~]# cat add.lua 
#!/usr/bin/lua

sum = 0
num = 1

while num <= 100 do
  sum = sum + num
  num = num + 1
end

print("sum=",sum)
[root@node1 ~]# lua add.lua 
sum=    5050

注意:
lua没有++ 或者 += 这样的操作


for循环
[root@node1 ~]# cat for.lua 
#!/usr/bin/lua

sum = 0
for i = 1,100 do
  sum = sum + i
end
print("sum=",sum)
[root@node1 ~]# lua for.lua 
sum=    5050

if-else判断语句(脚本测试不通过)

if age == 40 and sex == "Male" then
  print("大于40男人")
elseif age > 60 and sex ~= "Female" then
  print("非女人且大于60")
else
  local age = io.read()
  print('your age is'..age)
end