1.HTTP请求

yum -y install curl



客户端---------request----->服务端

服务端--------response---->客户端

request包含请求行,请求头部、请求数据

response包含状态行、消息报头、响应正文

使用linux发送http请求:


curl www.baidu.com

如果没有curl的话需要先安装curl


yum -y install curl
[root@VM_69_65_centos ~]# curl www.baidu.com
<!DOCTYPE html>
<!--STATUS OK--><html> <head><meta http-equiv=content-type content=text/html;charset=utf-8><meta http-equiv=X-UA-Compatible content=IE=Edge><meta content=always name=referrer><link rel=stylesheet type=text/css href=http://s1.bdstatic.com/r/www/cache/bdorz/baidu.min.css><title>百度一下,你就知道</title></head> <body link=#0000cc> <div id=wrapper> <div id=head> <div class=head_wrapper> <div class=s_form> <div class=s_form_wrapper> <div id=lg> <img hidefocus=true src=//www.baidu.com/img/bd_logo1.png width=270 height=129> </div> <form id=form name=f action=//www.baidu.com/s class=fm> <input type=hidden name=bdorz_come value=1> <input type=hidden name=ie value=utf-8> <input type=hidden name=f value=8> <input type=hidden name=rsv_bp value=1> <input type=hidden name=rsv_idx value=1> <input type=hidden name=tn value=baidu><span class="bg s_ipt_wr"><input id=kw name=wd class=s_ipt value maxlength=255 autocomplete=off autofocus></span><span class="bg s_btn_wr"><input type=submit id=su value=百度一下 class="bg s_btn"></span> </form> </div> </div> <div id=u1> <a href=http://news.baidu.com name=tj_trnews class=mnav>新闻</a> <a href=http://www.hao123.com name=tj_trhao123 class=mnav>hao123</a> <a href=http://map.baidu.com name=tj_trmap class=mnav>地图</a> <a href=http://v.baidu.com name=tj_trvideo class=mnav>视频</a> <a href=http://tieba.baidu.com name=tj_trtieba class=mnav>贴吧</a> <noscript> <a href=http://www.baidu.com/bdorz/login.gif?login&tpl=mn&u=http%3A%2F%2Fwww.baidu.com%2f%3fbdorz_come%3d1 name=tj_login class=lb>登录</a> </noscript> <script>document.write('<a href="http://www.baidu.com/bdorz/login.gif?login&tpl=mn&u='+ encodeURIComponent(window.location.href+ (window.location.search === "" ? "?" : "&")+ "bdorz_come=1")+ '" name="tj_login" class="lb">登录</a>');</script> <a href=//www.baidu.com/more/ name=tj_briicon class=bri style="display: block;">更多产品</a> </div> </div> </div> <div id=ftCon> <div id=ftConw> <p id=lh> <a href=http://home.baidu.com>关于百度</a> <a href=http://ir.baidu.com>About Baidu</a> </p> <p id=cp>©2017 Baidu <a href=http://www.baidu.com/duty/>使用百度前必读</a>  <a href=http://jianyi.baidu.com/ class=cp-feedback>意见反馈</a> 京ICP证030173号  <img src=//www.baidu.com/img/gs.gif> </p> </div> </div> </div> </body> </html>

这样请求只能看到请求正文,我们想要看到的更多就要用到


curl -v  www.baidu.com


[root@VM_69_65_centos ~]# curl -v  www.baidu.com
* About to connect() to www.baidu.com port 80 (#0)
*  Trying 180.149.131.98...
* Connected to www.baidu.com (180.149.131.98) port 80 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.29.0
> Host: www.baidu.com
> Accept: */*
> 
< HTTP/1.1 200 OK
< Server: bfe/1.0.8.18
< Date: Tue, 16 Jan 2018 06:11:06 GMT
< Content-Type: text/html
< Content-Length: 2381
< Last-Modified: Mon, 23 Jan 2017 13:27:36 GMT
< Connection: Keep-Alive
< ETag: "588604c8-94d"
< Cache-Control: private, no-cache, no-store, proxy-revalidate, no-transform
< Pragma: no-cache
< Set-Cookie: BDORZ=27315; max-age=86400; domain=.baidu.com; path=/
< Accept-Ranges: bytes
< 
<!DOCTYPE html>
<!--STATUS OK--><html> <head><meta http-equiv=content-type content=text/html;charset=utf-8><meta http-equiv=X-UA-Compatible content=IE=Edge><meta content=always name=referrer><link rel=stylesheet type=text/css href=http://s1.bdstatic.com/r/www/cache/bdorz/baidu.min.css><title>百度一下,你就知道</title></head> <body link=#0000cc> <div id=wrapper> <div id=head> <div class=head_wrapper> <div class=s_form> <div class=s_form_wrapper> <div id=lg> <img hidefocus=true src=//www.baidu.com/img/bd_logo1.png width=270 height=129> </div> <form id=form name=f action=//www.baidu.com/s class=fm> <input type=hidden name=bdorz_come value=1> <input type=hidden name=ie value=utf-8> <input type=hidden name=f value=8> <input type=hidden name=rsv_bp value=1> <input type=hidden name=rsv_idx value=1> <input type=hidden name=tn value=baidu><span class="bg s_ipt_wr"><input id=kw name=wd class=s_ipt value maxlength=255 autocomplete=off autofocus></span><span class="bg s_btn_wr"><input type=submit id=su value=百度一下 class="bg s_btn"></span> </form> </div> </div> <div id=u1> <a href=http://news.baidu.com name=tj_trnews class=mnav>新闻</a> <a href=http://www.hao123.com name=tj_trhao123 class=mnav>hao123</a> <a href=http://map.baidu.com name=tj_trmap class=mnav>地图</a> <a href=http://v.baidu.com name=tj_trvideo class=mnav>视频</a> <a href=http://tieba.baidu.com name=tj_trtieba class=mnav>贴吧</a> <noscript> <a href=http://www.baidu.com/bdorz/login.gif?login&tpl=mn&u=http%3A%2F%2Fwww.baidu.com%2f%3fbdorz_come%3d1 name=tj_login class=lb>登录</a> </noscript> <script>document.write('<a href="http://www.baidu.com/bdorz/login.gif?login&tpl=mn&u='+ encodeURIComponent(window.location.href+ (window.location.search === "" ? "?" : "&")+ "bdorz_come=1")+ '" name="tj_login" class="lb">登录</a>');</script> <a href=//www.baidu.com/more/ name=tj_briicon class=bri style="display: block;">更多产品</a> </div> </div> </div> <div id=ftCon> <div id=ftConw> <p id=lh> <a href=http://home.baidu.com>关于百度</a> <a href=http://ir.baidu.com>About Baidu</a> </p> <p id=cp>©2017 Baidu <a href=http://www.baidu.com/duty/>使用百度前必读</a>  <a href=http://jianyi.baidu.com/ class=cp-feedback>意见反馈</a> 京ICP证030173号  <img src=//www.baidu.com/img/gs.gif> </p> </div> </div> </div> </body> </html>
* Connection #0 to host www.baidu.com left intact
映射到空设备上
[root@VM_69_65_centos ~]# curl -v  www.baidu.com > /dev/null
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0* About to connect() to www.baidu.com port 80 (#0)
*   Trying 61.135.169.125...
* Connected to www.baidu.com (61.135.169.125) port 80 (#0)
> GET / HTTP/1.1                  //请求行 GET为请求方法 后面是请求的协议
> User-Agent: curl/7.29.0         //head的内容
> Host: www.baidu.com		  //报文信息
> Accept: */*
> 
< HTTP/1.1 200 OK 		//200状态码 OK报文
< Server: bfe/1.0.8.18
< Date: Tue, 16 Jan 2018 06:12:02 GMT
< Content-Type: text/html
//响应的head的信息
< Content-Length: 2381
< Last-Modified: Mon, 23 Jan 2017 13:27:29 GMT
< Connection: Keep-Alive
< ETag: "588604c1-94d"
< Cache-Control: private, no-cache, no-store, proxy-revalidate, no-transform
< Pragma: no-cache
< Set-Cookie: BDORZ=27315; max-age=86400; domain=.baidu.com; path=/
< Accept-Ranges: bytes
< 
{ [data not shown]
100  2381  100  2381    0     0   134k      0 --:--:-- --:--:-- --:--:--  136k
* Connection #0 to host www.baidu.com left intact
[root@VM_69_65_centos ~]#

2、Nginx日志类型


包括:error.log access_log


error.log记录处理http请求的错误状态以及Nginx本身服务的错误状态。


access_log记录Nginx每一次请求的访问状态




nginx服务器日志相关指令主要有两条,一条是log_format,用来设置日志格式,另外一条是access_log,用来指定日志文件的存放路径、格式和缓存大小,一般在nginx的配置文件中日记配置(/usr/local/nginx/conf/nginx.conf)。

nginx的log_format有很多可选的参数用于指示服务器的活动状态,默认的是:
log_format  access  '$remote_addr - $remote_user [$time_local] "$request" '    '$status $body_bytes_sent "$http_referer" '    '"$http_user_agent" "$http_x_forwarded_for"';




log_format的配置语法:


log_format


Syntax:log_format name[escape=default|json] string ...;


Default:logformat combined"...";


Context:http




我们去查看Nginx的log配置


路径:/etc/nginx/nginx.conf


user  nginx;
worker_processes  1;

error_log  /var/log/nginx/error.log warn;   //配置nginx错误日志的地方 warn是错误日志的级别(warn以上的错误记录)
pid        /var/run/nginx.pid;


events {
    worker_connections  1024;
}


http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';
    //$后面代表Nginx的变量
    access_log  /var/log/nginx/access.log  main;  //access_log的路径 main表示以main的格式来标识日志

    sendfile        on;
    #tcp_nopush     on;

    keepalive_timeout  65;

    #gzip  on;

    include /etc/nginx/conf.d/*.conf;
}



以下为error.log

2018/01/15 20:50:13 [error] 30536#30536: *1 open() "/usr/share/nginx/html/favicon.ico" failed (2: No such file or directory), client: 36.106.176.206, server: localhost, request: "GET /favicon.ico HTTP/1.1", host: "211.159.168.30"
2018/01/16 00:06:45 [error] 30536#30536: *10 open() "/usr/share/nginx/html/hndUnblock.cgi" failed (2: No such file or directory), client: 189.62.120.171, server: localhost, request: "GET /hndUnblock.cgi HTTP/1.1", host: "211.159.168.30"
2018/01/16 00:06:48 [error] 30536#30536: *11 open() "/usr/share/nginx/html/tmUnblock.cgi" failed (2: No such file or directory), client: 189.62.120.171, server: localhost, request: "GET /tmUnblock.cgi HTTP/1.1", host: "211.159.168.30"
2018/01/16 00:06:51 [error] 30536#30536: *12 open() "/usr/share/nginx/html/moo" failed (2: No such file or directory), client: 189.62.120.171, server: localhost, request: "GET /moo HTTP/1.1", host: "211.159.168.30"
2018/01/16 00:06:58 [error] 30536#30536: *14 open() "/usr/share/nginx/html/getcfg.php" failed (2: No such file or directory), client: 189.62.120.171, server: localhost, request: "POST /getcfg.php HTTP/1.1", host: "211.159.168.30"
2018/01/16 00:07:05 [error] 30536#30536: *15 open() "/usr/share/nginx/html/getcfg.php" failed (2: No such file or directory), client: 189.62.120.171, server: localhost, request: "POST /getcfg.php HTTP/1.1", host: "211.159.168.30"
2018/01/16 00:32:05 [error] 30536#30536: *16 open() "/usr/share/nginx/html/azenv.php" failed (2: No such file or directory), client: 95.213.187.190, server: localhost, request: "POST http://check.best-proxies.ru/azenv.php?auth=151603392515&a=PSCN&i=3550455838&p=80 HTTP/1.1", host: "check.best-proxies.ru", referrer: "http://best-proxies.ru/"
2018/01/16 02:18:28 [error] 30536#30536: *19 open() "/usr/share/nginx/html/echo.php" failed (2: No such file or directory), client: 139.162.88.63, server: localhost, request: "GET http://clientapi.ipip.net/echo.php?info=1234567890 HTTP/1.1", host: "clientapi.ipip.net"
2018/01/16 06:07:15 [error] 30536#30536: *24 open() "/usr/share/nginx/html/404/search_children.js" failed (2: No such file or directory), client: 120.132.3.65, server: localhost, request: "GET http://www.qq.com/404/search_children.js HTTP/1.1", host: "www.qq.com"
2018/01/16 06:38:59 [error] 30536#30536: *25 open() "/usr/share/nginx/html/webconfig.ini" failed (2: No such file or directory), client: 205.209.159.44, server: localhost, request: "GET /webconfig.ini HTTP/1.1", host: "211.159.168.30"
2018/01/16 08:20:52 [error] 30536#30536: *26 open() "/usr/share/nginx/html/azenv.php" failed (2: No such file or directory), client: 95.213.187.189, server: localhost, request: "POST http://check.best-proxies.ru/azenv.php?auth=151606205281&a=PSCN&i=3550455838&p=80 HTTP/1.1", host: "check.best-proxies.ru", referrer: "http://best-proxies.ru/"
2018/01/16 10:51:12 [error] 30536#30536: *32 open() "/usr/share/nginx/html/index.action" failed (2: No such file or directory), client: 58.218.201.54, server: localhost, request: "GET /index.action HTTP/1.1", host: "211.159.168.30"
2018/01/16 11:57:11 [error] 30536#30536: *33 open() "/usr/share/nginx/html/manager/html" failed (2: No such file or directory), client: 218.93.201.199, server: localhost, request: "GET /manager/html HTTP/1.1", host: "211.159.168.30:80"
2018/01/16 13:30:24 [error] 30536#30536: *34 open() "/usr/share/nginx/html/forum.php" failed (2: No such file or directory), client: 106.120.160.119, server: localhost, request: "GET /forum.php?mod=forumdisplay&fid=2 HTTP/1.1", host: "211.159.168.30", referrer: "http://211.159.168.30/forum.php?mod=forumdisplay&fid=2"
2018/01/16 13:30:56 [error] 30536#30536: *36 open() "/usr/share/nginx/html/forum.php" failed (2: No such file or directory), client: 220.181.132.198, server: localhost, request: "GET /forum.php?mod=forumdisplay&fid=2 HTTP/1.1", host: "211.159.168.30"
2018/01/16 13:30:56 [error] 30536#30536: *37 open() "/usr/share/nginx/html/forum.php" failed (2: No such file or directory), client: 171.13.14.145, server: localhost, request: "GET /forum.php?mod=forumdisplay&fid=2 HTTP/1.1", host: "211.159.168.30"
2018/01/16 13:31:10 [error] 30536#30536: *38 open() "/usr/share/nginx/html/forum.php" failed (2: No such file or directory), client: 182.118.20.158, server: localhost, request: "GET /forum.php?mod=forumdisplay&fid=2 HTTP/1.1", host: "211.159.168.30"
2018/01/16 13:31:18 [error] 30536#30536: *39 open() "/usr/share/nginx/html/forum.php" failed (2: No such file or directory), client: 182.118.20.145, server: localhost, request: "GET /forum.php?mod=forumdisplay&fid=2 HTTP/1.1", host: "211.159.168.30"
2018/01/16 13:31:23 [error] 30536#30536: *40 open() "/usr/share/nginx/html/forum.php" failed (2: No such file or directory), client: 182.118.20.149, server: localhost, request: "GET /forum.php?mod=forumdisplay&fid=2 HTTP/1.1", host: "211.159.168.30"
2018/01/16 14:40:23 [error] 30536#30536: *41 open() "/usr/share/nginx/html/home.php" failed (2: No such file or directory), client: 106.120.161.66, server: localhost, request: "GET /home.php HTTP/1.1", host: "211.159.168.30", referrer: "http://211.159.168.30/home.php"
2018/01/16 14:40:46 [error] 30536#30536: *42 open() "/usr/share/nginx/html/home.php" failed (2: No such file or directory), client: 101.199.108.53, server: localhost, request: "GET /home.php HTTP/1.1", host: "211.159.168.30"
2018/01/16 14:41:26 [error] 30536#30536: *43 open() "/usr/share/nginx/html/home.php" failed (2: No such file or directory), client: 171.13.14.145, server: localhost, request: "GET /home.php HTTP/1.1", host: "211.159.168.30"
2018/01/16 14:42:00 [error] 30536#30536: *44 open() "/usr/share/nginx/html/home.php" failed (2: No such file or directory), client: 171.13.14.132, server: localhost, request: "GET /home.php HTTP/1.1", host: "211.159.168.30"
2018/01/16 14:42:00 [error] 30536#30536: *45 open() "/usr/share/nginx/html/home.php" failed (2: No such file or directory), client: 171.13.14.151, server: localhost, request: "GET /home.php HTTP/1.1", host: "211.159.168.30"








以下为access.log


36.106.176.206 - - [15/Jan/2018:20:50:13 +0800] "GET /favicon.ico HTTP/1.1" 404 571 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" "-"
36.106.176.206 - - [15/Jan/2018:20:50:14 +0800] "GET / HTTP/1.1" 200 555 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; rv:11.0) like Gecko" "-"
106.120.161.66 - - [15/Jan/2018:20:51:04 +0800] "GET / HTTP/1.1" 200 555 "http://211.159.168.30/" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; Tablet PC 2.0)" "-"
220.181.132.195 - - [15/Jan/2018:20:51:32 +0800] "GET / HTTP/1.1" 200 555 "-" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.125 Safari/537.36" "-"
187.106.44.13 - - [15/Jan/2018:20:54:48 +0800] "t3 12.2.1" 400 173 "-" "-" "-"
189.62.120.171 - - [16/Jan/2018:00:06:45 +0800] "GET /hndUnblock.cgi HTTP/1.1" 404 169 "-" "Wget(linux)" "-"
189.62.120.171 - - [16/Jan/2018:00:06:48 +0800] "GET /tmUnblock.cgi HTTP/1.1" 404 169 "-" "Wget(linux)" "-"
189.62.120.171 - - [16/Jan/2018:00:06:51 +0800] "GET /moo HTTP/1.1" 404 169 "-" "Wget(linux)" "-"
189.62.120.171 - - [16/Jan/2018:00:06:54 +0800] "GET / HTTP/1.1" 200 555 "-" "Wget(linux)" "-"
189.62.120.171 - - [16/Jan/2018:00:06:58 +0800] "POST /getcfg.php HTTP/1.1" 404 169 "-" "Wget(linux)" "-"
189.62.120.171 - - [16/Jan/2018:00:07:05 +0800] "POST /getcfg.php HTTP/1.1" 404 169 "-" "Wget(linux)" "-"
95.213.187.190 - - [16/Jan/2018:00:32:05 +0800] "POST http://check.best-proxies.ru/azenv.php?auth=151603392515&a=PSCN&i=3550455838&p=80 HTTP/1.1" 404 571 "http://best-proxies.ru/" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0)" "-"
205.209.159.44 - - [16/Jan/2018:00:39:29 +0800] "GET / HTTP/1.1" 200 555 "-" "Mozilla/5.0 (compatible; Nmap Scripting Engine; https://nmap.org/book/nse.html)" "-"
47.93.112.232 - - [16/Jan/2018:01:29:16 +0800] "GET / HTTP/1.1" 200 555 "-" "-" "-"
139.162.88.63 - - [16/Jan/2018:02:18:28 +0800] "GET http://clientapi.ipip.net/echo.php?info=1234567890 HTTP/1.1" 404 169 "-" "Go-http-client/1.1" "-"
104.236.182.189 - - [16/Jan/2018:03:02:48 +0800] "GET / HTTP/1.1" 200 555 "-" "Mozilla/5.0 zgrab/0.x" "-"
120.132.3.65 - - [16/Jan/2018:06:07:15 +0800] "GET http://www.qq.com/404/search_children.js HTTP/1.1" 404 571 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.114 Safari/537.36" "-"
120.132.3.65 - - [16/Jan/2018:06:07:15 +0800] "\x04\x01\x00PpTi4\x00" 400 173 "-" "-" "-"
120.132.3.65 - - [16/Jan/2018:06:07:15 +0800] "\x05\x01\x00" 400 173 "-" "-" "-"
205.209.159.44 - - [16/Jan/2018:06:38:59 +0800] "GET /webconfig.ini HTTP/1.1" 404 169 "-" "Mozilla/5.0 (compatible; Nmap Scripting Engine; https://nmap.org/book/nse.html)" "-"
95.213.187.189 - - [16/Jan/2018:08:20:52 +0800] "POST http://check.best-proxies.ru/azenv.php?auth=151606205281&a=PSCN&i=3550455838&p=80 HTTP/1.1" 404 571 "http://best-proxies.ru/" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0)" "-"
47.93.90.175 - - [16/Jan/2018:09:34:09 +0800] "GET / HTTP/1.1" 200 555 "-" "-" "-"
123.59.146.153 - - [16/Jan/2018:10:15:57 +0800] "\x16\x03\x01\x01\x22\x01\x00\x01\x1E\x03\x03\xDA\xF2\x07\x92\x89\xD5\x16\xD0\xA8\x03\xA8\xEA\xDE\x95\xF7\x90\xDF\x98\x11\xB2\x01\xB5v\x0F\x13be\xAF^\xD3\xC56\x00\x00\x88\xC00\xC0,\xC0(\xC0$\xC0\x14\xC0" 400 173 "-" "-" "-"
123.59.146.153 - - [16/Jan/2018:10:16:05 +0800] "USER test +iw test :Test Wuz Here" 400 173 "-" "-" "-"
123.59.146.153 - - [16/Jan/2018:10:16:05 +0800] "GET / HTTP/1.1" 200 555 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:52.0) Gecko/20100101 Firefox/52.0" "-"
58.218.201.54 - - [16/Jan/2018:10:51:12 +0800] "GET / HTTP/1.1" 200 555 "-" "User-Agent:Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; .NET CLR 1.0.3705" "-"
58.218.201.54 - - [16/Jan/2018:10:51:12 +0800] "GET /index.action HTTP/1.1" 404 571 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36" "-"
218.93.201.199 - - [16/Jan/2018:11:57:11 +0800] "GET /manager/html HTTP/1.1" 404 571 "-" "Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.2; WOW64; Trident/6.0)" "-"
106.120.160.119 - - [16/Jan/2018:13:30:24 +0800] "GET /forum.php?mod=forumdisplay&fid=2 HTTP/1.1" 404 571 "http://211.159.168.30/forum.php?mod=forumdisplay&fid=2" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; Tablet PC 2.0)" "-"
220.181.132.180 - - [16/Jan/2018:13:30:41 +0800] "GET / HTTP/1.1" 200 555 "http://211.159.168.30/" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.125 Safari/537.36" "-"
220.181.132.198 - - [16/Jan/2018:13:30:56 +0800] "GET /forum.php?mod=forumdisplay&fid=2 HTTP/1.1" 404 571 "-" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.125 Safari/537.36" "-"
171.13.14.145 - - [16/Jan/2018:13:30:56 +0800] "GET /forum.php?mod=forumdisplay&fid=2 HTTP/1.1" 404 571 "-" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.125 Safari/537.36" "unknown"
182.118.20.158 - - [16/Jan/2018:13:31:10 +0800] "GET /forum.php?mod=forumdisplay&fid=2 HTTP/1.1" 404 571 "-" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.125 Safari/537.36" "unknown"
182.118.20.145 - - [16/Jan/2018:13:31:18 +0800] "GET /forum.php?mod=forumdisplay&fid=2 HTTP/1.1" 404 571 "-" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.125 Safari/537.36" "unknown"
182.118.20.149 - - [16/Jan/2018:13:31:23 +0800] "GET /forum.php?mod=forumdisplay&fid=2 HTTP/1.1" 404 571 "-" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.125 Safari/537.36" "unknown"
106.120.161.66 - - [16/Jan/2018:14:40:23 +0800] "GET /home.php HTTP/1.1" 404 571 "http://211.159.168.30/home.php" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; Tablet PC 2.0)" "-"
101.199.108.53 - - [16/Jan/2018:14:40:46 +0800] "GET /home.php HTTP/1.1" 404 571 "-" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.125 Safari/537.36" "-"
171.13.14.145 - - [16/Jan/2018:14:41:26 +0800] "GET /home.php HTTP/1.1" 404 571 "-" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.125 Safari/537.36" "unknown"
171.13.14.132 - - [16/Jan/2018:14:42:00 +0800] "GET /home.php HTTP/1.1" 404 571 "-" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.125 Safari/537.36" "unknown"
171.13.14.151 - - [16/Jan/2018:14:42:00 +0800] "GET /home.php HTTP/1.1" 404 571 "-" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.125 Safari/537.36" "unknown"
95.76.223.173 - - [16/Jan/2018:15:09:35 +0800] "GET / HTTP/1.0" 200 555 "-" "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)" "-"



3、Nginx变量

Http请求变量 


arg_PARAMETER(请求参数)、http_HEADER(请求头)、sent_http_HEADER(服务端返回的response的响应头)


举个例子:


[root@VM_69_65_centos ~]# curl -v www.baidu.com >/dev/null
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0* About to connect() to www.baidu.com port 80 (#0)
*   Trying 220.181.112.244...
* Connected to www.baidu.com (220.181.112.244) port 80 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.29.0 //这里证明request使用的是curl,用的是7.29.0的版本进行的请求
如果想要让日志记录User-Agent我们需要修改nginx.conf(/etc/nginx/nginx.conf)配置文件
http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;


    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

//修改上面这一段,下面为修改后的结果
修改时需要注意每一个字母都要小写,而且-要改为下划线_
log_format  main  '$http_user_agent' '$remote_addr - $remote_user [$time_local] "$request" '           '$status $body_bytes_sent "$http_referer" '
          '"$http_user_agent" "$http_x_forwarded_for"';


    access_log  /var/log/nginx/access.log  main;


    sendfile        on;
    #tcp_nopush     on;


    keepalive_timeout  65;


    #gzip  on;


    include /etc/nginx/conf.d/*.conf;
}

> Host: www.baidu.com
> Accept: */*
> 
< HTTP/1.1 200 OK
< Server: bfe/1.0.8.18
< Date: Tue, 16 Jan 2018 07:38:26 GMT
< Content-Type: text/html
< Content-Length: 2381
< Last-Modified: Mon, 23 Jan 2017 13:27:36 GMT
< Connection: Keep-Alive
< ETag: "588604c8-94d"
< Cache-Control: private, no-cache, no-store, proxy-revalidate, no-transform
< Pragma: no-cache
< Set-Cookie: BDORZ=27315; max-age=86400; domain=.baidu.com; path=/
< Accept-Ranges: bytes
< 
{ [data not shown]
100  2381  100  2381    0     0   119k      0 --:--:-- --:--:-- --:--:--  122k
* Connection #0 to host www.baidu.com left intact
[root@VM_69_65_centos ~]# 

    
    
     
     


之后我们使用
[root@VM_69_65_centos ~]# nginx -t -c /etc/nginx/nginx.conf
查看日志是否正确
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful之后我们再使用curl请本机127.0.0.1然后我们查看access.log











    access_log  /var/log/nginx/access.log  main;


    sendfile        on;
    #tcp_nopush     on;


    keepalive_timeout  65;


    #gzip  on;


    include /etc/nginx/conf.d/*.conf;
}

> Host: www.baidu.com
> Accept: */*
> 
< HTTP/1.1 200 OK
< Server: bfe/1.0.8.18
< Date: Tue, 16 Jan 2018 07:38:26 GMT
< Content-Type: text/html
< Content-Length: 2381
< Last-Modified: Mon, 23 Jan 2017 13:27:36 GMT
< Connection: Keep-Alive
< ETag: "588604c8-94d"
< Cache-Control: private, no-cache, no-store, proxy-revalidate, no-transform
< Pragma: no-cache
< Set-Cookie: BDORZ=27315; max-age=86400; domain=.baidu.com; path=/
< Accept-Ranges: bytes
< 
{ [data not shown]
100  2381  100  2381    0     0   119k      0 --:--:-- --:--:-- --:--:--  122k
* Connection #0 to host www.baidu.com left intact
[root@VM_69_65_centos ~]# 
[root@VM_69_65_centos ~]# nginx -t -c /etc/nginx/nginx.conf
查看日志是否正确
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful


127.0.0.1 - - [16/Jan/2018:15:52:02 +0800] "GET / HTTP/1.1" 200 555 "-" "curl/7.29.0" "-"
127.0.0.1 - - [16/Jan/2018:15:52:09 +0800] "GET / HTTP/1.1" 200 555 "-" "curl/7.29.0" "-"
127.0.0.1 - - [16/Jan/2018:15:52:10 +0800] "GET / HTTP/1.1" 200 555 "-" "curl/7.29.0" "-"
127.0.0.1 - - [16/Jan/2018:15:52:10 +0800] "GET / HTTP/1.1" 200 555 "-" "curl/7.29.0" "-"

之后我们发现配置生效





内置变量-Nginx内置


nginx的官网查看


http://nginx.org/en/docs/
http://nginx.org/en/docs/syslog.html
http://nginx.org/en/docs/http/ngx_http_log_module.html#access_log



之后我们来看一下nginx默认的log_format有哪些


log_format  main  '$http_user_agent' '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

参数

说明

示例

$remote_addr

客户端地址

211.28.65.253

$remote_user

客户端用户名称不开启认证模块没用

--

$time_local

访问时间和时区

18/Jul/2012:17:00:01 +0800

$request

请求的URI和HTTP协议

"GET /article-10000.html HTTP/1.1"

$http_host

请求地址,即浏览器中你输入的地址(IP或域名)


192.168.100.100

$status

HTTP请求状态

200

$upstream_status

upstream状态

200

$body_bytes_sent

发送给客户端文件内容大小

1547

$http_referer

url跳转来源(上一级页面是哪个,做防盗链可用)

https://www.baidu.com/

$http_user_agent

用户终端浏览器等信息(可以记录客户端用什么来访问的,比如说IE,curl)

"Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; SV1; GTB7.0; .NET4.0C;

$ssl_protocol

SSL协议版本

TLSv1

$ssl_cipher

交换数据中的算法

RC4-SHA

$upstream_addr

后台upstream的地址,即真正提供服务的主机地址

10.10.10.100:80

$request_time

整个请求的总时间

0.205

$upstream_response_time

请求过程中,upstream响应时间

0.002





自定义变量 -自己定义



以后再说。


四、Nginx模块讲解

Nginx官方模块


     Nginx中文文档:http://www.nginx.cn/doc/ 




   

[root@VM_69_65_centos ~]# nginx -V
查看nginx编译信息
nginx version: nginx/1.12.2
built by gcc 4.8.5 20150623 (Red Hat 4.8.5-16) (GCC) 
built with OpenSSL 1.0.2k-fips  26 Jan 2017
TLS SNI support enabled
configure arguments: --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib64/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx --with-compat --with-file-aio --with-threads --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --with-http_flv_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_mp4_module --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-mail --with-mail_ssl_module --with-stream --with-stream_realip_module --with-stream_ssl_module --with-stream_ssl_preread_module --with-cc-opt='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -fPIC' --with-ld-opt='-Wl,-z,relro -Wl,-z,now -pie'



--with-http_stub_status_module  Nginx的客户端状态



配置:


Syntax:stub_status;


Default:-(无默认)


Context:server,location(基于server或location下进行配置)



进入/etc/nginx/nginx.d/default.conf(默认配置) 其实这里配置的是访问路径,我们把这个模块放入了访问路径中了。




server {
    listen       80;
    server_name  localhost;

    #charset koi8-r;
    #access_log  /var/log/nginx/host.access.log  main;

    location / {
        root   /usr/share/nginx/html;
        index  index.html index.htm;
    }
location /zjlstatus{ //这里的是自定义名
   stub_status;
}
    #error_page  404              /404.html;

    # redirect server error pages to the static page /50x.html
    #
    error_page   500 502 503 504  /50x.html;
    location = /50x.html {
        root   /usr/share/nginx/html;
    }

    # proxy the PHP scripts to Apache listening on 127.0.0.1:80
    #
    #location ~ \.php$ {
    #    proxy_pass   http://127.0.0.1;
    #}

    # pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
    #
    #location ~ \.php$ {
    #    root           html;
    #    fastcgi_pass   127.0.0.1:9000;
    #    fastcgi_index  index.php;
    #    fastcgi_param  SCRIPT_FILENAME  /scripts$fastcgi_script_name;
    #    include        fastcgi_params;
    #}

    # deny access to .htaccess files, if Apache's document root
    # concurs with nginx's one
    #
    #location ~ /\.ht {
    #    deny  all;
    #}
}


nginx -tc /etc/nginx/nginx.conf        


然后我们重载服务


nginx -s reload -c /etc/nginx/nginx.conf


之后我们打开我们的服务器路径


http://211.159.XXX.XX/zjlstatus


Active connections: 5    //nginx当前活跃的连接数
server accepts handled requests 
 62 62 51  //第一个表示nginx握手的总的次数,第二个标识nginx所处理的连接数, 最后一个标识请求数
Reading: 0 Writing: 1 Waiting: 4 //第一个标识读的个数,第二个表示写的数目,第三个标识等待的数目



-with-http_random_index_module 目录中选择一个随机主页




Syntax:random_index on|off;


Default: random_index off;(默认off)


Context:local(只能配置在local下)


 



继续修改default.conf


location / {
        root   /usr/share/nginx/html;
        index  index.html index.htm;
    }




修改后为

location / {
        root   /opt/app/code;
        #index  index.html index.htm; #以后还是用正经注释吧。这里先把主页注释掉
	#之后添加随机主页
	random_index on;
    }





nginx -tc /etc/nginx/nginx.conf
systemctl reload rsyslog.service
systemctl reload nginx
nginx -s reload -c /etc/nginx/nginx.conf



目录下要加入三个主页(html),用于测试


我们进入这个目录并且将其设置为家目录(~)



/opt/app/code/
pwd

最后我们访问我们的主页,就会发现主页是随机的了。


这里需要注意,随机主页不能是以.开头的文件(.开头文件在linux中算是隐藏文件)


--with-http_sub_module HTTP内容替换



列举一些常见的使用:



Syntax:sub_filter string(替换前的字符串) replacement(替换后的字符串);



Default:-



Context:http,server,location





Syntax:sub_filter_last_modified on|off  (校验服务端的内容是否有发生过变更,判断是否有更新)



Default: sub_filter_last_modified off;



Context:http,server,location





Syntax:sub_filter_once on|off  (on只匹配第一个,OFF匹配所有指定内容)



Default: sub_filter_once on;



Context:http,server,location





location / {
        root   /opt/app/code;
        index  index.html index.htm;
		
	sub_filter '要替换的'  '替换成什么';
sub_filter_last_modified on;#检测更新
sub_filter_once off;#全局替换
}



Nginx的请求限制



连接频率限制 -limit_conn_ module



请求频率限制 -limit_req_ module





HTPP协议的请求与连接



HTTP1.0TCP不能复用



HTTP1.1顺序性TCP复用



HTTP2.0多路复用TCP复用




HTTP请求建立在一次TCP连接基础上


一次TCP请求至少产生一次HTTP请求


1.连接限制


Syntax:limit_conn_zone key zone=name:size;


Default:-


Context:http



Syntax:limit_conn zone number;


Default:-


Context:http,server,location



2.请求限制


Syntax:limit_req_zone key zone=name:size rate=rate;



Default:-



Context:http





Syntax:limit_req zone=name [burst=number] [nodelay];



Default:-



Context:http,server,location





limit_conn_zone $binary_remote_addr zone=conn_zone:1m;
limit_req_zone $binary_remtoe_addr zone=req_zone:1m rate=1r/s; #一个ip地址的客户端1秒允许发送一个请求 1MB
server {
    listen       80;
    server_name  localhost;

    #charset koi8-r;
    #access_log  /var/log/nginx/host.access.log  main;

    location / {
        root   /opt/app/code;
        index  index.html index.htm;
		#random_index on;
	#limit_conn conn_zone 1;
	#limit_req zone=req_zone burst=3 nodelay;
	#limit_req zone=req_zone burst=3;
	#limit_req zone=req_zone;
    }
	location /zjlstatus{
		stub_status;
	}	

    #error_page  404              /404.html;

    # redirect server error pages to the static page /50x.html
    #
    error_page   500 502 503 504  /50x.html;
    location = /50x.html {
        root   /usr/share/nginx/html;
    }

    # proxy the PHP scripts to Apache listening on 127.0.0.1:80
    #
    #location ~ \.php$ {
    #    proxy_pass   http://127.0.0.1;
    #}

    # pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
    #
    #location ~ \.php$ {
    #    root           html;
    #    fastcgi_pass   127.0.0.1:9000;
    #    fastcgi_index  index.php;
    #    fastcgi_param  SCRIPT_FILENAME  /scripts$fastcgi_script_name;
    #    include        fastcgi_params;
    #}

    # deny access to .htaccess files, if Apache's document root
    # concurs with nginx's one
    #
    #location ~ /\.ht {
    #    deny  all;
    #}
}



Nginx的访问控制



基于IP的访问控制 -http_access_module



允许



Syntax:allow address(IP地址) | CIDR(网段) |unix(scoket访问)|all(所有);



Default:-



Context:http,server,location,limit_except





不允许



Syntax:deny address(IP地址) | CIDR(网段) |unix(scoket访问)|all(所有);



Default:-



Context:http,server,location,limit_except








基于用户的信任登录 - http_auth_basic_module



default.conf
server {
    listen       80;
    server_name  localhost;

    #charset koi8-r;
    #access_log  /var/log/nginx/host.access.log  main;

 location / {
        root   /opt/app/code;
        #index  admin.html;
    }
	
location ~^/admin.html {
        root   /opt/app/code;
	deny   36.106.4.102;
	deny   36.106.4.103;
	deny   36.106.4.104;
	allow all;
    }

    #error_page  404              /404.html;

    # redirect server error pages to the static page /50x.html
    #
    error_page   500 502 503 504  /50x.html;
    location = /50x.html {
        root   /usr/share/nginx/html;
    }

    # proxy the PHP scripts to Apache listening on 127.0.0.1:80
    #
    #location ~ \.php$ {
    #    proxy_pass   http://127.0.0.1;
    #}

    # pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
    #
    #location ~ \.php$ {
    #    root           html;
    #    fastcgi_pass   127.0.0.1:9000;
    #    fastcgi_index  index.php;
    #    fastcgi_param  SCRIPT_FILENAME  /scripts$fastcgi_script_name;
    #    include        fastcgi_params;
    #}

    # deny access to .htaccess files, if Apache's document root
    # concurs with nginx's one
    #
    #location ~ /\.ht {
    #    deny  all;
    #}
}
nginx -t -c /etc/nginx/nginx.conf
nginx -s reload -c /etc/nginx/nginx.conf


http_access_module具有局限性






http_x_forwarded_for




nginx修改日志路径_html



http_x_forwarder_for =Client IP, Proxy(1) IP ,Proxy(2) IP,...




如何解决局限性?


方法一、采用别的HTTP头信息控制访问,如HTTP_X_FORWARD_FOR


方法二、结合geo模块作


方法三、通过HTTP自定义变量传递



http_auth_basic_module  基于用户的信任登录


Syntax:auth_basic string(表示开启,并在前端显示这个字符串)|off;


Default:   auth_basic off;
  
Context:http,server,location,limit_except



Syntax:auth_basic_user_file file(文件,文件的配置文件,用于存储帐号密码信息的);


Default:--


Context:http,server,location,limit_except



首先生成一个存用户密码的文件:


[root@VM_69_65_centos nginx]# htpasswd -c ./auth_conf jeson
New password: 
Re-type new password: 
Adding password for user jeson
生成的auth_conf文件中的内容:jeson:$apr1$XzHJd5JC$ueCc/zNNgHaU8FOQPgyee/

局限性:

用户信息依赖文件方式


操作管理机械,效率低下


3解决方案


Nginx结合LUA实现高校验证


Nginx和LDAP打通,利用nginx-auth-ldap模块



第三方模块


暂不介绍。