docker下logstash搭建_docker

1.下载镜像

老生常谈，没啥好说的。

 docker pull logstash:7.5.1

2.新建挂载文件

此处不详谈，如果有疑问可以参考上文redis安装，有具体解释。

mkdir -p /usr/local/logstash/conf.d
mkdir -p /usr/local/logstash/config
mkdir -p /usr/local/logstash/logs

3.赋权

chmod -777 /usr/local/logstash

4.挂载配置文件

1.挂载配置文件

将logstash.yml放入/usr/local/logstash/config/中，在容器启动后，使用的就是该文件配置。

​logstash.yml​：

http.host: "0.0.0.0"
xpack.monitoring.enabled: true
xpack.monitoring.elasticsearch.hosts: "http://192.168.xx.xx:9200"  #es地址
xpack.monitoring.elasticsearch.username: "elastic"  #es xpack账号
xpack.monitoring.elasticsearch.password: "xxxx"     #es xpack账号
path.config: /usr/share/logstash/config/conf.d/*.conf
path.logs: /usr/share/logstash/logs


注意 http.host: "0.0.0.0" 而不是指定ip


docker下logstash搭建_docker_02

2.挂载日志收集文件

将log_to_es.conf放入/usr/local/logstash/conf.d/，在收集日志时，使用的就是该配置。

3.log_to_es.conf配置

如果想具体了解配置含义，后续楼主会开展elk专栏，请持续关注楼主。

​log_to_es.conf：​

input{
        tcp {
          mode => "server"
        host => "0.0.0.0"
                port => 5000
                codec => json_lines
        type=> "datalog"
        }
    tcp {
          mode => "server"
        host => "0.0.0.0"
                port => 4999
                codec => json_lines
        type=> "loginlog"

        }
}
filter{
      if[type] == "loginlog"{
          grok {
          match => {"message" => "|%{GREEDYDATA:loginMsg}|%{GREEDYDATA:timeFormat}|%{GREEDYDATA:userName}"}
        }
        if([message] =~  "^(?!.*?登录系统).*$") {
          ### 丢弃
          drop{}
           }
        }
      if[type] == "datalog"{  
        grok {
          match => {"message" => "|%{DATA:userName}|%{GREEDYDATA:operationName}|%{DATA:timeFormat}|%{DATA:ip}|%{DATA:systemType}|%{GREEDYDATA:logType}|%{GREEDYDATA:method}|%{GREEDYDATA:input}"}
        } 
      }
            ruby {
        code => "event['time'] = event['@timestamp']"
      }
      mutate
      {
        add_field => ["time", "%{@timestamp}"]
      }        
}
output{

        if[type] == "datalog"{
            elasticsearch{
            hosts=>["192.168.xx.xx:9200"]
            user => "elastic"
            password => "xxxx"
            index => "xxxx-%{+YYYY.MM.dd}"
                }
            }
        if[type] == "loginlog"{
            elasticsearch{
            hosts=>["192.168.xx.xx:9200"]
            user => "elastic"
            password => "xxxx"
            index => "xxxx-%{+YYYY.MM.dd}"
                  }         
                        }
}

5.启动

docker run -p 5044:5044 -p 5000:5000-p 4999:4999--name=logstash \
  --restart=always --privileged=true\
  -e ES_JAVA_OPTS="-Xms1g -Xmx2g" \
  -v /usr/local/logstash/config/logstash.yml:/usr/share/logstash/config/logstash.yml \
  -v /usr/local/logstash/conf.d:/usr/share/logstash/config/conf.d \
  -v /usr/local/logstash/logs:/usr/share/logstash/logs  \
  -d logstash:7.5.1

参数详解：


  • -p 5044:5044 -p 5000:5000-p 4999:4999 ：映射的端口号 这里与上文log_to_es.conf input中一定要相同！！！！额外价格一个5044 为logstash地址
  • --name=logstash：容器名称
  • --restart=always --privileged=true：启动配置
  • -e ES_JAVA_OPTS="-Xms1g -Xmx2g"：指定内存
  • -v /usr/local/logstash/config/logstash.yml:/usr/share/logstash/config/logstash.yml：配置文件挂载
  • -v /usr/local/logstash/conf.d:/usr/share/logstash/config/conf.d：日志收集配置挂载位置
  • -v /usr/local/logstash/logs:/usr/share/logstash/logs：日志挂载位置
  • -d logstash:7.5.1：指定镜像

6.验证

访问kibana可以看到是否连接成功。

docker下logstash搭建_elasticsearch_03