hbase wal 瓶颈 hbase webui

一、实验目的及任务

1.1实验目的

• 认识Servlet框架
• 了解如何连接HBase数据库
• 了解如何用jsp展示数据

 1.2实验任务

1.  安装Tomcat
2. 创建Servlet目录结构
3.  编写utils层代码
4. 编写entity层代码
5.  编写dao层代码
6.  编写service层代码
7. 编写action层代码
8. 编写前端jsp页面

二、实验环境

•  Ubuntu16.04系统；
• 用户账号：stu/stu123;管理员：root/root123
• idea-2018.2.8-Linux without JBR
• Tomcat-9.0.54

 三、实验步骤

1.安装Tomcat

tomcat官网下载地址：https://tomcat.apache.org/download-90.cgi

$ sudo tar -zxvf ./apache-tomcat-9.0.70.tar.gz -C  /usr/local

$ cd /usr/local
$ sudo mv ./apache-tomcat-9.0.70 ./apache
$ sudo chown -R zxb:zxb ./apache

访问localhost:8080，检测是否能够正常运行

$ cd /usr/local/apache/bin
$ nohup ./startup.sh &

到火狐浏览器里面访问localhost:8080

 

2.创建Servlet目录结构

进入CallLogWeb模块，并添加Web支持和Maven支持，完整的Maven依赖如下：

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>com.project</groupId>
    <artifactId>CallLogWeb</artifactId>
    <version>1.0-SNAPSHOT</version>
    <repositories>
        <repository>
            <id>alibaba</id>
            <url>https://mvnrepository.com/artifact/</url>
        </repository>
    </repositories>
    <properties>
        <maven.compiler.source>8</maven.compiler.source>
        <maven.compiler.target>8</maven.compiler.target>
    </properties>
    <dependencies>
        <dependency>
            <groupId>org.apache.hbase</groupId>
            <artifactId>hbase-client</artifactId>
            <version>2.2.2</version>
        </dependency>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.11</version>
        </dependency>
        <dependency>
            <groupId>javax.servlet</groupId>
            <artifactId>javax.servlet-api</artifactId>
            <version>3.1.0</version>
        </dependency>
        <dependency>
            <groupId>com.alibaba</groupId>
            <artifactId>fastjson</artifactId>
            <version>1.2.78</version>
        </dependency>
        <dependency>
            <groupId>mysql</groupId>
            <artifactId>mysql-connector-java</artifactId>
            <version>8.0.15</version>
        </dependency>
    </dependencies>
</project>

然后在java目录下，创建如下的包：

1.编写utils层代码

Util是utiliy的缩写，是一个多功能、基于工具的包。如字符串处理、日期处理等,(建立数据库之间的连 接），例如与HBase、mysql等服务器的连接

在utils下创建HBaseUtils.java

package com.project.utils;

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.client.Connection;
import org.apache.hadoop.hbase.client.ConnectionFactory;
public class HbaseUtils {
    private static Configuration configuration = null;
    private static Connection connection = null;
    //static 静态代码块，加载类时执行一次且只执行一次
    static {
        try {
            configuration = new Configuration();
//zookeeper服务器的地址
            configuration.set("hbase.zookeeper.quorum","localhost:2181");
            connection = ConnectionFactory.createConnection(configuration);
        }catch (Exception e){
            e.printStackTrace();
        }
    }
    public static Connection getConnection(){
        return connection;
    }
}

2.编写entity层代码

entity层主要存放封装的数据结构类

在entity下创建CallLog.java

package com.project.entity;

import java.text.SimpleDateFormat;
/**
 * 通话记录的全部信息
 */
public class CallLog {
    private String caller ;
    private String callee ;
    private String callDate ;
    private String callDuration ;
    private boolean flag;
    public boolean isFlag() {
        return flag;
    }
    public void setFlag(boolean flag) {
        this.flag = flag;
    }
    public String getCaller() {
        return caller;
    }
    public void setCaller(String caller) {
        this.caller = caller;
    }
    public String getCallee() {
        return callee;
    }
    public void setCallee(String callee) {
        this.callee = callee;
    }
    public String getCallDate() {
//格式化时间
        try {
            SimpleDateFormat sf1 = new SimpleDateFormat();
            SimpleDateFormat sf2 = new SimpleDateFormat();
            sf1.applyPattern("yyyyMMddHHmmss");
            sf2.applyPattern("yyyy-MM-dd HH:mm:ss");
            return sf2.format(sf1.parse(callDate));
        }catch (Exception e){
            e.printStackTrace();
        }
        return null;
    }
    public void setCallDate(String callDate) {
        this.callDate = callDate;
    }
    public String getCallDuration() {
        return callDuration;
    }
    public void setCallDuration(String callDuration) {
        this.callDuration = callDuration;
    }
}

3.编写dao层代码

dao层叫数据访问层，全称为data access object，属于一种比较底层，比较基础的操作，具体到对于某 个表、某个实体的增删改查，对外提供稳定访问数据库的方法

在dao/callLog下创建CallLogDao.java接口

package com.project.dao.callLog;

import com.project.entity.CallLog;
import java.util.List;
public interface CallLogDao {
    List<CallLog> findCallLogs();
}

创建CallLogDaoImpl.java实现接口

package com.project.dao.callLog;

import com.project.entity.CallLog;
import com.project.utils.HbaseUtils;
import org.apache.hadoop.hbase.TableName;
import org.apache.hadoop.hbase.client.*;
import org.apache.hadoop.hbase.util.Bytes;
import java.util.ArrayList;
import java.util.Iterator;
import java.util.List;
public class CallLogDaoImpl implements CallLogDao{
    @Override
    public List<CallLog> findCallLogs(){
        List<CallLog> list = new ArrayList<CallLog>();
        Connection connection = HbaseUtils.getConnection();
        try {
            HTable table =
                    (HTable)connection.getTable(TableName.valueOf("ns1:calllog"));
            Scan scan = new Scan();
            ResultScanner rs = table.getScanner(scan);
            Iterator<Result> it = rs.iterator();
            boolean flag = false;
            CallLog log = null ;
            while(it.hasNext()){
                log = new CallLog();
                Result r = it.next();
                String rowKey = Bytes.toString(r.getRow());
                String[] row = rowKey.split(",");
                flag = row[3].equals("1");
                log.setCallDate(row[2]);
                log.setCallee(row[4]);
                log.setCaller(row[1]);
                log.setCallDuration(row[5]);
                log.setFlag(flag);
                list.add(log);
            }
            return list ;
        } catch (Exception e) {
            e.printStackTrace();
        }
        return null;
    }
}

4.编写service层代码

Service是管理具体的功能的接口

在service/callLog下创建CallLogService.java接口

package com.project.service.callLog;

import com.project.entity.CallLog;
import java.util.List;
public interface CallLogService {
    List<CallLog> findAllCallLogs();
}

创建CallLogServiceImpl.java实现接口

package com.project.service.callLog;

import com.project.dao.callLog.CallLogDao;
import com.project.dao.callLog.CallLogDaoImpl;
import com.project.entity.CallLog;
import java.util.List;
public class CallLogServiceImpl implements CallLogService{
    private CallLogDao callLogDao = new CallLogDaoImpl();
    @Override
    public List<CallLog> findAllCallLogs(){
        return callLogDao.findCallLogs();
    }
}

5.编写action层代码

action层主要存放Servlet控制器，用于向前端页面传输数据

在action下创建CallLogFindAllServlet.java

package com.project.action;

import com.project.entity.CallLog;
import com.project.service.callLog.CallLogService;
import com.project.service.callLog.CallLogServiceImpl;
import javax.servlet.ServletException;
import javax.servlet.annotation.WebServlet;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.io.IOException;
import java.util.List;
@WebServlet(name = "Controller_getAll", urlPatterns = "/getAll")
public class CallLogFindAllServlet extends HttpServlet {
    //关联service层
    private CallLogService callLogService = new CallLogServiceImpl();
    @Override
    protected void doGet(HttpServletRequest req, HttpServletResponse resp)
            throws ServletException, IOException {
        List<CallLog> callLogs =callLogService.findAllCallLogs();
//将数据库获取的数据保存到request范围内
        req.setAttribute("callLogs", callLogs);
//转发到目标页面，使得数据信息能在前端页面显示
        req.getRequestDispatcher("callLogs.jsp").forward(req,resp);
    }
}

6.编写jsp前端页面

接收到数据后，以循环的方式在列表中展现

在web下创建callLogs.jsp

<%--
  Created by IntelliJ IDEA.
  User: zxb
  Date: 22-12-28
  Time: 下午7:28
  To change this template use File | Settings | File Templates.
--%>
<%@ page import="java.util.ArrayList" %>
<%@ page import="com.project.entity.CallLog" %>
<%@ page import="java.util.List" %>
<%@ page contentType="text/html;charset=UTF-8" language="java" %>
<html>
<head>
    <title>所有通话记录</title>
    <link rel="stylesheet" type="text/css" href="./css/my.css">
</head>
<body>
<%
    List<CallLog> callLogs = (ArrayList)(request.getAttribute("callLogs")); //获取数据
%>
<table id="t1" border="1px" class="t-1" style="width: 800px">
    <tr>
        <td>电话1</td>
        <td>电话2</td>
        <td>主(被)叫</td>
        <td>通话时间</td>
        <td>通话时长</td>
    </tr>
    <%
        for(int i=0; i<callLogs.size(); i++){
            CallLog callLog = callLogs.get(i);
    %>
    <tr>
        <td><%= callLog.getCaller() %></td>
        <td><%= callLog.getCallee() %></td>
        <td><%= callLog.isFlag() %></td>
        <td><%= callLog.getCallDate() %></td>
        <td><%= callLog.getCallDuration() %></td>
    </tr>
    <%
        }
    %>
    <tr>
        <td colspan="7" style="text-align: right">
        </td>
    </tr>
</table>
</body>
</html>

css样式文件

在web/css下创建my.css

body{
    border : 1px solid blue ;
    margin: 0px;
    /*background-color: aquamarine;*/
}
table {
    border: 1px solid cadetblue;
    border-collapse: collapse;
    width: 400px;
    margin: 50px auto;
}
table td {
    text-align: center;
    height: 45px;
}
table tr:nth-child(even) {
    background-color: beige;
}

3.配置idea中的tomcat

点击Run---EDit Configurations...

 点击左侧“+”号，找到Tomcat Server---Local（若是没有找到Tomcat Server 可以点击最后一行 34 items more）

 自定义Name，点击 Configuare，添加本地 Tomcat 路径，修改默认浏览器为火狐

 Tomcat Server 的路径是这样的，这是， 自己安装的tomcat的路径

 

 点击 Deployment，添加项目，点击+号

选择需要添加 tomcat 的项目；右侧 Application content，可以设置访问路径名，这里使用 / （即没有 路径名），读者可自行设置

 （已经启动hdfs和hbase的环境下）点击运行

 在弹出的网页输入url，localhost:8080/getAll

我这里是8080端口被占用了，改了8089端口，不影响

 

 后端简单的数据展示部分成功。

四、额外内容（加分项）

• 利用表单等api实现查询功能，可以使用HBase原生语言，也可以 Hive关联HBase
• 统计数据，进行可视化分析
• 实现网络页面的实时更新