hbase开源项目 hbase开发实例

转载

编程小达 2023-08-18 22:01:26

文章标签 hbase开源项目 hbase hadoop apache maven 文章分类 Hbase 数据库

HBase实现谷粒微博案例

前言
一、启动集群
二、功能实现

1.创建工程
2.constants包
3.utils包

3.1 createNameSpace 创建命名空间
3.2 isTableExist 判断表是否存在
3.3 createTable 创建表

4.dao包

4.1 发微博功能
4.2 关注功能
4.3 取消关注
4.4 获得用户初始页
4.5 获得用户全部微博内容

5 test包测试

总结
参考

前言

最近刚刚在b站上看完了尚硅谷hbase相关课程，在这里记录一下，完成这个项目遇到的坑，以及案例结果(而且不要吐槽的英语注释，我是个英语渣，大家尽量意会一下…)。如果博客有什么不对的地方，欢迎大佬指正。

一、启动集群

由于HBase是基于hadoop的一个非结构化数据库，所以需要启动hadoop集群，并且还需要启动我们的“润滑器” zookeeper。
我使用的版本是
CentOS 6
Hadoop 3
HBase 3
Zookeeper 3 （每一个虚拟机都要启动zookeeper）
虚拟机一共三台，分别是hadoop102，hadoop103，hadoop104
编译器使用的是IDEA。

二、功能实现

1.创建工程

这里我只是给出代码以及一些相关解释，具体的需求分析以及更加详细的解释，可以看后面的参考链接，我将会给出b站尚硅谷视频的链接，有兴趣的可以去看看。
打开IDEA，创建一个新的maven工程，然后进行pom.xml格式如下。这里是需要导入相关的依赖包，以及各种依赖的版本，build一下就会开始进行下载。

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <parent>
        <artifactId>HBase</artifactId>
        <groupId>org.example</groupId>
        <version>1.0-SNAPSHOT</version>
    </parent>
    <modelVersion>4.0.0</modelVersion>

    <artifactId>guli-weibo</artifactId>
    <dependencies>
        <dependency>
            <groupId>org.apache.hbase</groupId>
            <artifactId>hbase-server</artifactId>
            <version>1.3.1</version>
        </dependency>
        <dependency>
            <groupId>org.apache.hbase</groupId>
            <artifactId>hbase-client</artifactId>
            <version>1.3.1</version>
        </dependency>
    </dependencies>

    <properties>
        <maven.compiler.source>8</maven.compiler.source>
        <maven.compiler.target>8</maven.compiler.target>
    </properties>

</project>

然后在工程中创建5个文件夹（其中一个是空的，也可以不创建），后续需要添加代码。如图：

hbase开源项目 hbase开发实例_hbase

ok，现在我们就可以快乐的编写代码了。

2.constants包

在这里我们定义的是一些常量，例如是命名空间、表明、列名等，以便后面我们需要使用的时候反复重写，防止写错，也可以进行一个解耦合。

package constants;

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.*;

public class Constants {


    public static Configuration CONFIGURATION = null;

    // HBase setting 这里不知道为什么要这要写才可以，视频里面没有写这个也可以运行，
    // 但是我就会报错，说无法连接，希望有大佬可以解释一下。
    // 那个set里面第二个放的是你们集群的名称，可能会不一样，看你们自己的命名了。
    static {
        CONFIGURATION = HBaseConfiguration.create();
        CONFIGURATION.set("hbase.zookeeper.quorum", "hadoop102,hadoop103,hadoop104");
    }

    // HBase namespace 这个是命名空间
    public static final String NAMESPACE = "weibo";

    // weibo content table 这个是内容表
    public static final String CONTENT_TABLE = "weibo:content";
    public static String CONTENT_TABLE_CF = "info";
    public static int CONTENT_TABLE_VERSIONS = 1;

    // user relationship table 这个是关系表
    public static final String RELATION_TABLE = "weibo:relation";
    public static final String RELATION_TABLE_CF1 = "attends";
    public static final String RELATION_TABLE_CF2 = "fans";
    public static final int RELATION_TABLE_VERSIONS = 1;

    // receipt email box 这个是收件表
    public static final String INBOX_TABLE = "weibo:inbox";
    public static final String INBOX_TABLE_CF = "info";
    public static final int INBOX_TABLE_VERSIONS = 2;

}

3.utils包

这里主要是进行DDL的操作，例如创建命名空间，以及对表之类的操作。
代码总览，下面再分段解释：

package utils;

import constants.Constants;
import org.apache.hadoop.hbase.HColumnDescriptor;
import org.apache.hadoop.hbase.HTableDescriptor;
import org.apache.hadoop.hbase.NamespaceDescriptor;
import org.apache.hadoop.hbase.TableName;
import org.apache.hadoop.hbase.client.Admin;
import org.apache.hadoop.hbase.client.Connection;
import org.apache.hadoop.hbase.client.ConnectionFactory;
import org.apache.hadoop.hbase.client.Table;

import java.io.IOException;

/**
 * 1 create namespace
 * 2 judge table exist or not
 * 3 create three tables
 * 4
 */

public class HBaseUtil {

    // 1 create namespace
    public static void createNameSpace(String namespace) throws IOException {

        // 1 get the connection object
        Connection connection = ConnectionFactory.createConnection(Constants.CONFIGURATION);

        // 2 get the admin object
        Admin admin = connection.getAdmin();

        // 1 create namespace description
        NamespaceDescriptor build = NamespaceDescriptor.create(namespace).build();

        // 2 create namespace
        admin.createNamespace(build);

        // 3 close resource
        admin.close();
        connection.close();
    }

    // 2 judge the table exist or not
    private static boolean isTableExist(String tableName) throws IOException {

        // 1 get the connection object
        Connection connection = ConnectionFactory.createConnection(Constants.CONFIGURATION);

        // 2 get the admin object
        Admin admin = connection.getAdmin();

        // 3 judge the table exist or not
        boolean result = admin.tableExists(TableName.valueOf(tableName));

        // 4 close resource
        admin.close();
        connection.close();

        // return value
        return result;
    }

    // 3 create table
    public static void createTable(String tableName, int versions, String... cfs) throws IOException {

        // 1 judge the command value
        if (cfs.length <= 0) {
            System.out.println("column information is not available.");
            return ;
        }

        // 2 judge the table exist or not
        if (isTableExist(tableName)) {
            System.out.println(tableName + "table is exist.");
            return;
        }

        // 3 get the connection object
        Connection connection = ConnectionFactory.createConnection(Constants.CONFIGURATION);

        // 4 get the admin object
        Admin admin = connection.getAdmin();

        // 5 create table description
        HTableDescriptor hTableDescriptor = new HTableDescriptor(TableName.valueOf(tableName));

        // 6 cycle add column information
        for (String cf : cfs) {

            HColumnDescriptor hColumnDescriptor = new HColumnDescriptor(cf);
            // 7 set the version
            hColumnDescriptor.setMaxVersions(versions);

            hTableDescriptor.addFamily(hColumnDescriptor);
        }

        // 8 create table
        admin.createTable(hTableDescriptor);

        // close resource
        admin.close();
        connection.close();
    }
}

3.1 createNameSpace 创建命名空间

// 1 create namespace
    public static void createNameSpace(String namespace) throws IOException {

        // 1 get the connection object 创建一个连接的对象
        Connection connection = ConnectionFactory.createConnection(Constants.CONFIGURATION);

        // 2 get the admin object 创建一个admin的对象，
        // 对hbase操作基本上通admin来操作
        Admin admin = connection.getAdmin();

        // 1 create namespace description 创建一个命名空间的解释器
        NamespaceDescriptor build = NamespaceDescriptor.create(namespace).build();

        // 2 create namespace 创建命名空间
        admin.createNamespace(build);

        // 3 close resource 关闭资源
        admin.close();
        connection.close();
    }

3.2 isTableExist 判断表是否存在

// 2 judge the table exist or not
    private static boolean isTableExist(String tableName) throws IOException {

        // 1 get the connection object
        Connection connection = ConnectionFactory.createConnection(Constants.CONFIGURATION);

        // 2 get the admin object
        Admin admin = connection.getAdmin();

        // 3 judge the table exist or not. 
        // tableExists这个函数是需要传入TableName参数的，所以需要转换。
        boolean result = admin.tableExists(TableName.valueOf(tableName));

        // 4 close resource
        admin.close();
        connection.close();

        // return value
        return result;
    }

3.3 createTable 创建表

这一块蛮重要的，但是也是固定的套路。

// 3 create table
    public static void createTable(String tableName, int versions, String... cfs) throws IOException {

        // 1 judge the command value 判断一下传入的参数是否小于等于0
        if (cfs.length <= 0) {
            System.out.println("column information is not available.");
            return ;
        }

        // 2 judge the table exist or not 判断表是否存在
        if (isTableExist(tableName)) {
            System.out.println(tableName + "table is exist.");
            return;
        }

        // 3 get the connection object
        Connection connection = ConnectionFactory.createConnection(Constants.CONFIGURATION);

        // 4 get the admin object
        Admin admin = connection.getAdmin();

        // 5 create table description 
        // 创建表描述器，对表的操作都是通过描述器来操作。
        HTableDescriptor hTableDescriptor = new HTableDescriptor(TableName.valueOf(tableName));

        // 6 cycle add column information
        for (String cf : cfs) {
			// 这里是创建一个列描述器
            HColumnDescriptor hColumnDescriptor = new HColumnDescriptor(cf);
            // 7 set the version
            hColumnDescriptor.setMaxVersions(versions);
			// 将列描述器加入到表描述器当中，也就是把列族加进去。
            hTableDescriptor.addFamily(hColumnDescriptor);
        }

        // 8 create table
        admin.createTable(hTableDescriptor);

        // close resource
        admin.close();
        connection.close();
    }

4.dao包

这里是一些DML的操作，例如一些在表中插入数据，或者是删除数据的操作。
老规矩，先来一个代码总览(比较长…)：

package dao;


import constants.Constants;
import org.apache.hadoop.hbase.Cell;
import org.apache.hadoop.hbase.CellUtil;
import org.apache.hadoop.hbase.TableName;
import org.apache.hadoop.hbase.client.*;
import org.apache.hadoop.hbase.filter.CompareFilter;
import org.apache.hadoop.hbase.filter.RowFilter;
import org.apache.hadoop.hbase.filter.SubstringComparator;
import org.apache.hadoop.hbase.util.Bytes;

import java.io.IOException;
import java.util.ArrayList;

/**
 * 1 send the weibo
 * 2 delete the weibo
 * 3 focus the user
 * 4 check off user
 * 5 get the user's weibo detail
 * 6 get the user's initialize page
 */
public class HBaseDao {

    // 1 send the weibo
    public static void publishWeiBo(String uid, String content) throws IOException {

        // 1 get the connection object
        Connection connection = ConnectionFactory.createConnection(Constants.CONFIGURATION);

        // 2 get the admin object
        Admin admin = connection.getAdmin();

        // 1 operate weibo content table
        // 1.1 get the weibo content table object
        Table conTable = connection.getTable(TableName.valueOf(Constants.CONTENT_TABLE));

        // 1.2 get the current time
        long ts = System.currentTimeMillis();

        // 1.3 get the Rowkey
        String rowKey = uid + "_" + ts;

        // 1.4 create put object
        Put conPut = new Put(Bytes.toBytes(rowKey));

        // 1.5 Assignment the put object
        conPut.addColumn(Bytes.toBytes(Constants.CONTENT_TABLE_CF), Bytes.toBytes("content"), Bytes.toBytes(content));

        // 1.6 execute insert data operation
        conTable.put(conPut);

        // 2 operate weibo email box
        // 2.1 get user relation table object
        Table relaTable = connection.getTable(TableName.valueOf(Constants.RELATION_TABLE));

        // 2.2 get the fans's information of the person who publish weibo in current
        Get get = new Get(Bytes.toBytes(uid));
        get.addFamily(Bytes.toBytes(Constants.RELATION_TABLE_CF2));
        Result result = relaTable.get(get);

        // 2.3 create a map, in order to store the put object of weibo content table
        ArrayList<Put> inboxPuts = new ArrayList<>();

        // 2.4 traverse the fans list
        for (Cell cell : result.rawCells()) {

            // 2.5 create the put object of weibo infobox table
            Put inboxPut = new Put(CellUtil.cloneQualifier(cell));

            // 2.6 assignment the information to the inbox table
            inboxPut.addColumn(Bytes.toBytes(Constants.INBOX_TABLE_CF), Bytes.toBytes(uid), Bytes.toBytes(rowKey));

            // 2.7 store the put object into the inbox table
            inboxPuts.add(inboxPut);
        }

        // 2.8 judge fans exist or not
        if (inboxPuts.size() > 0) {

            // get the inbox object
            Table inboxTable = connection.getTable(TableName.valueOf(Constants.INBOX_TABLE));

            // execute the insert operation
            inboxTable.put(inboxPuts);

            // close the inbox table
            inboxTable.close();
        }

        // close the resource
        relaTable.close();
        conTable.close();
        connection.close();
    }

    // 2 focus the user
    public static void addAttends(String uid, String... attends) throws IOException {

        // judge if or not exist attends
        if (attends.length <= 0) {
            System.out.println("please choose attention person");
            return ;
        }

        // get the connection object
        Connection connection = ConnectionFactory.createConnection(Constants.CONFIGURATION);

        // first, operate user relationship table
        // 1 get the user relationship table object
        Table relaTable = connection.getTable(TableName.valueOf(Constants.RELATION_TABLE));

        // 2 create a map, in order to store a put object of user relationship table
        ArrayList<Put> relaPuts  = new ArrayList<>();

        // 3 create an operator put object
        Put uidPut = new Put(Bytes.toBytes(uid));

        // 4 cycle create a put object of attended person
        for (String attend : attends) {

            // 5 assignment value to operator's put object
            uidPut.addColumn(Bytes.toBytes(Constants.RELATION_TABLE_CF1), Bytes.toBytes(attend), Bytes.toBytes(attend));

            // 6 create a put object of attended person
            Put attendPut = new Put(Bytes.toBytes(attend));

            // 7 assignment value to attended person's put object
            attendPut.addColumn(Bytes.toBytes(Constants.RELATION_TABLE_CF2), Bytes.toBytes(uid), Bytes.toBytes(uid));

            // 8 put the attended person's put object into map
            relaPuts.add(attendPut);
        }

        // 9 add operator's put object into map
        relaPuts.add(uidPut);

        // 10 execute user relationship table the insertion operation
        relaTable.put(relaPuts);

        // second operate inbox table
        // 1 get the weibo content table object
        Table contTable = connection.getTable(TableName.valueOf(Constants.CONTENT_TABLE));

        // 2 create put object of inbox table
        Put inboxPut = new Put(Bytes.toBytes(uid));

        // 3 cycle attends, get the every attended person current published weibo
        for (String attend : attends) {

            // 4 get the attended person's current published weibo -> map resultScanner
            Scan scan = new Scan(Bytes.toBytes(attend + "_"), Bytes.toBytes(attend + "|"));
            ResultScanner resultScanner = contTable.getScanner(scan);

            // define a time stamp
            long ts = System.currentTimeMillis();

            // 5 traverse the value
            for (Result result : resultScanner) {

                // 6 assignment the value into the inbox table
                inboxPut.addColumn(Bytes.toBytes(Constants.INBOX_TABLE_CF), Bytes.toBytes(attend), ts++, result.getRow());

            }
        }


        // judge the put object exist or not
        if (!inboxPut.isEmpty()) {
            // get the inbox table object
            Table inboxTable = connection.getTable(TableName.valueOf(Constants.INBOX_TABLE));

            // insert data
            inboxTable.put(inboxPut);

            // close inboxTable connection
            inboxTable.close();

        }

        // close resource
        relaTable.close();
        contTable.close();
        connection.close();

    }

    // 3 delete attention
    public static void deleteAttends(String uid, String... dels) throws IOException {

        if (dels.length <= 0) {
            System.out.println("please choose deletion person");
            return;
        }

        // get the connection object
        Connection connection = ConnectionFactory.createConnection(Constants.CONFIGURATION);

        // first operate user relationship table
        // 1 get the user relationship table object
        Table relatTable = connection.getTable(TableName.valueOf(Constants.RELATION_TABLE));

        // 2 create a map, in order to store the delete object of user relationship table
        ArrayList<Delete> relatDelete = new ArrayList<>();

        // 3 create the delete object of operator
        Delete uidDelete = new Delete(Bytes.toBytes(uid));

        // 4 cycle create the delete object of delete attention
        for (String del : dels) {

            // 5 assignment value to operator's delete object
            uidDelete.addColumns(Bytes.toBytes(Constants.RELATION_TABLE_CF1), Bytes.toBytes(del));

            // 6 create the conceal person's(取关者...) deletion object
            Delete delDelte = new Delete(Bytes.toBytes(del));

            // 7 assignment value to the conceal person's deletion object
            delDelte.addColumns(Bytes.toBytes(Constants.RELATION_TABLE_CF2), Bytes.toBytes(uid));

            // 8 add the conceal person's deletion object into the map
            relatDelete.add(delDelte);
        }

        // 9 add the operator's deletion object into the map
        relatDelete.add(uidDelete);

        // 10 delete the data in user relationship table
        relatTable.delete(relatDelete);

        // second operate inbox table
        // 1 get the inbox table object
        Table inboxTable = connection.getTable(TableName.valueOf(Constants.INBOX_TABLE));

        // 2 create the deletion object of operator
        Delete inboxDelete = new Delete(Bytes.toBytes(uid));

        // 3 assignment value to operator's deletion object
        for (String del : dels) {

            inboxDelete.addColumns(Bytes.toBytes(Constants.INBOX_TABLE_CF), Bytes.toBytes(del));
        }

        // 4 delete data in inbox table
        inboxTable.delete(inboxDelete);

        // close resource
        relatTable.close();
        inboxTable.close();
        connection.close();
    }

    // 4 get the initial page data
    public static void getInit(String uid) throws IOException {

        // 1 get the connection object
        Connection connection = ConnectionFactory.createConnection(Constants.CONFIGURATION);

        // 2 get inbox table object
        Table inboxTable = connection.getTable(TableName.valueOf(Constants.INBOX_TABLE));

        // 3 get the content table object
        Table conTable = connection.getTable(TableName.valueOf(Constants.CONTENT_TABLE));

        // 4 create inbox get object, and get the data max version
        Get inboxGet = new Get(Bytes.toBytes(uid));
        inboxGet.setMaxVersions();
        Result result = inboxTable.get(inboxGet);

        // 5 traverse the data
        for (Cell cell : result.rawCells()) {

            // 6 construct the content table get object
            Get conGet = new Get(CellUtil.cloneValue(cell));

            // 7 get the object data
            Result conResult = conTable.get(conGet);

            // 8 print data
            for (Cell conCell : conResult.rawCells()) {

                System.out.println("RK:" + Bytes.toString(CellUtil.cloneRow(conCell)) +
                        ", CF:" + Bytes.toString(CellUtil.cloneFamily(conCell)) +
                        ", CN:" + Bytes.toString(CellUtil.cloneQualifier(conCell)) +
                        ", Value:" + Bytes.toString(CellUtil.cloneValue(conCell)));
            }
        }
        // 9 close resource
        inboxTable.close();
        conTable.close();
        connection.close();

    }

    // 5 get the all the content of somebody weibo
    public static void getWeibo(String uid) throws IOException {

        // 1 get the connection
        Connection connection = ConnectionFactory.createConnection(Constants.CONFIGURATION);

        // 2 get the weibo content table
        Table conTable = connection.getTable(TableName.valueOf(Constants.CONTENT_TABLE));

        // construct scan object
        Scan scan = new Scan();

        // construct filter
        RowFilter rowFilter = new RowFilter(CompareFilter.CompareOp.EQUAL, new SubstringComparator(uid + "_"));

        scan.setFilter(rowFilter);

        // 4 get the data
        ResultScanner resultScanner = conTable.getScanner(scan);

        // 5 printf data
        for (Result result : resultScanner) {
            for (Cell cell : result.rawCells()) {

                System.out.println("RK:" + Bytes.toString(CellUtil.cloneRow(cell)) +
                        ", CF:" + Bytes.toString(CellUtil.cloneFamily(cell)) +
                        ", CN:" + Bytes.toString(CellUtil.cloneQualifier(cell)) +
                        ", Value:" + Bytes.toString(CellUtil.cloneValue(cell)));
            }
        }

        // 6 close resource
        conTable.close();
        connection.close();
    }
}

4.1 发微博功能

这一个功能比较难，因为不仅对于当前发布微博的人的内容表需要进行更新，并且需要对当前这个人的粉丝的收件表也需要更新，因为粉丝需要看到关注的人的最新动态。所以这里需要对两张表，多个人进行操作。

// 1 publish the weibo
    public static void publishWeiBo(String uid, String content) throws IOException {

        // 1 get the connection object
        Connection connection = ConnectionFactory.createConnection(Constants.CONFIGURATION);

        // 2 get the admin object
        Admin admin = connection.getAdmin();

        // 1 operate weibo content table 对微博内容表进行操作
        // 1.1 get the weibo content table object
        Table conTable = connection.getTable(TableName.valueOf(Constants.CONTENT_TABLE));

        // 1.2 get the current time
        long ts = System.currentTimeMillis();

        // 1.3 get the Rowkey
        String rowKey = uid + "_" + ts;

        // 1.4 create put object 将需要put的内容对象先创建好
        Put conPut = new Put(Bytes.toBytes(rowKey));

        // 1.5 Assignment the put object 给put对象赋值
        conPut.addColumn(Bytes.toBytes(Constants.CONTENT_TABLE_CF), Bytes.toBytes("content"), Bytes.toBytes(content));

        // 1.6 execute insert data operation
        // 执行put操作，即对表进行插入数据
        conTable.put(conPut);

        // 2 operate weibo email box 操作微博收件表
        // 2.1 get user relation table object
        Table relaTable = connection.getTable(TableName.valueOf(Constants.RELATION_TABLE));

        // 2.2 get the fans's information of the person who publish weibo in current
        // 得到最近发布微博的被关注者的粉丝的信息
        Get get = new Get(Bytes.toBytes(uid));
        get.addFamily(Bytes.toBytes(Constants.RELATION_TABLE_CF2));
        Result result = relaTable.get(get);

        // 2.3 create a map, in order to store the put object of weibo content table
        // 创建一个集合为了存储微博的内容
        ArrayList<Put> inboxPuts = new ArrayList<>();

        // 2.4 traverse the fans list
        // 遍历uid的粉丝
        for (Cell cell : result.rawCells()) {

            // 2.5 create the put object of weibo infobox table
            // 创建粉丝的单元格
            Put inboxPut = new Put(CellUtil.cloneQualifier(cell));

            // 2.6 assignment the information to the inbox table
            // 对粉丝的收件表进行赋值，更新关注的人的最新微博
            inboxPut.addColumn(Bytes.toBytes(Constants.INBOX_TABLE_CF), Bytes.toBytes(uid), Bytes.toBytes(rowKey));

            // 2.7 store the put object into the inbox table
            inboxPuts.add(inboxPut);
        }

        // 2.8 judge fans exist or not
        // 判断当前发布微博的人是否有粉丝
        if (inboxPuts.size() > 0) {

            // get the inbox object
            Table inboxTable = connection.getTable(TableName.valueOf(Constants.INBOX_TABLE));

            // execute the insert operation
            inboxTable.put(inboxPuts);

            // close the inbox table
            inboxTable.close();
        }

        // close the resource
        relaTable.close();
        conTable.close();
        connection.close();
    }

4.2 关注功能

关注某个人之后，需要获得这个人的最近发布的微博，以及在微博关系表当中需要添加新的关系。代码也比较长…

// 2 focus the user
    public static void addAttends(String uid, String... attends) throws IOException {

        // judge if or not exist attends
        if (attends.length <= 0) {
            System.out.println("please choose attention person");
            return ;
        }

        // get the connection object
        Connection connection = ConnectionFactory.createConnection(Constants.CONFIGURATION);

        // first, operate user relationship table
        // 操作微博关系表
        // 1 get the user relationship table object
        Table relaTable = connection.getTable(TableName.valueOf(Constants.RELATION_TABLE));

        // 2 create a map, in order to store a put object of user relationship table
        // 创建关注的集合，因为有可能一次性关注多个人。
        ArrayList<Put> relaPuts  = new ArrayList<>();

        // 3 create an operator put object
        Put uidPut = new Put(Bytes.toBytes(uid));

        // 4 cycle create a put object of attended person
        for (String attend : attends) {

            // 5 assignment value to operator's put object
            uidPut.addColumn(Bytes.toBytes(Constants.RELATION_TABLE_CF1), Bytes.toBytes(attend), Bytes.toBytes(attend));

            // 6 create a put object of attended person
            // 这是被关注者对象
            Put attendPut = new Put(Bytes.toBytes(attend));

            // 7 assignment value to attended person's put object
            attendPut.addColumn(Bytes.toBytes(Constants.RELATION_TABLE_CF2), Bytes.toBytes(uid), Bytes.toBytes(uid));

            // 8 put the attended person's put object into map
            relaPuts.add(attendPut);
        }

        // 9 add operator's put object into map
        relaPuts.add(uidPut);

        // 10 add data into user relationship table
        relaTable.put(relaPuts);

        // second operate inbox table
        // 1 get the weibo content table object
        Table contTable = connection.getTable(TableName.valueOf(Constants.CONTENT_TABLE));

        // 2 create put object of inbox table
        Put inboxPut = new Put(Bytes.toBytes(uid));

        // 3 cycle attends, get the every attended person current published weibo
        for (String attend : attends) {

            // 4 get the attended person's current published weibo -> map resultScanner
            // 获得被关注者最近发布的微博
            Scan scan = new Scan(Bytes.toBytes(attend + "_"), Bytes.toBytes(attend + "|"));
            ResultScanner resultScanner = contTable.getScanner(scan);

            // define a time stamp
            long ts = System.currentTimeMillis();

            // 5 traverse the value
            for (Result result : resultScanner) {

                // 6 assignment the value into the inbox table
                // 循环放入粉丝的收件表中
                inboxPut.addColumn(Bytes.toBytes(Constants.INBOX_TABLE_CF), Bytes.toBytes(attend), ts++, result.getRow());

            }
        }


        // judge the put object exist or not
        if (!inboxPut.isEmpty()) {
            // get the inbox table object
            Table inboxTable = connection.getTable(TableName.valueOf(Constants.INBOX_TABLE));

            // insert data
            inboxTable.put(inboxPut);

            // close inboxTable connection
            inboxTable.close();

        }

        // close resource
        relaTable.close();
        contTable.close();
        connection.close();

    }

4.3 取消关注

// 3 delete attention
    public static void deleteAttends(String uid, String... dels) throws IOException {

        if (dels.length <= 0) {
            System.out.println("please choose deletion person");
            return;
        }

        // get the connection object
        Connection connection = ConnectionFactory.createConnection(Constants.CONFIGURATION);

        // first operate user relationship table
        // 1 get the user relationship table object
        Table relatTable = connection.getTable(TableName.valueOf(Constants.RELATION_TABLE));

        // 2 create a map, in order to store the delete object of user relationship table
        ArrayList<Delete> relatDelete = new ArrayList<>();

        // 3 create the delete object of operator
        Delete uidDelete = new Delete(Bytes.toBytes(uid));

        // 4 cycle create the delete object of delete attention
        for (String del : dels) {

            // 5 assignment value to operator's delete object
            uidDelete.addColumns(Bytes.toBytes(Constants.RELATION_TABLE_CF1), Bytes.toBytes(del));

            // 6 create the conceal person's(取关者...) deletion object
            Delete delDelte = new Delete(Bytes.toBytes(del));

            // 7 assignment value to the conceal person's deletion object
            delDelte.addColumns(Bytes.toBytes(Constants.RELATION_TABLE_CF2), Bytes.toBytes(uid));

            // 8 add the conceal person's deletion object into the map
            relatDelete.add(delDelte);
        }

        // 9 add the operator's deletion object into the map
        relatDelete.add(uidDelete);

        // 10 delete the data in user relationship table
        relatTable.delete(relatDelete);

        // second operate inbox table
        // 1 get the inbox table object
        Table inboxTable = connection.getTable(TableName.valueOf(Constants.INBOX_TABLE));

        // 2 create the deletion object of operator
        Delete inboxDelete = new Delete(Bytes.toBytes(uid));

        // 3 assignment value to operator's deletion object
        for (String del : dels) {

            inboxDelete.addColumns(Bytes.toBytes(Constants.INBOX_TABLE_CF), Bytes.toBytes(del));
        }

        // 4 delete data in inbox table
        inboxTable.delete(inboxDelete);

        // close resource
        relatTable.close();
        inboxTable.close();
        connection.close();
    }

4.4 获得用户初始页

// 4 get the initial page data
    public static void getInit(String uid) throws IOException {

        // 1 get the connection object
        Connection connection = ConnectionFactory.createConnection(Constants.CONFIGURATION);

        // 2 get inbox table object
        Table inboxTable = connection.getTable(TableName.valueOf(Constants.INBOX_TABLE));

        // 3 get the content table object
        Table conTable = connection.getTable(TableName.valueOf(Constants.CONTENT_TABLE));

        // 4 create inbox get object, and get the data max version
        Get inboxGet = new Get(Bytes.toBytes(uid));
        inboxGet.setMaxVersions();
        Result result = inboxTable.get(inboxGet);

        // 5 traverse the data
        for (Cell cell : result.rawCells()) {

            // 6 construct the content table get object
            Get conGet = new Get(CellUtil.cloneValue(cell));

            // 7 get the object data
            Result conResult = conTable.get(conGet);

            // 8 print data
            for (Cell conCell : conResult.rawCells()) {

                System.out.println("RK:" + Bytes.toString(CellUtil.cloneRow(conCell)) +
                        ", CF:" + Bytes.toString(CellUtil.cloneFamily(conCell)) +
                        ", CN:" + Bytes.toString(CellUtil.cloneQualifier(conCell)) +
                        ", Value:" + Bytes.toString(CellUtil.cloneValue(conCell)));
            }
        }
        // 9 close resource
        inboxTable.close();
        conTable.close();
        connection.close();

    }

4.5 获得用户全部微博内容

// 5 get the all the content of somebody weibo
    public static void getWeibo(String uid) throws IOException {

        // 1 get the connection
        Connection connection = ConnectionFactory.createConnection(Constants.CONFIGURATION);

        // 2 get the weibo content table
        Table conTable = connection.getTable(TableName.valueOf(Constants.CONTENT_TABLE));

        // construct scan object
        Scan scan = new Scan();

        // construct filter
        RowFilter rowFilter = new RowFilter(CompareFilter.CompareOp.EQUAL, new SubstringComparator(uid + "_"));

        scan.setFilter(rowFilter);

        // 4 get the data
        ResultScanner resultScanner = conTable.getScanner(scan);

        // 5 printf data
        for (Result result : resultScanner) {
            for (Cell cell : result.rawCells()) {

                System.out.println("RK:" + Bytes.toString(CellUtil.cloneRow(cell)) +
                        ", CF:" + Bytes.toString(CellUtil.cloneFamily(cell)) +
                        ", CN:" + Bytes.toString(CellUtil.cloneQualifier(cell)) +
                        ", Value:" + Bytes.toString(CellUtil.cloneValue(cell)));
            }
        }

        // 6 close resource
        conTable.close();
        connection.close();
    }

5 test包测试

在test包当中，编写测试代码，相对简单。

package test;

import constants.Constants;
import dao.HBaseDao;
import utils.HBaseUtil;

import java.io.IOException;

public class TestWeiBo {

    public static void init() {

        try {
            // create namespace
            HBaseUtil.createNameSpace(Constants.NAMESPACE);

            // create weibo content table
            HBaseUtil.createTable(Constants.CONTENT_TABLE, Constants.CONTENT_TABLE_VERSIONS, Constants.CONTENT_TABLE_CF);

            // create user relationship table
            HBaseUtil.createTable(Constants.RELATION_TABLE, Constants.RELATION_TABLE_VERSIONS,
                    Constants.RELATION_TABLE_CF1, Constants.RELATION_TABLE_CF2);

            // create inbox table
            HBaseUtil.createTable(Constants.INBOX_TABLE, Constants.INBOX_TABLE_VERSIONS, Constants.INBOX_TABLE_CF);

        } catch (IOException e) {
            e.printStackTrace();
        }
    }

    public static void main(String[] args) throws IOException, InterruptedException {
        // initialize
        init();
        System.out.println("init done");

        // 1001 publish weibo
        HBaseDao.publishWeiBo("1001","这是1001的微博，Hello World");
        System.out.println("publish done");

        // 1002 attend 1001, 1003
        HBaseDao.addAttends("1002", "1001", "1003");
        System.out.println("attend done");

        // get the 1002 initial page
        HBaseDao.getInit("1002");
        System.out.println("*********************111**********************");

        // 1003 publish 3 weibo, at the same time  1002 publish 2 weibo
        HBaseDao.publishWeiBo("1003", "快结束了HBase了！！");
        Thread.sleep(10);
        HBaseDao.publishWeiBo("1001", "那你还是个弟弟啊！！");
        Thread.sleep(10);
        HBaseDao.publishWeiBo("1003", "哦，不我要当哥哥！！");
        Thread.sleep(10);
        HBaseDao.publishWeiBo("1001", "弟弟！！");
        Thread.sleep(10);
        HBaseDao.publishWeiBo("1003", "叫哥哥！！");

        // get 1002 initial page
        HBaseDao.getInit("1002");

        System.out.println("*********************222***********************");

        // 1002 conceal 1003
        HBaseDao.deleteAttends("1002", "1003");

        // get the 1002 initial page
        HBaseDao.getInit("1002");

        System.out.println("*********************333***********************");

        // 1002 attend 1003 again
        HBaseDao.addAttends("1002", "1003");

        // get the 1002 initial page
        HBaseDao.getInit("1002");

        System.out.println("*********************444***********************");

        // get 1001 detail initial page
        HBaseDao.getWeibo("1001");
        System.out.println("*********************555***********************");
    }
}

总结

不知道为什么在configuration配置的时候总是存在问题，最后参考以前写的测试API的代码，加上了configuration.set()，才可以跑通。否则一直显示的是连接超时，不存在zookeeper的master。具体问题还不清楚，感觉像是代码找不到连接zookeeper的端口了。
最后我这个弱鸡终于成功跑通了代码，感觉还是蛮不错的，记录一下遇到的坑。
1、编写代码的时候，需要记住addColumns()函数里面是需要对列进行操作，好几次写成了table，忘了加CF了，然后报错。并且还需要转换成字节的形式。
2、需要弄清楚业务的逻辑，因为里面只是设计了三张表所以逻辑相对简单。但是尽可能的还是多考虑考虑一些解耦合的操作或者是需求分析的时候对创建表考虑清楚。

最后，加油吧，诸君。

本文章为转载内容，我们尊重原作者对文章享有的著作权。如有内容错误或侵权问题，欢迎原作者联系我们进行内容更正或删除文章。

上一篇：java线程安全的Integer java线程安全的容器

下一篇：sql server 2008 多表联合查询分页 sql如何多表联合查询

提问和评论都可以，用心的回复会被更多人看到评论

发布评论

相关文章

官方博客	全部文章	热门标签	班级博客
了解我们	网站地图	意见反馈

鸿蒙开发者社区	51CTO学堂
51CTO	软考资讯