相关Linux命令总结:
echo ${JAVA_HOME} 显示出jdk的安装目录
echo 2 > myid 把2写到myid文件(覆盖原内容)
cat myid 查看myid文件的内容
zkServer.sh start 启动zookeeper
zkServer.sh status 查看zookeeper的启动状态
netstat -nltp 查看端口占用情况
netstat -nltp | grep 2345 查看进程号为2345的端口占用情况
mv zoo_sample.cfg zoo.cfg 改名
ZooKeeper介绍:
Zookeeper集群中的服务器角色有三种,
leader server
follower server
observer server
leader特殊之处在于它有决定权,具有Request Processor
(observer server 与follower server的区别就在于不参与leader选举)
ZooKeeper的安装
1.将zookeeper-3.4.6.tar上传至linux并执行 tar -zxvf zookeeper-3.4.6.tar.gz 解压
2.进入到解压后的目录/home/hadoop-node-01/zookeeper-3.4.6/conf中,
执行 mv zoo_sample.cfg zoo.cfg 将zoo_sample.cfg改名为zoo.cfg,该文件参数介绍如下:
tickTime=2000 ----follower向leader发送心跳的时间间隔,2秒心跳一次
initLimit=10 ----启动初始化的时间为10个心跳
syncLimit=5 ----leader向follower发送请求让follower同步数据,若5个心跳没有收到回应,就认为超时了
dataDir=/tmp/zookeeper ----zookeeper数据目录
clientPort=2181 ----zookeeper接收客户端请求的端口号
一般只配置dataDir,(其他配置一般默认就好),dataDir=/home/hadoop-node-01/zookeeper-3.4.6/data (data目录自己新建)
另外需要新加配置:(给每台机器设置一个id,如:server.1 server.2 server.3)
server.1=hadoop-01-node-server:2888:3888 ----其中2888是leader和follower通信的端口,3888是选举通信的端口
server.2=hadoop-02-node-server:2888:3888
server.3=hadoop-03-node-server:2888:3888
3.进入到/home/hadoop-node-01/zookeeper-3.4.6/下,执行 echo 1 > data/myid 新建文件data/myid,并将自己机器设置过的id写入该文件
4.单台机器配置完毕,将zookeeper-3.4.6复制到其他机器,并进入/home/hadoop-node-01/zookeeper-3.4.6/data/myid设置各自机器的id。
(进入另外两台机器的/home/hadoop-node-01/zookeeper-3.4.6/data下分别执行echo 2 > myi decho 3 > myid)
5.启动ZooKeeper
执行/home/hadoop-node-01/zookeeper-3.4.6/bin/zkServer.sh start (配置环境变量后直接执行zkServer.sh start)
启动成功后jps,出现QuorumPeerMain进程
ZooKeeper客户端的运行:
ZooKeeper是一个集群,作为一个整体对外提供服务,任何一个客户端只要能连上就可以使用它提供的服务~~~
ZooKeeper的兼容性很好,Windows中也可以安装运行ZooKeeper的客户端(ZooKeeper的bin中就有.cmd的shell脚本zkServer.cmd 和 zkCli.cmd)
Windows中启动ZooKeeper客户端:C:\zookeeper-3.4.6\bin\zkCli.cmd -server hadoop-node-01-server:2181
(注意:hadoop-node-01-server是C:\Windows\System32\drivers\etc\hosts中配置的主机映射名,
2181在ZooKeeper的配置文件中配置过,是ZooKeeper接收客户端请求的端口号)
启动后执行help看一下常用命令
ZooKeeper节点有四种类型
PERSISTENT ----永久的(只要客户端不删除,就一直在)
PERSISTENT_SEQUENTIAL ----永久且有序的
EMPEMERAL ----短暂的(只要客户端掉线,则会被自动删除)
EMPEMERAL_SEQUENTIAL ----短暂且有序的
ZooKeeper的目录结构:
ZooKeeper的文件系统不像一般的文件系统,它是树状结构的“目录树”,每一个节点叫做znode,
每一个znode都自己的path,都携带一份用户的数据,都有自己的类型。
创建节点:create [-s] [-e] path data acl (不带[-s] [-e] 创建的默认就是永久的 如:create /LQ 'this is my data')
拿数据:get /LQ 修改数据:set /LQ 'this is my data2'
删除节点:rmr /LQ
ZooKeeper的基本api:
导包:
核心包:C:\zookeeper-3.4.\zookeeper-3.4.6.jar
依赖包:C:\zookeeper-3.4.6\lib\下5个
代码:
public class zookeeper {
private ZooKeeper zk = null;
@Before
public void init() throws Exception {
zk = new ZooKeeper("hadoop-node-01-server:2181", 2000, new Watcher(){
// ZooKeeper监听器的回调函数,Watcher是接口,此处是匿名内部类
@Override
public void process(WatchedEvent event) {
if (event.getType() == event.getType().None) return;
System.out.println(event.getType());
System.out.println(event.getPath());
System.out.println(event.getState());
try {
// 重复监听该节点的数据变化
zk.getData("/home/myhome3", true, null);
// 重复监听该节点的子节点的创建
zk.getChildren("/home", true);
} catch (KeeperException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
});
}
@Test
public void testCreateZnode() throws KeeperException, InterruptedException, UnsupportedEncodingException{
//
zk.create("/home", "这是我的家!!".getBytes("utf-8"), Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);
zk.create("/home/myhome3", "这是我家的数据1!!".getBytes("utf-8"), Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);
zk.create("/home/myhome4", "这是我家的数据2!!".getBytes("utf-8"), Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);
zk.close();
}
@Test
public void testDeleteZnode() throws InterruptedException, KeeperException{
zk.delete("/home", -1);
Stat exists = zk.exists("/home", false);
System.out.println("exists" +exists);
}
@Test
public void testUpdateZnode() throws KeeperException, InterruptedException, UnsupportedEncodingException{
byte[] data = zk.getData("/home/myhome3", false, null);
System.out.println(new String(data,"utf-8"));
zk.setData("/home/myhome3", "大黄蜂".getBytes("utf-8"), -1);
data = zk.getData("/home/myhome3", false, null);
System.out.println(new String(data,"utf-8"));
}
@Test
public void testGetChildren() throws KeeperException, InterruptedException{
List<String> children = zk.getChildren("/home", false);
for (String child: children) {
System.out.println(child);
}
}
//监听器是一次性的,只监听一次后就失效了
@Test
public void testWatch() throws KeeperException, InterruptedException{
// 监听该节点的数据变化
zk.getData("/home/myhome3", true, null);
// 监听该节点的子节点的创建
zk.getChildren("/home", true);
Thread.sleep(Long.MAX_VALUE);
}
// 上传文件,(比如上传配置文件,其他程序可从这读下来该配置文件自己去解析)
@Test
public void TestUploadConfigFileToZookKeeper() throws IOException, KeeperException, InterruptedException{
String zookeeper_txt = FileUtils.readFileToString(new File("C:/zookeeper笔记.txt")); (----FileUtils需要导包commons-io-2.4.jar)
zk.create("/conf", null, Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);
zk.create("/conf/zookeeper.txt", zookeeper_txt.getBytes("utf-8"), Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);
zk.close();
}
}