3.操作hdfs的api(python)
原创
©著作权归作者所有:来自51CTO博客作者liuyunshengsir的原创作品,请联系作者获取转载授权,否则将追究法律责任
1.安装依赖包
2.连接
# 连接hdfs服务
from hdfs import InsecureClient
client = InsecureClient('http://183.168.10.11:50070', user='root')
3.列出当前目录下的所有文件
4.创建新文件,并写入内容
data = '''
this is new file by lys!
'''
with client.write('/foodir/myfile.txt') as writer:
writer.write(data)
5.读取文件
with client.read('/foodir/myfile.txt') as reader:
data = reader.read()
print data
6.文件追加内容
# 通过设置append参数,向一个已经存在的文件追加写入数据
with client.write('/foodir/myfile.txt', append=True) as writer:
writer.write('this is append text by lys! \n')
7.重命名
client.rename('/foodir/myfile.txt', '/foodir/myfile2.txt')
8.下载到指定目录
# 下载到指定目录/home
client.download('/foodir/myfile.txt', '/home/myfile.txt', n_threads=3)
9.创建文件夹
client.makedirs('/testdiretory')
10.上传文件
client.upload(‘目标路径’, ‘本地源路径’)
client.upload('/testdiretory/myfile.txt','/home/myfile.txt' )
11.设置权限
client.set_permission(filepath, 777)