sqoop常用命令记录

sqoop从msyq导入数据到hive，自定建表

sqoop import \
--connect 'jdbc:mysql://xxx.xxx.xxx.xx:3306/database?useUnicode=true&characterEncoding=utf8&zeroDateTimeBehavior=CONVERT_TO_NULL' \
--username root\
--password 123456 \
--table student_info \
--hive-import \
--hive-drop-import-delims \
--create-hive-table \
--hive-database lods \
--hive-table  lods_source_dxxbs_student_info \
--fields-terminated-by "\001" \
--lines-terminated-by "\n" \
-m 1

注意：一定要添加 --hive-drop-import-delims 否则，mysql字段中包含\r\n会导致数据错位

 

sqoop导mysql数据到hive，覆盖hive已有数据

/data/software/sqoop-1.4.7/bin/sqoop import \
--connect "jdbc:mysql://xxx.xxx.xxx.xxx:3306/database?useUnicode=true&characterEncoding=utf8&zeroDateTimeBehavior=CONVERT_TO_NULL"  \
--username root \
--password 123456 \
--table student_info \
--fields-terminated-by "\001" \
--hive-drop-import-delims \
--lines-terminated-by "\n" \
--hive-import \
--hive-overwrite \
--hive-database lods \
-–create-hive-table \
--hive-table lods_source_dxxbs_student_info \
-m 1

 sqoop到出hive收据到mysql。且根据指定字段更新数据

 /data/software/sqoop-1.4.7/bin/sqoop export \
--connect "jdbc:mysql://192.168.0.69:3306/interface_library?useUnicode=true&characterEncoding=utf8&zeroDateTimeBehavior=CONVERT_TO_NULL" \
--username gxmanager \
--password VItnc5IdF0#DV4VT \
--table ldwd_basic_dxxbs_student_info_all \
--export-dir /user/hive/warehouse/ldwd.db/ldwd_basic_dxxbs_student_info_all/ \
--update-key student_key \
--update-mode allowinsert \
--input-fields-terminated-by '\001' \
--input-null-string 'null' \
--input-null-non-string 'null' \
-m 1

 

author@nohert