HIVE HOW TO LOAD DATA_Candice Jing_百度空间
HIVE HOW TO LOAD DATA
How to load data into Hive
------------------------
In order to load data into Hive, we need to tell Hive the format of the data
through "CREATE TABLE" statement:
FileFormat: the data has to be in Text or SequenceFile.
Format of the row:
If the data is in delimited format, use MetadataTypedColumnsetSerDe
If the data is in delimited format and has more than 1 levels of delimitor,
use DynamicSerDe with TCTLSeparatedProtocol
If the data is a serialized thrift object, use ThriftSerDe
The steps to load the data:
1 Create a table:
CREATE TABLE t (foo STRING, bar STRING)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY '\t'
STORED AS TEXTFILE;
CREATE TABLE t2 (foo STRING, bar ARRAY<STRING>)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY '\t'
COLLECTION ITEMS TERMINATED BY ','
STORED AS TEXTFILE;
CREATE TABLE t3 (foo STRING, bar MAP<STRING,STRING>)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY '\t'
COLLECTION ITEMS TERMINATED BY ','
MAP KEYS TERMINATED BY ':'
STORED AS TEXTFILE;
CREATE TABLE t4 (foo STRING, bar MAP<STRING,STRING>)
ROW FORMAT SERIALIZER 'org.apache.hadoop.hive .serde2.MetadataTypedColumnsetSerDe'
WITH SERDEPROPERTIES ('columns'='foo,bar','SERIALIZATION.FORMAT'='9');
(RegexDeserializer is not done yet)
CREATE TABLE t5 (foo STRING, bar STRING)
ROW FORMAT SERIALIZER 'org.apache.hadoop.hive.serde2.RegexDeserializer'
WITH SERDEPROPERTIES ('regex'='([a-z]*) ([a-z])');
2 Load the data:
LOAD DATA LOCAL INPATH '../examples/files/kv1.txt' OVERWRITE INTO TABLE t;
How to read data from Hive tables
------------------------
In order to read data from Hive tables, we need to know the same 2 things:
File Format
Row Format
Then we just need to directly open the HDFS file and read the data.
CREATE TABLE table1 (a STRING, b STRING)
ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t' ESCAPED BY '\\'
STORED AS TEXTFILE;
ESCAPED BY 指定转义字符
