Friday, 25 May 2018

Sqoop Export-exporting Data From HDFS to RDBMS

1) We have a text file named "employee_ctc".


2) We need to put this file on HDFS.


i) hdfs dfs -mkdir /sqoopexport

ii) hdfs dfs -copyFromLocal /home/amir/employee_ctc /sqoopexport



3) Create a table in MySQL having all the column as we have for each record in "employee_ctc" file.


we have 6 fields: empid, name, age, gender, desig and ctc.

So we create a table with six fields and "empid" as Primary key.



4) Confirming table is empty


select * from employee_ctc;


5) The following command is used to export the table data (which is in employee_ctc file on HDFS) to the employee_ctc table in database of Mysql database server.


Go to the Sqoop directory in which sqoop is installed.

bin/sqoop export -connect jdbc:mysql://localhost:3306/sampleJava -username root -password 12345 --table employee_ctc --export-dir /sqoopexport/employee_ctc.txt

sampleJava is my database name.

If your command runs successfully, you will get following information at the end.



which clearly shows 14 records have been retrieved from HDFS . Now let's see whether these records have been exported to the employee_ctc table.


6) Checking whether record from HDFS has been exported to the RDBMS table or not.




As we can see, the record from HDFS path /sqoopexport/employee_ctc file has been exported successfully to RDBMS table employee_ctc.

No comments:

Post a Comment