Mohd Amir Ahmad: May 2018

Friday, 25 May 2018

Sqoop Export-exporting Data From HDFS to RDBMS

1) We have a text file named "employee_ctc".

2) We need to put this file on HDFS.

i) hdfs dfs -mkdir /sqoopexport

ii) hdfs dfs -copyFromLocal /home/amir/employee_ctc /sqoopexport

3) Create a table in MySQL having all the column as we have for each record in "employee_ctc" file.

we have 6 fields: empid, name, age, gender, desig and ctc.

So we create a table with six fields and "empid" as Primary key.

4) Confirming table is empty

select * from employee_ctc;

5) The following command is used to export the table data (which is in employee_ctc file on HDFS) to the employee_ctc table in database of Mysql database server.

Go to the Sqoop directory in which sqoop is installed.

bin/sqoop export -connect jdbc:mysql://localhost:3306/sampleJava -username root -password 12345 --table employee_ctc --export-dir /sqoopexport/employee_ctc.txt

sampleJava is my database name.

If your command runs successfully, you will get following information at the end.

which clearly shows 14 records have been retrieved from HDFS . Now let's see whether these records have been exported to the employee_ctc table.

6) Checking whether record from HDFS has been exported to the RDBMS table or not.

As we can see, the record from HDFS path /sqoopexport/employee_ctc file has been exported successfully to RDBMS table employee_ctc.

Monday, 21 May 2018

Sqoop Import-Importing Data From RDBMS to HDFS

How to import data from MySQL database to Hadoop HDFS

1) First create a table named "customer" in MySQL

2) Insert data into the "customer" table.

3) Check the data in the table "customer"

4) Make sure to start Hadoop daemons before performing Sqoop operations.

5) Go to the directory in which Sqoop is installed and run the following line command.

bin/sqoop import -connect jdbc:mysql://localhost:3306/sampleJava -username root -password 12345 --table customer --target-dir /sqoopimport01 -m 1

If it runs successfully, then you will get following lines at the end.

which clearly shows 5 records have been retrieved from MySQL database. Now let's see whether these records have been imported to our HDFS.

6) Check whether a directory named as "/sqoopimport01" is created or not.

Earlier:

After running import:

As you can the required directory has been created.

7) Checking whether data has been imported to HDFS.

"/sqoopimport01" contains two files. "_SUCCESS" indicates that the process has been successfull and that import has been done.

Using "cat" command, we can see the data from "customer" table has been imported to HDFS and each record is seperated by comma.

Saturday, 19 May 2018

JSP Sample Project

Sample JSP Project- Registration and Login functionality

1) Download and install Eclipse.

2) Create a new Dynamic Project in Eclipse.
Steps:
1) File -> New -> Dynamic Web Project
2) Enter Project name
3) Click on Finish

3) Add the following jsp and html files in "WebContent" directory.

i) index.jsp

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
<title>JSP Sample</title>
</head>
<body>
<a href="Login.html">Login</a>
</br>
</br>
<a href="Reg.html">Register</a>
</body>
</html>

ii) Login.html

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
<title>JSP Login Page</title>
</head>
<body>
<form action="login.jsp" method="post">

User name :<input type="text" name="user" /></br></br>
password :<input type="password" name="password" /></br></br>
<input type="submit" />

</form>
</body>
</html>

iii) login.jsp

iv) Reg.html

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
<title>JSP Registration</title>
</head>
<body>
<form action="reg.jsp" method="post">

User Name :<input type="text" name="username" /><br/><br/>
Password :<input type="password" name="password" /><br/><br/>
First Name :<input type="text" name="fname" /><br/><br/>
Last Name :<input type="text" name="lname" /><br/><br/>
Email :<input type="text" name="email" /><br/><br/>
<br/><br/>

<input type="submit" />

</form>
</body>
</html>

v) reg.jsp

<%@ page language="java" contentType="text/html; charset=ISO-8859-1"
pageEncoding="ISO-8859-1"%>
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
<title>JSP Registration</title>
</head>
<body>
<%@ page import ="java.sql.*" %>
<%@ page import ="javax.sql.*" %>
<%
String user=request.getParameter("username");
session.putValue("userid",user);
String pwd=request.getParameter("password");
String fname=request.getParameter("fname");
String lname=request.getParameter("lname");
String email=request.getParameter("email");
Class.forName("com.mysql.jdbc.Driver");
java.sql.Connection con = DriverManager.getConnection("jdbc:mysql://localhost:3306/jsptest","root","12345");
Statement st= con.createStatement();
ResultSet rs;
int i=st.executeUpdate("insert into jspusers values ('"+user+"','"+pwd+"','"+fname+"', '"+lname+"','"+email+"')");

out.println("Registered");

%>
<a href ="Login.html">Login</a><br/><br/>
<a href="index.html">Home</a>
</body>
</html>

4) Add the "mysql-connector jar" file in the directory WebContent -> WEB-INF -> lib

5) Create a table named "jspusers" in the database "jsptest".

create table jspusers (user varchar(20),
password varchar(15),
fname varchar(15),
lname varchar(15),
email varchar(20)
);

6) Add Tomcat server in the eclipse to run the Project.
How to add Tomcat?