Utilizing Sqoop and Hive for data analysis can allow you to reuse your SQL skills to take advantage of the Hadoop ecosystem without having to learn a new language and lots of new processes. This article assumes you already have the Hortonworks Data Platform already installed on your machine/virtual host. This article will show you how to take data from a SQL store, import it into Hive, and then do a basic Hive query.
Step 1
First, we’ll import a test data set that has plenty of records in it. This data set is more than a small 100 row set file but actually includes 100K records of generated user info and 50K records of generated order data. Using https://www.generatedata.com/, the dataset was assembled and made available as a public gist. It can be accessed as follows:
git clone https://gist.github.com/4c5599b08483264e1e22.git ./data_sql
This will go and make a local copy the SQL file and save it in a directory called data_sql.
Step 2
Next, we will go ahead and import the SQL file we just copied into the local MySQL on HDP.
mysql