Hive Project - Visualising Website Clickstream Data with Apache Hadoop

Analyze clickstream data of a website using Hadoop Hive to increase sales by optimizing every aspect of the customer experience on the website from the first mouse click to the last.
Each project comes with 2-5 hours of micro-videos explaining the solution.
Code & Dataset
Get access to 50+ solved projects with iPython notebooks and datasets.
Project Experience
Add project experience to your Linkedin/Github profiles.

What will you learn

  • Analyze JSON data;Loading JSON format to Hive
  • Create a Schema to the fields in the table.
  • Creating queries to set up the EXTERNAL TABLE in Hive
  • Create new desired TABLE to copy the data.
  • Creating query to populate and filter the data.
  • Analyze log files in HIVE.

Project Description

Clicksteam data records the flow or trail of a user when he/she visits a website. For example, if you have pages A-Z and want to see how many people land on Page G and then go to Page B - you can analyze this data and see the clickstream pattern of your visitors. This data is stored in semi structured web logs. Often you will hear the term web log analysis - this is the same as analyzing clickstream data. Segmenting, and analyzing this clickstream data will give you a more refined look at your customer's behavior patterns - from the time they land on your website till the time they either buy your product or leave without buying. 

You have built a wonderful website and your transaction page has all the information that is required for someone to know before buying the product. Still you see that a huge number of your website visitors leave before buying a single product. This is because of a broken link or path somewhere which prevents users to quickly and easily buy your product. Hadoop helps you to extract, store and analyze the clickstream data or web log data and merge it with the traditional customer data - in order to get better insights into the behavior of the visitor and optimize the path to product buying. Hive is the easiest of the Hadoop tools to learn. If you are from a data warehousing background and know SQL well - it will be a breeze to work on Hive. Hive is a data warehouse infrastructure built on top of Hadoop and is quite versatile in its usage, as it supports different storage types such as plain text, RCFile, Amazon S3, HBase, ORC, etc. Hive has its own SQL like language called HiveQL with schemas - which transparently converts queries to MapReduce or Apache Spark jobs. 

You will be working on solving these business problems for the end-user in this Hadoop Hive Project:

  • Optimizing the click through path of the users

  • Which is the most optimum path for a user to follow in order to buy the product?

  • After how many clicks does a user lose interest to buy a product?

  • Which products do users usually buy together?

  • Where can the website resources be allocated to provide the best user experience to a visitor to make him return again?

Curriculum For This Mini Project

  Starting and troubleshooting Hortonworks
  Introduction to the dataset and business problem
  Create Tables in Hive
  Data Selection
  Setting up ODBC to connect to hive tables
  Gelocation query
  Advanced data selection