Finding Unique URL's using Hadoop Hive

Finding Unique URL's using Hadoop Hive

Hive Project -Learn to write a Hive program to find the first unique URL, given 'n' number of URL's.

Videos

Each project comes with 2-5 hours of micro-videos explaining the solution.

Code & Dataset

Get access to 50+ solved projects with iPython notebooks and datasets.

Project Experience

Add project experience to your Linkedin/Github profiles.

Customer Love

Read All Reviews

SUBHABRATA BISWAS

Lead Consultant, ITC Infotech

The project orientation is very much unique and it helps to understand the real time scenarios most of the industries are dealing with. And there is no limit, one can go through as many projects... Read More

Nathan Elbert

Senior Data Scientist at Tiger Analytics

This was great. The use of Jupyter was great. Prior to learning Python I was a self taught SQL user with advanced skills. I hold a Bachelors in Finance and have 5 years of business experience.. I... Read More

What will you learn

Learn to write a Hive program

Project Description

You have a file that contains 200 billion URLs. How will you find the first unique URL using Hadoop Hive? 

Similar Projects

In this hadoop project, we are going to be continuing the series on data engineering by discussing and implementing various ways to solve the hadoop small file problem.

In this big data project, we will embark on real-time data collection and aggregation from a simulated real-time system using Spark Streaming.

In this big data project, we will continue from a previous hive project "Data engineering on Yelp Datasets using Hadoop tools" and do the entire data processing using spark.

Curriculum For This Mini Project

23-Nov-2017
09m