Big Data Engineer

Company Name: Quicken Loans
Location: Detroit, MI
Date Posted: 15th Dec, 2016

The Data Warehouse Engineer for Big Data (DWE) is responsible for the full life cycle of the back-end development of a Hadoop data warehouse. Responsibilities include developing ETL processes, designing structured data models from unstructured datasets, and integrating data in Hadoop with an SQL Server data warehousing environment. Projects range from bringing in structured, semi-structured, and unstructured data to provide analytics and new insights that allow the business to make data-driven decisions. The DWE works closely with Business Analysts to work through business requirements and develop processes necessary to provide visibility into the data that is stored in Hadoop. The DWE helps ensure that Hadoop and MapReduce jobs are running optimally.


  • Develop ETL processes to populate a Hadoop data warehouse with large datasets from a variety of sources, and integrate Hadoop within an SQL Server data warehousing environment
  • Create MapReduce programs in Java or Python, and leverage tools like Pig and Hive to transform and query large datasets
  • Assist with Hadoop administration to ensure the health and reliability of the cluster
  • Monitor and troubleshoot performance issues on a Hadoop cluster
  • Follow the design principles and best practices defined by the team for data warehousing techniques and architecture
  • Bachelor’s degree in computer science or equivalent experience
  • Two years experience with Hadoop (includes Hive and Pig)
  • Two years working in a Linux environment
  • Two years of programming experience (Java or Python preferred)
  • Two years of experience with SQL and relational databases
  • Demonstrated ability to performance-tune MapReduce jobs
  • Strong analytical and research skills
  • Demonstrated ability to work independently as well as with a team
  • Ability to troubleshoot problems and quickly resolve issues
  • Strong communication skills

What’ll Make You Special

  • Experience administering a Hadoop cluster
  • Experience using SQL Server