Hadoop Big Data Engineer

Company Name: Meridian Health Plan
Location: MI-Detroit
Date Posted: 06th Sep, 2016
Description:
  • Install and configure Hadoop components and related utilities
  • Develop processes for source data ingestion, transformation, and database loading
  • Develop processes for data quality monitoring
  • Develop processes to support “Data as a Service (DaaS)”
  • File system management, monitoring, support and maintenance for HDFS, KUDU, and HBASE
  • Create scalable and high-performance web services for data tracking
  • Assisting with data lake folder design
  • Recommending and establishing security policies and procedures for the Hadoop environment
  • Assisting in the development and implementation of various strategic initiatives.
  • Contribute to development of Architecture Policies, Standards and Governance for the Hadoop and Big Data environment
  • Research and development with promising leading edge big data technologies
  • Participating in data architecture design and review processes, including planning and monitoring efforts, reviewing deliverables, and communicating to management
  • Responding to change and engaging in multiple projects simultaneously
  • Works with minimal guidance; seeks guidance on only the most complex tasks
Qualification:
  • Bachelor’s degree in related field is required
  • 7+ years of experience as an IT professional
  • 2+ years working with Hadoop (Cloudera or Horton Works or MAPR)
  • 3+ years working with data design or database development
  • Experience with reporting tools such as Tableau, Qlikview, Datameer, etc would be a plus
  • Prior experience in a complex, highly integrated services environment
  • Working knowledge of Red Hat LINUX
  • Good aptitude in multi-threading and concurrency concepts
  • Understanding of and experience developing in Hadoop
  • Working knowledge of Kafka, Flume, Hive, Spark, Impala, Sqoop Oozie, Hbase, Zookeeper, HUE
  • Expert level SQL knowledge and experience with a relational database
  • Working knowledge of Pig Latin, HiveQL, Python or Java
  • Substantial understanding of reporting and analytics tools
  • Experience working with data lakes
  • Pre and post installation of Hadoop software and good understanding of Hadoop ecosystems dependencies
  • Implementing data ingress and egress-Facilitating generic input/output, moving bulk data into and out of Hadoop
  • Expertise in setting up, configuration and management of data security
  • On-going support for various Hadoop environments - DEMO, TEST, UAT, and PROD;
  • Familiarity with Cloudera, Kafka, Spark, data lakes, and Flume is desired
Apply Now