Hadoop Developer/Engineer

Company Name: TEG GLOBAL
Location: San Diego, CA
Date Posted: 03rd Jun, 2016
  • Provide technical and development support to build and maintain a modernized Enterprise Data Warehouse (EDW) by expanding the current on-premises Hadoop cluster to accommodate an increased volume of data flowing into the enterprise data warehouse.
  • Perform data formatting involves cleaning up the data.
  • Assign schemas and create HIVE tables Apply other HDFS formats and structure (Avro, Parquet, etc. ) to support fast retrieval of data, user analytics and analysis
  • Assess the suitability and quality of candidate data sets for the Data Lake and the EDW Design and prepare technical specifications and guidelines.
  • Act as self-starter with the ability to take on complex projects and analyses independently
  • Ensure secure coding practices are adhered to in all phases of the secure development lifecycle.
  • Be knowledgeable in all NGC SSA Programs HIPAA compliance requirements and proactively address any HIPAA concerns.
  • Become knowledgeable on the HIPAA policies and procedures for the program and ensure awareness of HIPAA breach process.



Basic Qualifications: 

  • 2+ years of proven experience in a range of  etc. Big data architectures and frameworks including Hadoop ecosystem, Java MapReduce, Pig, Hive, Spark, Impala
  • 2 years of proven experience working with, processing and managing large data sets (multi TB scale). Proven experience in ETL (Syncsort DMX-h, Ab Initio, IBM- InfoSphere Data Replication, etc.), mainframe skills, JCL

Preferred Qualifications: 

  • Experience with Apache Hadoop Administration (Preferred Cloudera Framework).
  • Experience with Linux Administration (Centos and Red Hat).
  • Experience in coding shell scripting in Python.
  • Experience in Big Data Storage and File System Design.
  • Experience in performing troubleshooting procedures and designing resolution scripts
  • Experience with Mainframe development including Java Batch development, Architectural Design/Analysis and Database development.
  • Experience in analytic programming, data discovery, querying databases/data warehouses and data analysis. Experience in data ingestion technologies with reference to relational databases (i.e. DB2, Oracle, SQL)
  • Experience with advanced SQL query writing and data retrieval using "Big Data" Analytics.
  • Experience with enterprise scale application architectures.
  • Proven ability to work with senior technical managers and staff to provide expert-level support for the installation, maintenance, upgrading, and administration of full-featured database management systems.
  • Knowledge and experience with application integration, thorough understanding of complex network topologies, solid understanding of system security & risk management.