Company Name: TEG GLOBAL
Location: San Diego, CA
Date Posted: 03rd Jun, 2016
- Provide technical and development support to build and maintain a modernized Enterprise Data Warehouse (EDW) by expanding the current on-premises Hadoop cluster to accommodate an increased volume of data flowing into the enterprise data warehouse.
- Perform data formatting involves cleaning up the data.
- Assign schemas and create HIVE tables Apply other HDFS formats and structure (Avro, Parquet, etc. ) to support fast retrieval of data, user analytics and analysis
- Assess the suitability and quality of candidate data sets for the Data Lake and the EDW Design and prepare technical specifications and guidelines.
- Act as self-starter with the ability to take on complex projects and analyses independently
- Ensure secure coding practices are adhered to in all phases of the secure development lifecycle.
- Be knowledgeable in all NGC SSA Programs HIPAA compliance requirements and proactively address any HIPAA concerns.
- Become knowledgeable on the HIPAA policies and procedures for the program and ensure awareness of HIPAA breach process.
- 2+ years of proven experience in a range of etc. Big data architectures and frameworks including Hadoop ecosystem, Java MapReduce, Pig, Hive, Spark, Impala
- 2 years of proven experience working with, processing and managing large data sets (multi TB scale). Proven experience in ETL (Syncsort DMX-h, Ab Initio, IBM- InfoSphere Data Replication, etc.), mainframe skills, JCL
- Experience with Apache Hadoop Administration (Preferred Cloudera Framework).
- Experience with Linux Administration (Centos and Red Hat).
- Experience in coding shell scripting in Python.
- Experience in Big Data Storage and File System Design.
- Experience in performing troubleshooting procedures and designing resolution scripts
- Experience with Mainframe development including Java Batch development, Architectural Design/Analysis and Database development.
- Experience in analytic programming, data discovery, querying databases/data warehouses and data analysis. Experience in data ingestion technologies with reference to relational databases (i.e. DB2, Oracle, SQL)
- Experience with advanced SQL query writing and data retrieval using "Big Data" Analytics.
- Experience with enterprise scale application architectures.
- Proven ability to work with senior technical managers and staff to provide expert-level support for the installation, maintenance, upgrading, and administration of full-featured database management systems.
- Knowledge and experience with application integration, thorough understanding of complex network topologies, solid understanding of system security & risk management.