Hadoop Data Analyst
Company Name: Shane Co
Date Posted: 11th Feb, 2018
- Influence the design and implementation of data platforms and processes to capture, integrate, analyze and distribute information in the enterprise.
- Support business and functional requirements gathering for reporting and analytics solutions.
- Identify strategy, tools required, and design and implement solutions for data ingestion, storage, processing and provisioning using data platform.
- Design and implement distributed data processing pipelines using Spark, Hive, Sqoop, Python, and other tools and languages prevalent in the Hadoop ecosystem.
- Lead the design of data models for storing structure and unstructured data on data platform.
- Oversee data extraction, transformation and load from variety of data stores.
- Responsible for implementation of data governance and data security components on data platform.
- Data governance - including data modeling, data cleansing and normalization, security.
- Develops and maintains technical documentation.
- Translates business concepts to technical implementations to drive alignment and decisions making.
- Keeping track of Hadoop Cluster connectivity and security.
- Resolve issues regarding development, operations, implementations, and system status.
- Defining job flows within the Hadoop environment.
- Work with team to extract data relevant for analysis.
- Plan and develop big data analytics projects based on business requirements.
- Ability to work efficiently with and support our offshore team (India).
- Bachelor's degree or equivalent experience in MIS, CS or related field. Master's preferred.
- Three years in an enterprise Hadoop environment.
- Ability to work with team members and clients to assess needs, provide assistance and resolve problems, using excellent problem-solving skills, verbal/written communication, and the ability to explain technical concepts to business people
- Proven ability to explore and dig into unknown spaces, with little supervision, while being technically driven and curious
- Proven comfort with agile development methodologies.
- Demonstrated strong work ethic, ability to take ownership, and habit of excellence.
- Proven understanding of cloud and distributed systems principles, including load balancing, networks, scaling, in-memory vs. disk, etc.; and experience with large-scale, big data solutions, such as Hadoop
- Familiarity with data loading tools like Flume, Sqoop and knowledge of workflow/schedulers like Oozie.
- Experience with programing languages Java or Scala, Python, Node.js, SQL.
- Demonstrated strong communication skills, interpersonal skills, and ability to be a team player.