Senior Developer, Big Data

Company Name: Macy's
Location: San Francisco, CA
Date Posted: 01st Dec, 2016
Description:

Essential Functions:

• Build real time stream-processing systems, using solutions such as Storm and/or Spark-Streaming

• Design, build and support pipelines of data ingestion ,transformation, conversion and  validation

• Design and support effective storage and retrieval using NoSQL databases, such as HBase, Cassandra, MongoDB

• Design and implement best practices for cloud based cluster deployments of Hadoop, Spark, and other BigData eco-system tools

• Develop architecture and design patterns to process and store high volume data sets

• Build continuous integration and test driven development environment

• Research and deploy new tools, frameworks and patterns to build a sustainable big data platform

• Regular, dependable attendance & punctuality

 

Qualification:

Education/Experience:

• Bachelor's Degree in Computer Science/Engineering required

• 7+ years of experience in large-scale software development

• 4+ years of experience with data analytics and high volume data processing

• Hands-on development experience working on large projects with Hadoop stack (HIVE, Pig, Hadoop streaming, Spark streaming) and MapReduce required

• Experience with messaging & Complex event processing systems such as Kafka, Storm and Spark

• Experience with Machine Learning algorithms a plus

• Statistical analysis with R, Python or similar a plus

• Strong object-oriented design experience, applied use of design patterns, and expert knowledge of the Java, Scala and Python technology platform

• Cluster upgrades and maintenance automation

 Communication Skills:

• Strong communication and collaboration skills

 Reasoning Ability:

• Strong analytical and problem solving skills, good understanding of different data structures, algorithms and their usage in solving business problems

• Proven ability to work independently; designing, developing and deploying solutions, and to deliver projects on time with minimal direction

 Other Skills:

• Innovative and creative, able to champion new ideas and initiatives within an energized,  collaborative and entrepreneurial environment

• Ability to visualize data using various web based technologies

 

Apply Now