Sr Spark Developer

Company Name: CyberCoders
Location: Redwood City
Date Posted: 26th Mar, 2018
  • Work on our data pipeline, ETL systems, and real-time data
  • Come up with solutions for scaling data infrastructure
  • Define and extend an interface for expressing domain-specific analytic queries
  • Translate product requirements into analytic queries and data structures, extending the processing framework and query language as needed
  • Optimize computational primitives for performance and parallelism
  • Architect for performance and scalability: we deliver a real-time experience to customers at scale. You should have experience with resilient, redundant, and performant systems, and know how to optimize at scale.
  • Be pragmatic: Handling multiple projects at the same time is a daily occurrence; you must be able to assess tradeoffs in an efficient manner and handle deliverables on short deadlines
  • Strive for innovation: Continuous learning is essential to this role; you are encouraged to add to our institutional knowledge and find ways to incorporate the latest and the greatest in our system
  • Be a team player: Everything happens within a team; you will have the opportunity to work with domain experts (understand domains and users), product managers (define roadmaps and scopes), and the broader engineering team (creates infrastructures and features essential for our models)
  • BS in CS or related field
  • 5+ years of industry experience building data pipelines, analytics systems, business intelligence reports, database internals, or distributed processing engines
  • Strong in Java
  • Strong in concurrent programming
  • Strong understanding of relational databases
  • Experience with frameworks like Spark, MapReduce, Hive, Pig at scale
  • Experience with distributed systems, scalability, and fault tolerance
  • Experience with end-to-end performance profiling and optimization (Java, database, OS, network)
  • Amazon Web Services experience highly desirable