Data Engineer, Apache Spark

Company Name: Capital One
Location: San Francisco, California
Date Posted: 25th Nov, 2016
  • You yearn to be part of cutting edge, high profile projects and are motivated by delivering world-class solutions on an aggressive schedule
  • Someone who is not intimidated by challenges; thrives even under pressure; is passionate about their craft; and hyper focused on delivering exceptional results
  • You love to learn new technologies and mentor junior engineers to raise the bar on your team
  • It would be awesome if you have a robust portfolio on Github and / or open source contributions you are proud to share
  • Passionate about intuitive and engaging user interfaces, as well as new/emerging concepts and techniques in web design
  • You use, or are passionate about learning, Spark, Scala, Akka, Cassandra, Accumulo, HBase, Hadoop, HDFS, AVRO, MongoDB, and Mesos – and anything else that will help you solve big, messy problems 

- Bachelor’s Degree or military experience

- At least 4 years of experience in C, Java, or Scala

- At least 4 years of experience with Unix/Linux systems with scripting experience in Shell, Perl or Python

- At least 3 years of experience building data pipelines

- At least 2 years of experience with leading big and fast data technologies like Spark, Scala, Akka, Cassandra, Accumulo, Hbase, Hadoop, HDFS, AVRO, MongoDB, Mesos

Preferred Qualifications:

 - Master's Degree

- Experience with recognized industry patterns, methodologies, and techniques

- Familiarity with Agile engineering practices

- 2+ years of experience with Spark, Scala, and/or Akka

- 2+ years of experience with Spark Streaming, Storm, Flink, or other Stream

  Processing technologies

- 3+ years of experience with NoSQL implementation (Mongo, Cassandra, etc. a


- 3+ years of experience developing Java based software solutions