Software Engineer - Big Data Specialist
Company Name: JP Morgan Chase & Co
Location: New York
Date Posted: 12th Jan, 2018
- You’ll be a key member in the application design and development of the distributed compute cluster, web server and database setup, and accessory scripts and processes.
- Development will include logical data modelling, setting up a cluster, partitioning, ingestion of real time streaming feeds and an interface to allow 100’s of simultaneous interactive web requests. Sub second responses are expected.
- The team will develop solutions with both on-the-fly analytic calculations and pre-computed analytics efficiently stored for quick reads. The platform will ingest streaming and stored content, and store both raw and computed content. Compute will include aggregations, filtering, bespoke analytic calculations across bonds and indices spanning 30 years of historic data.
- 7+ years of Java Experience
- 3+ years of experience in Big Data technologies
- Experience setting up and maintaining distributed clusters
- Experience implementing distributed and scalable algorithms (Spark, Hadoop)
- Experience with Cassandra, Spark, HDFS, Impala, HBASE, Kafka. Vertica a plus.
- Proficiency in Avro, Parquet, JSON Formats.
- Hands on programming experience with YARN/Spark using Java/Scala
- Experience with Map Reduce
- Experience with stream processing
- Experience developing optimum strategies for distributing data over a cluster
- Demonstrate strong interest in process automation and orchestration.
- Very good understanding of relational as well as columnar databases
- SQL application development
- Experience developing object-oriented applications in Java
- Experience developing on web technologies (Apache/Tomcat)
- Experience in working with streaming data and time-series analytics
- Experience working with different caching strategies
- Experience working in development teams using agile techniques
- BA or MS degree in Engineering, Computer Science or Information Technology