Big Data Engineer
Company Name: Prudential
Location: Mountain View, CA, US
Date Posted: 09th Jan, 2018
- Build distributed, scalable, and reliable data pipelines that ingest and process data at scale and in real-time.
- Collaborate with other teams to design and develop and deploy data tools that support both operations and product use cases.
- Own product features from the development phase through to production deployment.
- Troubleshoot data issues, deep dive into root cause analysis, recommend, test and implement solutions
- Develop and document technical requirements and solutions
- Participate in design and code reviews
- Troubleshoot issues making recommendations and delivering on those recommendations
- BS in Computer Science or related area
- Around 8 years software development experience
- Minimum 5 years of active SQL development, Performance tuning, Data Modeling and design preferably in Oracle.
- Minimum 1-2 Year Experience on Cloud computing, AWS preferable.
- Strong automation skills using Python and Ansible is highly preferred
- Good analytical and programming experience with Java, Oracle, SQL
- Flair for data, schema, data model, how to bring efficiency in big data related life cycle
- Proficiency with agile or lean development practices
- Strong object-oriented design and analysis skills
- Excellent written and verbal communication skills
Top skill sets / technologies in the ideal candidate:
- Database – Oracle 11g / 12c, complex SQL queries, performance tuning concepts, Backups, Recovery, DR, BCP
- Programming language -- Java, Python, SQL
- AWS -- RDS, EC2, RedShift, S3 and others. AWS cloud Data migration, AWS Security
- NoSQL -- HBase, MongoDB, Cassandra, Riak
- ETL Tools -- Data Stage, Informatica
- Code/Build/Deployment -- Ansible, Git, SVN, Maven, sbt, Jenkins, Bamboo, Terraform
- AWS Associate / Solution Architect certified
- Batch processing -- Hadoop MapReduce, Cascading/Scaling, Apache Spark, AWS EMR
- Stream processing -- Spark streaming, Apache Storm, Flink
- Excellent communication and decision making skills are essential.
- Strong analytical, problem solving and decision-making skills.
- Zeal to lean new technologies, frameworks and appetite for growth
- Identify project risks and recommend mitigation efforts.
- Identify project issues, communicate them and assist in their resolution.
- Assist in continuous improvement efforts in enhancing project team methodology and performance.
- Cooperative team focused attitude