Big Data Framework Developer
Company Name: Apple
Location: Santa Clara Valley, CA
Date Posted: 10th Nov, 2017
- Architect, design and build a Big-Data Framework that provides proper abstractions for common building blocks of big data analytics systems. The abstractions will be optimally assembled into data processing and machine learning pipelines in an easy to use manner by developers and end business users. This framework should make Big-Data Technology agnostic to the users, so users can concentrate on the business logic.
- The framework will have a clear distinction about the cold data, warm data and hot data, and the framework, while serving the user with a unified interface, has the ability to move the data behind the scene transparently. Essentially the framework is to implement the tiered storage architecture.
- The framework will federate the data from different systems where the data is retrieved in parallel from those systems.
- Designs and develops the features to address the above stated requirements Designs Self - Serve data structures, sets up the Data Pipe Lines for these structures;
- Schedules the data pipe lines for daily and history processing; Works closely with cross functional teams to gather requirements.
Key Qualifications -
- 5+ years of experience in Scala, core Java and/or Python
- 3+ years of experience in Spark systems
- 3+ years of Experience in machine learning, for example, Tensor flow, Turi, Spark ML lib, and R.
- Good knowledge in Hadoop eco-systems
- Practical knowledge in design patterns
- Expertise in architecting, designing and developing big data frameworks for analytics and data management
- Direct experience with highly available systems
- In depth knowledge of developing large distributed systems
BS in Computer Science, Information Technology, or EE, or relevant industry experience is required