Hadoop Data Engineer

Company Name: VISA
Location: Palo Alto, California, United States
Date Posted: 14th Dec, 2016


  • Work with data team to efficiently use hadoop infrastructure to analyze data, build models, and generate reports/visualizations
  • Create and optimized distributed algorithms to scale out recommendation engine
  • Integrate massive datasets from multiple data sources for data modeling
  • Implement methods for automation of all parts of the predictive pipeline to minimize labor in development and production
  • Formulate business problems as technical data problems while ensuring key business drivers are captured in collaboration with product management
  • Knowledge in machine learning algorithms especially in recommender systems


  • BS, MS or PhD in Computer Science/Engineering or Electrical Engineering
  • 5+ years of working experience in specifically Hadoop Data Engineering
  • Strong grasp of one or more programming languages such as Java, C++, Python etc.
  • Strong understanding of algorithms and data structures
  • Strong understanding of multi-threading and resource management
  • Experience in writing optimized MapReduce code
  • Experience working with large datasets in Hadoop (using scripts and tools - Pig/Hive etc.)
  • Familiar with machine learning algorithms, including regression, GBM, recommendation systems, clustering, etc.
  • Experience working with tabular databases like MySQL
  • Strong analytics and problem solving capabilities combined with ambition to solve real-world problems
  • Strong verbal and written communication skills