Lead Big Data Developer, Data Platform as a Service
- Palo Alto, CA, USA
Common Purpose, Uncommon Opportunity. Everyone at Visa works with one goal in mind – making sure that Visa is the best way to pay and be paid, for everyone everywhere. This is our global vision and the common purpose that unites the entire Visa team. As a global payments technology company, tech is at the heart of what we do: Our VisaNet network processes over 13,000 transactions per second for people and businesses around the world, enabling them to use digital currency instead of cash and checks. We are also global advocates for financial inclusion, working with partners around the world to help those who lack access to financial services join the global economy. Visa’s sponsorships, including the Olympics and FIFA™ World Cup, celebrate teamwork, diversity, and excellence throughout the world. If you have a passion to make a difference in the lives of people around the world, Visa offers an uncommon opportunity to build a strong, thriving career. Visa is fueled by our team of talented employees who continuously raise the bar on delivering the convenience and security of digital currency to people all over the world. Join our team and find out how Visa is everywhere you want to be.
You may have noticed that payment has become a very active/hot area in the last several years. Given the strong demands and innovation, this will be a very exciting area in the next 5 to 10 years. VISA is clearly a very strong leader in the payment industry, and is in a fast pace of technology transformation. For a payment system to work well, advance technologies such as data platform, big data, data mining, machine learning, cloud, etc. are critical. In VISA, we have all of these. The Data Platform Department is in the center for many of these technologies and development.
The Data Platform team is a key component of Visa's Technology organization that provides an enriched and valuable ecosystem of data platform and data services that drive innovation for our partners and clients, within Visa and globally. Data is the most valuable asset in Visa. The Data Platform Department is dedicated to develop advanced technology (e.g. Cloud, Machine Learning and Big Data), systems and services to make data secure, high quality, rich, fast, and easy to use, therefore enabling Visa the ability to leverage its data asset in an effective and timely manner to maximize technology/business development and differentiate Visa from others in the payment industry. The department maintains tens of petabytes of data supporting over 100 services for various internal lines of business and external clients. Our technical stack has two layers: Platform as a Service (PaaS) and Data as a service (DaaS).
Visa is seeking a Lead Software Engineer in the Data Platform department to be part of the Big Data engineering team to lead, build and manage Visa’s Big Data platforms which are key assets to the company in the data Platform as a Service (PaaS) layer.
This individual will act as a trusted technical lead and be comfortable in a fast-paced environment with significant changes; developing/implementing strategies and processes that enable the organization to innovate and grow fast. This position reports to the head of PaaS Engineering group and will be based in Palo Alto, CA. The ideal candidate will have strong leadership with rich technical hands on experience in the big data and/or large scale software platform areas.
- Lead, mentor, and grow a team of strong and enthusiastic developers to manage large scale open source data platforms (such as Hadoop and Spark) and develop tools, fixes to bugs/issues and contribute back to the community.
- Getting into the source code of the related open source packages deeply and being able to make changes/improvements are important part of the day to day responsibilities, including troubleshooting, diagnostics, and problem solving on the large scale platforms
- Apply architectural patterns and principles to achieve reusability, testability, scalability and maintainability
- Articulate technology solutions as well as explain the competitive advantages of various technology alternatives.
- Apply creative thinking/approach to determine technical solutions that further business goals and align with corporate technology strategies, keeping in mind performance, reliability, scalability, usability, security, flexibility, and cost.
- Work on highly complex issues around scale and performance with large varied workloads, optimize, tune and manage health of the platform
- Collaborate with the key stakeholders, other product development groups and software vendors to identify, define, develop and implement solutions for complex business and technical requirements.
- Drive internal proof of concept initiatives. When needed, quickly design and implement a prototype of a system or component with a proper architecture, and then hand over to (may lead) a small group of devs to finish.
- Build relationships with key architects across technology organizations and collaborate on promoting architectural best practices across technology
- Provide mentorship and help team growth especially on technical side.
You get to:
- Contribute directly to the development of Apache Hadoop HDFS, Yarn, Spark and other open source projects
- Write high quality, maintainable code. Mentor other engineers
- Build scalable big-data distributed data processing systems. Using Hadoop, MapReduce, Java, or Spark, develop infrastructure for processing huge amounts of data
- Collaboratively architect the system design along with others, including leveraging Apache open source
- Develop, contribute, and commit to open source projects. Being or becoming a committer. We are big users of open source and want to give back to the community.
- Experienced technologist with a minimum of 12+ years of software development experience including 5+ years of application or platform architecture experience with deep technology expertise in the following:
- Extensive experience on architecture design and development of large scale platforms and application with simple solutions
- Passion for exceptionally high quality and maintainable code with strong experience in working with Core Java, C++ or equivalent at the data platform level
- DD/BDD experience, including experience with refactoring, testing frameworks (JUnit, xUnit and similar) and Continuous Integration and Continuous Delivery
- Experience with SOLID principles and object-oriented patterns, and developed applications
- Solid foundation in data structures, algorithms and software design with strong analytical and debugging skills, and understanding of continuous deployment into a modern container based environment
- Extensive hands on design/coding experience/skills using Java, J2EE, C++, C#. Strong on elegant coding, can write very solid prototypes or core components quickly.
- Object-Oriented design and development, SQL/noSQL technology, Web Services (REST/ SOAP)
- Service Oriented Architecture
- Experience on application/platform instrumentation, measurement, log data processing, and monitoring.
- Familiar with Database, ETL and Business Intelligence technologies like Oracle, DB2, SQL Server, etc.
- Deep understanding of how distributed systems work and how to efficiently model for a mixed workload of heavy write traffic with bursts of analytic style reads.
- Deep knowledge and hands on experience on big data and cloud computing technologies including Hadoop, HDFS, Yarn, Spark, Hive, HBase, etc. This includes deep understanding of the source code and architecture of these open source packages, experience in making changes including bug fixes and improvement, and/or a strong ability to get into this level of knowledge with confidence quickly based on experience on other large scale software systems.
- Participated as a committer or a contributor in a top level Apache project like HDFS/Yarn/Spark etc.
- Strong at identifying problems, solving complex problems with simple solutions.
- Demonstrated proficiency and experience in one or more of the following areas: application architecture, solutions architecture, software systems architecture, and data architecture.
- Strong on driving for results and self-motivated, strong learning mindset, with good understanding of related advanced/new technology. Keep up with the technology development in the related areas in the industry, which could be leveraged to enhance current architectures and build durable new ones.
- Outstanding verbal, written, presentation, facilitation, and interaction skills, including ability to effectively communicate architectural issues and concepts to multiple organization levels and executive management.
- Bachelor's degree in Computer Science or related technical discipline required; advanced degree is a plus.
- Fast learner on both technical side and business side
- Strong customer centric mindset
- Strong leadership and team player.
- Strong skills on mentoring/growing junior people
You are the kind of engineer that loves solving the hardest data scaling problems and are comfortable rolling up your sleeves and going to the depth of the problem.
Visa will consider for employment qualified applicants with criminal histories in a manner consistent with EEOC guidelines and applicable local law.