Lead Engineer- Data Platform
- Bengaluru, KA, India
You may have noticed that payment has become a very active/hot area in the last several years. Given the strong demands and innovation, this will be a very exciting area in the next 5 to 10 years. VISA is clearly a very strong leader in the payment industry, and is in a fast pace of technology transformation. For a payment system to work well, advance technologies such as data platform, big data, data mining, machine learning, cloud, etc. are critical. In VISA, we have all of these. The Data Platform Department is in the centre for many of these technologies and development.
The Data Platform team is a key component of Visa's Technology organisation that provides an enriched and valuable ecosystem of data platform and data services that drive innovation for our partners and clients, within Visa and globally. Data is the most valuable asset in Visa. The Data Platform Department is dedicated to develop advanced technology (e.g. Cloud, Machine Learning and Big Data), systems and services to make data secure, high quality, rich, fast, and easy to use, therefore enabling Visa the ability to leverage its data asset in an effective and timely manner to maximise technology/business development and differentiate Visa from others in the payment industry. The department maintains tens of petabytes of data supporting over 100 services for various internal lines of business and external clients.
Visa is seeking a Lead Software Engineer in the Data Platform department to act as one of key technology leaders to build and manage Visa’s technology assets in the Integrated Data Service Platform (IDSP) organisation. IDSP develops large scale distributed data serving platforms for mission critical applications and new initiatives, and run them as a service with secure, reliable and high speed API or streaming access. For example, the team builds and manages a peta-byte scale transaction history store that powers Visa dispute services. The IDSP team is also responsible for building fast and scale-able solutions to data encryption/decryption service and privacy preservation services.
vTrust is a new program for Visa’s privacy program. As a Lead Engineer, you will be working on architecture, design and development of vTrust System which is one of the critical software system to manage the privacy metadata, build and run tools for sensitive data scanning in large peta byte data stores, and manage/build legal query databases for compliance. You will have the opportunity to lead, participate, guide, and mentor other engineers in the team on design and development. This position reports to the Director of Data Platform and will be based in Bangalore, India.
Primary responsibilities will include:
- Lead, design and develop of privacy metadata system and integration with several in house systems and as well as external systems/products in the privacy domain
- Develop high scalable, low latency sensitive data scanning solution using ML and AI for very large data stores/application in order to comply with GDPR, CCPA , LGPD and other ongoing privacy standards/compliances
- Design and development of common services ( distributed tracing, distributed logging, monitoring, auditing, operations metrics, workflow, reports, APIs) which can be reused across the group
- Be a role model to drive/build privacy engineering systems within the group
- Actively participate in all phases of software development lifecycle: analysis, technical design, planning, development, CI/CD, release, post production/escalation support
- Drive technology standard and best practices in the Data Platform areas, and lead key projects in the privacy domain
- Collaborate with the key stakeholders like Global Privacy Office, other product development groups and software vendors to identify and develop solutions for complex business and technical requirements.
- Drive internal proof of concept initiatives , design and prototype of a system or component with a proper architecture and lead from end to end.
- Provide mentorship and help team growth especially on technical side.
Experienced technology leader with a minimum of 14+ years of software development experience with deep technology expertise in the following
- Architecture design and development of large scale platforms and applications with simple solutions
- Deep knowledge and hands on experience on big data and cloud computing technologies including Hadoop, Spark, Hive, HBase, etc.
- Experience in streaming platforms (Kafka) and messaging systems
- Deep knowledge and hands on experience in Machine Learning and Deep Learning technologies (Spark ML, PyTroch, Tensorflow, NLP )
- Extensive hands on design and high performance/quality coding experience using languages like Java or Scala
- Experience in SQL/NoSQL technologies (Cassandra, Elastic Search, MongoDB) and familiar with databases like MySQL, Oracle, DB2
- Experience with Web Services (REST/ SOAP) and APIs
- Experience on application/platform instrumentation, measurement, log data processing , monitoring and observability
- Technical background in data with deep understanding of issues in multiple areas such as data acquisition and processing, data management, distributed processing, and high availability is required.
- Outstanding verbal, written, presentation, facilitation, and interaction skills, including ability to effectively communicate architectural/design issues and concepts to multiple organization levels and leadership team.
- Bachelor's degree in Computer Science or related technical discipline required; advanced degree is a plus.
- Strong skills on mentoring/growing interns , graduate hires and senior software/data engineers
- Experience in Payment industry and privacy domain is a plus.
All your information will be kept confidential according to EEO guidelines.