Staff Hadoop Systems Engineer
- Ashburn, VA, USA
Common Purpose, Uncommon Opportunity. Everyone at Visa works with one goal in mind – making sure that Visa is the best way to pay and be paid, for everyone everywhere. This is our global vision and the common purpose that unites the entire Visa team. As a global payments technology company, tech is at the heart of what we do: Our VisaNet network processes over 13,000 transactions per second for people and businesses around the world, enabling them to use digital currency instead of cash and checks. We are also global advocates for financial inclusion, working with partners around the world to help those who lack access to financial services join the global economy. Visa’s sponsorships, including the Olympics and FIFA™ World Cup, celebrate teamwork, diversity, and excellence throughout the world. If you have a passion to make a difference in the lives of people around the world, Visa offers an uncommon opportunity to build a strong, thriving career. Visa is fueled by our team of talented employees who continuously raise the bar on delivering the convenience and security of digital currency to people all over the world. Join our team and find out how Visa is everywhere you want to be.
This is a Systems Engineering position focused on Big Data and Hadoop applications. The successful candidate will be a technical expert who works directly with multiple project development teams in order to translate business requirements to infrastructure design and implementation. The candidate will be responsible for performance tuning and increase operational efficiency on a continuous basis. You will be responsible for monitoring the health and effective usage of the Data platform. You will work closely with development and operation teams for key deliverables of the project or initiative but also ensuring availability, security, scalability and resiliency. The candidate should work towards standardization and automate all the manual tasks involved in building clusters or performing SOPs. The candidate should be able to perform capacity planning on large multi-tenant clusters.
- Minimum of 5 years working as a Systems Engineer specialized in managing Big Data Infra
- In-depth hands-on experience in Linux; either RHEL or Ubuntu
- Experience in automation using Shells, AWK and/or Ansible
- Experience in configuring and setting up Hadoop clusters, and providing support
- Experience in Map Reduce tuning, data node, NN recovery, etc.
- Experience in setting up Kerberos authentication on Linux
- Experience with Hadoop (Map Reduce, Hive, Pig, HDFS, H-catalog, Zookeeper and OOzie)
- Experience in designing and managing x86 hardware for big data. Dell or Hp is preferable
- Hands-on Experience in SQL (Oracle) and NoSQL Databases (HBASE/Cassandra/Mongo DB)
- Knowledge on Spark and Kafka is a plus
- Hands on experience with HDFS and GPFS is a plus
- Experience on Cloudera and/or Hortonworks distribution preferred
- Excellent oral and written communication and presentation skills
- Must be analytical and has very strong problem solving skills
- Self-driven with the ability to work independently and as part of a team with proven track record developing and launching products at scale
- Minimum of four year technical degree required
All your information will be kept confidential according to EEO guidelines.
This position must sit onsite in Ashburn, VA