Senior Software QA Automation Engineer – Big Data
- Aundh, Pune, Maharashtra, India
PubMatic is the automation solutions company for an open digital media industry. Featuring the leading omni-channel revenue automation platform for publishers and enterprise-grade programmatic tools for media buyers, PubMatic’s publisher-first approach enables advertisers to access premium inventory at scale. Processing nearly one trillion ad impressions per month, PubMatic has created a global infrastructure to activate meaningful connections between consumers, content and brands. Since 2006, PubMatic’s focus on data and technology innovation has fueled the growth of the programmatic industry as a whole. Headquartered in Redwood City, California, PubMatic operates eleven offices and six data centers worldwide.
PubMatic is one of the leaders in tech stack when it comes to Big data infrastructure and data processing. We at PubMatic process more than 150 Bn ad impressions a day which contribute to around 100 tera bytes of uncompressed data. To maintain and process this raw data we have our own data centres across the globe and do in house ingestion, ETL and aggregation using a mammoth Hadoop infrastructure on top of thousands of BareMetal nodes built from scratch. Having most things built in house, PubMatic is a very early adaptor of new technologies that come along in the big data space.
We are looking for individuals with knowledge and experience of working with distributed environments for becoming a part of our big data test engineering group. The individual will also be responsible for automating various big data flows, performance engineering and fine tuning the big data pipeline and making sure the quality of the above mentioned giant business critical infrastructure is intact. Individual will also get an opportunity to work with the high speed GPU machines used for rapid computing of large data sets.
- Should have minimum 3 years of experience on working BigData technologies
- Good Programming skills.
- Hands on Experience in Automating Backend Applications (e.g. db, REST API's)
- Hands on experience with Automating any backend applications (e.g db , server side).
- Knowledge of relational databases and SQL
- Good debugging skills.
- Strong working experience working in Linux/Unix environment.
- Strong understanding of testing methodologies.
- Hands on experience in working on Big Data technologies like Hadoop, Spark
- Hand on experience in working with ETL Testing
- Hands on experience in QA Automation Framework development & Design & Strong hold on data structures.
- Preferred language Python/Shell Scripting
- Strong Understanding of OS and performance benchmarking
- Quick learner and good team member with positive attitude.
- Good verbal and written communication skills.
Duties and Responsibilities:
- Testing big data ingestion and aggregation flows using spark shell and related queries
- Developing automation framework using programming languages such as python and automate the big data workflows such as ingestion, aggregation, ETL processing etc
- Debugging and troubleshooting issues within the big data ecosystem
- Set up the Big data platform and Hadoop ecosystem for testing
- Define test strategy and write test plan for the data platform enhancements and new features/services built on it.
- Define the operating procedures, service monitors and alerts and work with the NOC team to get them implemented.
- Responsible for system & performance testing of the data platform and applications
- Solve problems and establish plans and provide technical consultation in the design, development and test effort of complex engineering projects
- Review product specifications and write test cases, develop test plans for assigned areas.
- Identifies issues and technical interdependencies and suggest possible solutions.
- Recreate complex customer and production reported issues to determine root cause and verify the fix.
Primary (Mandatory) Skills:
- QA with good hands on experience with Unix/Linux
- QA experience in Networking and/or Big Data domain.
- Automation experience into Python.
- QA Methodologies understanding.
Secondary Skills (Good to have):
- Experience in Big data platform & data analytics testing is an advantage.
- The Senior Software Engineer will have the end to end ownership of feature starting from testing, automation (if applicable), deployment and helping with monitoring of the feature(s)
Coronavirus notice: PubMatic is actively working to ensure candidate and employee safety. Currently, all hiring and onboarding processes at PubMatic will be carried out remotely through virtual meetings until further notice.
Benefits: Our benefits package includes the best of what leading organizations provide, such as stock options, paternity/maternity leave, healthcare insurance, broadband reimbursement. As well, when we’re back in the office, we all benefit from a kitchen loaded with healthy snacks and drinks and catered lunches and much more!
Diversity and Inclusion: PubMatic is proud to be an equal opportunity employer; we don’t just value diversity, we promote and celebrate it. We do not discriminate on the basis of race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status.