Unix Systems Administrator with Hadoop

  • San Jose, CA
  • Full-time

Company Description



eBay is a technology company driving the ‘future of commerce’!  We are enabling consumers to buy anything, anywhere, anyhow, thus unifying the online buying experience across the mobile, local, social and digital space. 


With more than 100 million active users globally (as of Q4 2011), we are the world's largest online marketplace. Our collective impact on ecommerce is staggering: In 2011, the total value of goods sold on eBay was $68.6 billion -- more than $2,100 every second. And we need you!


Even after 17 years (which is like, eons in Internet time), eBay is committed to innovation and growth. We're here to stay, and playing to win in a competitive space. Financially, we're rock solid. As a company and as individuals, we're growing, and making a difference every day, come join us!

Job Description

• Provide day to day production support of our Hadoop infrastructure including; Implement new Hadoop hardware infrastructure Cluster maintenance Hadoop Cluster capacity planning HDFS support and maintenance Adding and removing cluster nodes Troubleshooting MapReduce Jobs as required Cluster Monitoring and Troubleshooting


• Provide system hardware architecture and configuration recommendations, perform vendor evaluation and drive system provisioning automation

•  Provide engineering and design support for DPE Operations build, configuration, monitoring and supportability

• Work with Architecture and Development teams to test and benchmark new versions, patches and software components for functionality and reliability

• Contribute to the selection, prototyping, implementation and support of tools such as capacity planning and system monitoring for large scale data environments

• Review, develop, and implement strategies that preserve the integrity, availability, stability, and scalability of DPE Operations systems and designs, consistent with cost (Capital and Labor) constraints

• Engage with the architecture team to design system and architecture standards to support internal and external development groups

• Work with internal and external technical leaders to develop innovative solutions that meet market needs with respect to functionality, performance, scalability, and reliability, realistic implementation schedules, and adherence to architectural goals and principles

• Review existing software/hardware architecture and identify areas for improvement in the areas of scalability, maintainability, and performance

• Ongoing corrective and preventative maintenance to improve reliability and availability


 Hadoop,Cluster, Troubleshooting,MapReduce, Sqoop, Pig, Hive, Flume, HBase a plus


• Preparation of proposals, reports, architectures, designs, specifications, and operational documentation


• Perform training and be on-call to investigate system failures and assist in restoring functionality

• Fosters relationships of trust, respect, judgments, and maturity with peers, customers, management


• Hands on experience with the Hadoop stack (MapReduce, Sqoop, Pig, Hive, Flume, Hbase a Pluss) Knowledge of NoSQL platforms will be good to have.

Expertise in typical system administration and programming skills such as storage capacity management, performance tuning, system dump analysis.

Demonstrated ability to apply problem analysis and resolution techniques to complex system problems.


• Expert level experience with large scale analytical data environments build and support including thought leadership, design, configuration, installation, capacity planning, capacity management, and performance tuning and monitoring.

• Expert level knowledge (hands-on) with the overall eco-system (Solaris, Linux, etc.)

• Expertise in UNIX architecture, systems administration and UNIX tools

• Knowledge and experience with parallel ETL tools

• Extensive experience in designing and implementing large scale analytics environments with particular attention to maintainability, scalability and high availability

• Experience with versioning, change control, problem management

• Solid experience with monitoring frameworks, and analyzing production platforms, UNIX servers and mission critical systems with alerting and self-healing systems

• Expert level capacity planning and capacity management skills

• Strong presentation and leadership skills

• Solid understanding of how businesses operate and use this context to drive innovation and ethical decision-making

• Proven results oriented engineer with a laser sharp delivery focus

• Sought out as an expert in all aspects of technical field with a solid business context

• Proven customer centricity who can foster effective partnerships with colleagues, customers, management

• Master level problem solving, root cause analysis, and comprehensive and strategic resolution to complex engineering problems 


Other Related 4yr Technical Degree

8 years' work related history.