Senior Big Data Engineer
- Full-time
Company Description
Eyeview is a video marketing technology company and the industry leader in outcome-based video marketing. Eyeview delivers superior return on investment through 1-to- 1 video. Through our proprietary VideoIQ® technology, Eyeview easily leverages brand, product and consumer data to create and deliver 1-to- 1 video ads to every consumer and ultimately drive sales. VideoIQ® provides an elemental knowledge of video variables that powers a results-driven decisioning engine, capable of making billions of decisions each day, delivering the most relevant message to every consumer across television, desktop, mobile and Facebook.
Headquartered in New York City, with offices in LA, Detroit, Seattle, Chicago and London, Eyeview serves the nation’s top brands and was ranked as the 2nd fastest growing company in the Tri-State Area by Deloitte’s Technology Fast 500TM.
Job Description
Client: EyeView Video Marketing.
Location: NYC
Job Type: Full-Time
About the Role:
Join our client in building the next generation data-processing platform that helps Fortune 500 companies optimize video ads to directly increase sales. Our platform is built to handle over 10 billion events daily, process all events in real-time utilizing streaming technology, and run predictive modeling to optimize our clients’ ads performance.
You’ll take part in creating a scalable system to handle all our data and creating production level, parallelized big-data solutions. We strive to write robust and scalable code utilizing a combination of Java, Scala, and Python. Some of the frameworks and databases we’re using are: AWS, Amazon Kinesis, Apache Spark, Databricks, Amazon Redshift, DynamoDB, and Aerospike, among others.
We are looking for a Big Data Engineer that will work on the collecting, storing, processing, and analyzing huge sets of data. The primary focus will be on choosing optimal solutions to use for these purposes, then maintaining, implementing, and monitoring them. You will also be responsible for integrating them with the architecture used across the company.
Responsibilities:
Selecting and integrating any Big Data tools and frameworks required to provide requested capabilities
Implementing ETL processes
Monitoring performance and advising any necessary infrastructure/process changes
Qualifications
Requirements:
5+ years hands-on programming experience in data engineering – they do not want a senior person.
Solid understanding of CS fundamentals
Proficiency programming in Java
Proficient in another modern programming language: Scala, Python, etc.
Proficient understanding of distributed computing principles
Ability to solve any ongoing issues with operating the cluster/instances
Proficiency with AWS ecosystem or Hadoop v2 ecosystem (Cloudera/MapR/Hortonworks), MapReduce, etc.
Experience with building stream-processing systems, using solutions such as Spark-Streaming, Storm or Kinesis
Experience with integration of data from multiple data sources
Degree in Computer Science, Computer Engineering or similar
Nice to Haves:
Experience with Spark
Experience with NoSQL databases, such as HBase, Cassandra, MongoDB
Knowledge of various ETL techniques and frameworks, such as Flume
Experience with various messaging systems, such as Kafka or RabbitMQ
Experience with Big Data ML toolkits, such as SparkML, scikit-learn, H2O, etc.
Additional Information
Perks & Benefits:
Unlimited Vacation Policy, Medical, Dental & Vision Benefits, 401K with a match, Cell Phone Reimbursement, Employee Stock Purchase Plan, Free drinks & snacks, Sports teams, Catered Company lunches every Friday. Casual environment – jeans are ok. No shorts and tee shirts. Offsite, team building events.
Client is an advertising marketing company.
They don’t need Informatica or another ETL tool. But, they should know ETL and its complex processes and data requirements.
150 people on their team with 10 billion events/transactions.
4 teams: Data team, Bidding team, media team – building tools, platform team: UI Developers, UX Engineers, QA, full stack, video.
Interviews: Phone, possible on line test, face to face – 2- 2.5 hours on site in NYC.
All your information will be kept confidential according to EEO guidelines.