In Summary we seek a lead/Principle data engineer, hands-on mentor for 2 data engineers, expertise must include heavy SQL, Python, Spark, AWS and snowflake. There is a coding assessment. You will work on HBO MAX data analytics, work with analytics, marketing and other departments.
The Data and Engineering Services team within the Media Technology & Information Services (MTIS) group is looking to add a Data Engineering Manager to lead the architecture, design and implementation of projects focused on collecting, aggregating, storing, reconciling and making data accessible from disparate sources to enable analysis and decision making. As an independent contributor, this individual will collaborate within a team of technologists to produce enterprise scale solutions to meet our clients needs.
In this role, responsibilities will touch on all stages of the software engineering lifecycle: understanding customer requirements, designing the application, reviewing code, and monitoring the system for issues. There will also be an opportunity to exercise highly developed collaboration abilities interacting with other technology teams on broader cross-organization initiatives.
Be a thought leader across the Data Engineering & Services organization to build new and improved data tools and services that can scale with the company.
Take loosely defined business questions and translate them into clearly defined technical/data specifications for implementation.
Write complex ETL (Extract / Transform / Load) processes, design and develop solutions for real-time and offline analytic processing on AWS-hosted Data and Analytics Platform.
Ensure that data pipelines are scalable, repeatable and secure.
Deliver quality work on defined tasks with little oversight and direction.
Ensure all deliverables are of high quality throughout the project by adhering to coding standards and best practices and participating in code reviews.
Implement activities that generally impact multiple components / processes and the work of own and possibly other units / teams / projects.
Interacts with internal and external peers and management to share highly complex information related to areas of expertise and/or to gain acceptance of new or enhanced technology / business solutions.
Mentor less experienced members of the team.
6+ years of experience designing and implementing large-scale, complex, data-driven applications on the cloud, preferably on AWS.
6+ years of hands-on programming experience with Python, frameworks (Spark and Airflow) and streaming technologies (Kafka/Kinesis)
6+ years of hands-on experience using SQL to perform complex data manipulation
3+ years of experience modeling data warehouses (Kimball dimensional modeling)
An understanding of technologies and design patterns in fields such as: microservices, streaming/queuing systems, key-value stores and high-performance solutions (vectorization, task and data parallelism)
Ability to deal with ambiguity and work with rapidly-changing business data
Appreciation of the need and ability to design for flexible systems that adapt to evolving user requirements