P
Lead Cloud Data, Streaming and AI Platforms Engineer
Not specifiedNot specifiedDevOps & SRE
Lead a specialist platform team and own enterprise data, streaming, MLOps and AI platforms across on-premise and cloud environments.
Responsibilities
- Manage and mentor a specialist platform team.
- Own data and streaming platforms across on-premise and cloud environments.
- Manage Confluent Kafka, including Kafka Connect, Schema Registry, KSQL, REST Proxy and Confluent Cloud.
- Own Azure Synapse, MLOps, DataOps and GPU platform infrastructure.
- Lead platform implementation, upgrade and migration projects.
- Troubleshoot complex production incidents and lead resolution processes.
- Provide technical and architectural guidance to Data Engineering and Data Science teams.
- Advance automation, monitoring and observability.
- Lead high availability, capacity planning and disaster recovery initiatives.
- Work with vendors, technology providers, development teams and architecture stakeholders.
Requirements
- At least 3 years of experience managing a technology team.
- At least 5 years of experience building, maintaining and operating data or streaming platforms.
- Significant Linux experience.
- Significant hands-on experience with Confluent Kafka on-premise, including its major components.
- Experience troubleshooting complex production incidents.
- Experience working with development and architecture teams.
- Experience with Azure Synapse, Spark, Databricks, Cloudera or Hadoop.
- Experience with Ansible.
- Experience in MLOps, DataOps or AI platforms.
- Professional leadership, systems thinking and architectural planning skills.
- Availability to provide support during exceptional incidents when required.
Nice to have
- Experience with NiFi.
- Experience in enterprise cloud environments.
- Experience with Trino, Terraform and containers.
- Experience with Confluent Cloud.
- Python or PySpark development experience.
- Experience in a large enterprise organization.