P
Principal Data Engineer
В офисеПолная занятостьData EngineeringCybersecurity
Lead the design, implementation, and maintenance of cloud data pipelines and data platforms supporting cybersecurity attack detection and security research.
Обязанности
- Lead the design and implementation of complex end-to-end cloud data pipelines and data platforms.
- Collect, analyze, and translate data requirements from analysts, developers, researchers, and managers.
- Build scalable cloud data solutions with Python and distributed processing frameworks.
- Model and manage data in cloud MPP databases and data lakes.
- Collaborate with security and AI researchers to support cybersecurity attack detection capabilities.
- Use AI throughout the development cycle to generate, review, and test code.
Требования
- At least 4 years of experience as a Senior Data Engineer, Data Engineering Lead, or Data Architect in cloud environments.
- At least 4 years of experience with data modeling across data platforms and databases.
- At least 4 years of Python experience focused on data engineering and distributed processing.
- At least 4 years of experience with cloud MPP databases or data lakes, such as BigQuery, Redshift, Snowflake, Azure SQL Data Warehouse, or Azure Kusto.
- Experience designing and implementing complex end-to-end data pipelines and platforms.
- Experience with Apache Spark, Apache Beam/Dataflow, GKE, or comparable distributed processing technologies.
- Experience with Docker and Kubernetes or GKE.
- Fluency in SQL.
- Experience using AI prompts to generate, review, and test code, including writing MCPs, skills, and markdown files.
Будет плюсом
- GCP experience
- A data science background
- Cybersecurity knowledge