H
Senior Data Engineer
В офисеПолная занятостьData EngineeringCybersecurity
Own the end-to-end data layer for a security platform processing large volumes of logs, scans, and telemetry. Design scalable schemas, pipelines, aggregations, and data-quality systems with high autonomy in a fast-paced startup environment.
Обязанности
- Design and own table structures, schemas, keys, types, relationships, and unified data models.
- Design storage and partitioning strategies for large-scale log data.
- Build and optimize complex aggregations and cross-source joins.
- Build and maintain ETL/ELT pipelines from cloud providers, CDNs/WAFs, and external scanning tools.
- Monitor pipeline health, freshness, quality, and alerting to maintain data reliability and service levels.
- Manage data governance, including access control, retention, and privacy for sensitive customer data.
- Own the scaling strategy for data storage, queries, aggregations, and costs.
- Prepare clean, feature-ready datasets and pipelines for future machine learning use cases.
- Make architectural decisions independently and maintain the data layer end to end.
Требования
- 5+ years of data engineering or strongly data-focused backend experience.
- Expert-level SQL for schema design, data modeling, and complex aggregation and cross-referencing queries.
- Strong Python skills for building ETL/ELT pipelines and data tooling.
- Hands-on experience with high-volume log or event data, partitioning, and performance optimization.
- Experience building and maintaining reliable pipelines from multiple, diverse sources.
- Strong understanding of data integrity, consistency, and reliability across distributed environments.
- Ability to design scalable data and aggregation strategies.
- Equivalent experience with comparable technologies is accepted.