P
Senior Big Data Engineer
Не указаноПолная занятостьData EngineeringEnterprise SoftwareData Infrastructure
Design, develop, and maintain scalable data infrastructure and production pipelines that power a location intelligence analytics platform.
Обязанности
- Design, build, and optimize scalable data pipelines for large-volume data processing and integration
- Implement data validation, cleansing, and enrichment to maintain data quality
- Identify bottlenecks and optimize data processing and storage performance
- Design and manage scalable cloud data infrastructure
- Collaborate with analysts, data scientists, product managers, and software engineers
- Implement data governance practices supporting privacy, security, and regulatory compliance
- Monitor pipeline health, troubleshoot issues, and maintain highly available infrastructure
- Provide technical guidance and mentorship to junior data engineers
Требования
- Strong hands-on Apache Spark experience operating production pipelines
- Proficiency in PySpark or Scala
- Demonstrated experience delivering ETL pipelines and large-scale data integration
- Strong SQL skills and knowledge of data modeling
- Production experience with AWS, GCP, or Azure
- Experience with distributed systems and big data formats such as Parquet and Delta Lake
Будет плюсом
- Experience with Apache Airflow or other pipeline orchestration tools
- Exposure to geospatial or location analytics
- Familiarity with Hadoop ecosystem components
- Experience with both Python and Scala beyond Spark development
Условия и преимущества
- Competitive salary
- Excellent benefits
- Work with experienced technical talent
- Opportunity to take a central role in a growing analytics business