Senior Data Engineer
A technology team is seeking a Senior Data Engineer to develop scalable data solutions in an AWS environment. The full-time role is hybrid, with two remote days per week, in Central Israel.
Responsibilities
- Develop and maintain complex ETL/ELT pipelines using AWS Glue, Python, PySpark, and Spark.
- Process diverse data sources and structured or semi-structured formats, including CSV, JSON, and text.
- Work with relational and non-relational databases and optimize complex SQL queries.
- Design data solutions across data warehouse and data lake environments.
- Integrate data from APIs and web sources.
- Address performance and scalability challenges.
- Collaborate with systems analysts, infrastructure teams, and BI developers.
Requirements
- At least 4 years of data engineering experience.
- Experience developing complex ETL/ELT processes.
- Hands-on experience with AWS Glue, Python, PySpark, and Spark.
- Strong SQL skills, including writing and optimizing complex queries.
- Experience with relational and non-relational databases and structured and semi-structured data.
- Experience integrating APIs and web sources.
- Knowledge of data warehouses, data modeling, and data lake architecture.
- Ability to address performance and scalability challenges.