I
Mid-Level Data Engineer
במקום העבודהמשרה מלאהData EngineeringCloud Computing Services
Join a cloud services team in Ra'anana to design, develop, and maintain AWS- and Databricks-based data platforms. The role combines data engineering, database administration, lakehouse architecture, and analytics engineering.
תחומי אחריות
- Develop and maintain ETL/ELT pipelines with Python and PySpark on Databricks and transform data in Snowflake.
- Implement Bronze, Silver, and Gold data layers using Medallion architecture and Delta Lake.
- Configure Databricks SQL Warehouses, optimize queries, and support data exploration.
- Use AWS services to manage storage, compute resources, and secure data movement.
- Manage indexing, clustering, vacuuming, and performance tuning across Delta Lake and relational or warehouse environments.
- Design Star Schema and Snowflake models in the Gold layer for Power BI, Tableau, and other BI tools.
- Manage metadata, permissions, and data lineage with Unity Catalog and platform access controls.
- Implement automated data-quality tests within CI/CD data pipelines.
דרישות
- At least 5 years of professional experience as a Data Engineer or Data Engineer/DBA.
- Hands-on experience with AWS services such as S3, IAM, and EC2.
- Strong experience with Snowflake and modern cloud data warehouses.
- At least 1 year of intensive hands-on Databricks experience, including Notebooks and Workflows.
- High proficiency in PySpark or Spark Scala.
- Advanced SQL skills, including window functions and CTEs.
- Experience with query profiling, indexing, partition pruning, and database performance tuning.
- Practical knowledge of Delta Lake, Parquet, and cloud storage such as AWS S3 or Azure ADLS.
- Experience designing fact and dimension tables.
- Ability to troubleshoot distributed processes and database bottlenecks.
- Ability to work independently, lead tasks end-to-end, and collaborate with analysts, product managers, and data scientists.
יתרון
- Experience with Delta Live Tables or dbt on Databricks or Snowflake.
- Familiarity with MLOps concepts and MLflow.
- AWS, Snowflake, or Databricks certification.
- Experience with Apache Airflow or other external orchestration tools.