T
Senior Data Engineer
ГибридПолная занятостьData EngineeringFood & Beverage
Architect and own end-to-end web-scale data ingestion pipelines, combining resilient scraping, distributed systems, and AI coding assistants to turn massive unstructured web data into actionable insights. Hybrid role in Tel Aviv.
Обязанности
- Own end-to-end architecture and delivery of web-scale data ingestion pipelines
- Design and maintain scrapers that bypass anti-bot measures and handle dynamic content at scale
- Leverage AI coding assistants to accelerate development and optimize pipeline logic
- Build and manage distributed, cloud-based data pipelines for real-time analytics and AI model deployment
- Proactively identify and resolve data quality issues before they reach production
- Architect scalable API and system designs connecting pipeline decisions to business value
- Monitor, troubleshoot, and continuously improve data workflows
Требования
- 5+ years of experience in Data Engineering or Backend roles focused on high-volume data processing
- Experience with Airflow and Databricks
- Expert-level web scraping skills, including bypassing anti-bot measures
- Extreme ownership of pipelines end-to-end
- High AI-literacy with active use of LLMs and AI coding assistants
- Strong technical depth in distributed systems, cloud architecture, and low-latency data modeling
- Data literacy to connect technical decisions to business value
- Experience with system design thinking and building scalable API architectures