Senior Data Engineer
Senior Data Engineer needed to design, build, and maintain robust data pipelines and infrastructure supporting ML/AI-powered products. Collaborate with engineering and data science teams to ensure data quality, reliability, and scalability.
Responsibilities
- Design, implement, and maintain scalable and reliable data pipelines and backend systems.
- Ensure data processing is optimized for speed, efficiency, and fault tolerance.
- Monitor and improve uptime, reliability, and observability of data infrastructure.
- Build and maintain systems for data quality, consistency, and usability.
- Work closely with product and engineering teams to deliver new features rapidly.
- Drive innovation in data features, backend services, and AI product integrations.
- Participate in on-call rotations with a service-oriented approach.
- Lead scalability efforts for increasing data volumes and AI/ML initiatives.
Requirements
- At least 5 years of experience with Python in backend or data engineering roles.
- Experience designing and operating large-scale data pipelines and data infrastructure.
- Hands-on experience with ML/AI-powered products in production environments.
- Familiarity with modern LLM and RAG technologies.
- Experience with data processing and streaming technologies (Spark, DBT, Airflow, Kafka, etc.).
- API development skills (FastAPI, micro-services, SFTP).
- Data storage expertise (Data Lakehouse, Apache Iceberg, Vector Databases, RDS).
- Cloud infrastructure experience with AWS (S3, Firehose, Lambda, Athena) and Kubernetes.
- Demonstrated ability to optimize performance, availability, and scalability.
- Strong foundation in data quality, governance, security, and observability.
- Effective collaboration with cross-functional engineering, data science, and product teams.
- Innovative mindset for feature-level implementations and AI/data integrations.
Benefits
- Significant equity
- Hybrid work opportunities
- Mental health days off when needed