W
AI Data Quality Analyst
·Israel
HybridFull-timeHR tech
We're seeking an AI Data Quality Analyst to evaluate LLM-generated conversations, perform high-quality data annotation, and identify patterns to improve AI model performance. You'll collaborate with AI Engineers and Product Managers to ensure data quality and shape the future of HR and talent acquisition AI products.
Responsibilities
- Review and evaluate LLM-generated conversations and other text-based outputs against quality criteria and success metrics.
- Perform high-quality annotation and data labeling to create reliable datasets for model training and evaluation.
- Monitor real production data to identify quality trends, regressions, and areas for improvement.
- Identify qualitative patterns, recurring failure modes, regressions, and opportunities to improve model behavior and user experience.
- Analyze evaluation results and communicate insights, risks, and recommendations to cross-functional stakeholders.
- Work closely with AI Engineers and Product Managers to understand data needs and ensure data quality processes align with product goals.
Requirements
- Experience with data labeling or annotation, ideally involving textual data, conversational data, or generative AI.
- Strong qualitative research and analytical skills, with ability to detect meaningful patterns in large volumes of textual data.
- Strong attention to detail, sound judgment, and high bar for consistency and quality.
- Creative, out-of-the-box thinking and proactive approach to investigating ambiguous or unexpected model behavior.
- Technical curiosity and confidence working with web-based tools, labeling platforms, dashboards, and internal interfaces.
- Strong English fluency, both written and verbal.
- Research-oriented background, such as Linguistics, Human-Computer Interaction, Cognitive Science, Psychology, or related field.
- Ability to work from the Tel Aviv office.
Nice to have
- Experience working directly with LLM analysis, model evaluation, conversation quality assessment, or related AI quality workflows.
- Hands-on experience using AI tools such as Claude Code, Cursor, or similar tools.