E
AI Workflow Optimization Specialist
·Israel
HybridFull-timeAI EngineeringFinancial Services
You will lead the quality and accuracy of AI-driven fraud and risk review workflows. Responsibilities include evaluating LLM outputs, improving prompt logic through controlled experiments, and interpreting operational metrics to drive iterative enhancements. Join a financial services team in a hybrid Petah Tikva role.
Responsibilities
- Evaluate AI agent decisions to identify gaps and prioritize improvements.
- Refine prompt and rubric files to enhance decision logic and prevent errors.
- Design and run experiments to validate changes before deployment.
- Maintain a suite of ground-truth test cases to ensure reliable AI performance.
- Monitor agreement trends and failure patterns using operational data.
- Collaborate with human reviewers to incorporate feedback and verify fixes.
Requirements
- Strong analytical and written communication skills.
- Experience interpreting LLM reasoning chains and identifying issues.
- Proficiency with AI-based IDEs such as Cursor or Claude Code.
- Hypothesis-driven approach with a focus on measurable outcomes.
- Keen attention to detail, especially in regulatory or risk-sensitive areas.
Nice to have
- Background in fraud operations or financial services risk.
- Prompt engineering or iterative AI improvement experience.
- Familiarity with A/B testing concepts and statistical significance.
- Basic ability to read Python scripts.