F
AI Evaluation Team Lead
לא צויןמשרה מלאהAI & Machine LearningArtificial IntelligenceEnterprise Software
Lead the team responsible for how production tabular machine learning systems are evaluated. Build shared benchmarks, curated datasets, and reproducible tooling used by research, engineering, and customer-facing teams.
תחומי אחריות
- Build a shared evaluation platform for engineering, research, and Applied AI teams
- Maintain models and curated datasets so teams can run benchmarks and investigate results independently
- Define standards for metrics, data splits, leakage prevention, calibration, uncertainty, and benchmark contamination
- Build reproducible pipelines with versioned inputs and artifacts, and integrate regression checks into research and release workflows
- Benchmark the model against competing approaches using fair tuning budgets, data access, compute, and latency protocols
- Support customer proof-of-concept evaluations with tooling, methodological guidance, and analysis
- Measure predictive quality, latency, and cost across deployment configurations, tasks, and dataset characteristics
- Translate findings into research priorities, release recommendations, and evidence-backed customer improvement plans
- Hire and develop a small team, set priorities, and remain hands-on with code and experimental design
דרישות
- Experience owning evaluation for production tabular machine learning systems or systems used in consequential customer decisions
- Strong statistical judgment, including metric and validation-scheme selection, uncertainty estimation, cross-dataset model comparison, and accounting for repeated experiments
- Experience identifying data leakage in preprocessing, feature construction, joins, temporal dependencies, and related entities across data splits
- Strong Python and SQL skills; familiarity with scikit-learn and gradient-boosted trees
- Experience building reliable machine learning tools or platforms for other teams
- Experience designing fair model comparisons, including hyperparameter search, resource budgets, and end-to-end latency measurement
- Prior people management experience, including hiring, technical coaching, and performance feedback, while remaining technically involved
- Clear written and spoken communication with researchers, engineers, and customer data scientists
יתרון
- Experience evaluating tabular foundation models or AutoML systems
- Experience measuring the effects of model optimizations on predictive quality and inference performance
- Experience with relational or multi-table data and Snowflake or Databricks
- Experience evaluating automated or agent-driven machine learning workflows
הטבות
- Competitive compensation with salary and equity
- Health coverage for employees and dependents
- Paid parental leave, including for adoptive and surrogate journeys
- Relocation support for employees moving to an office location