J
Senior QA Automation Engineer
RemoteFull-timeQA & Test AutomationOnline Marketplaces
Build AI-powered testing agents and dependable test infrastructure for a fully remote, global engineering organization. Help product teams adopt reusable quality engineering tools and practices.
Responsibilities
- Build and operate AI agents for test design, generation, maintenance, execution, and failure triage.
- Own infrastructure for dependable agent-driven testing, including environments, test data, tool interfaces, guardrails, runtime management, and cost controls.
- Create reliable methods for evaluating agent output so teams can trust automated results.
- Apply AI-driven evaluation methods to non-deterministic and AI-powered product capabilities.
- Partner with product engineering teams to keep testing tools relevant to product needs.
- Develop reusable testing patterns, tools, and enablement practices so engineers can own quality.
- Drive adoption of quality engineering capabilities across teams.
- Contribute to technical decisions, system design, and continuous improvement of testing infrastructure and engineering practices.
Requirements
- At least 5 years of professional software engineering experience shipping production software.
- Experience building and owning test infrastructure, developer tooling, internal platforms, or similar systems used by engineers.
- Practical experience with LLMs or agent frameworks, including prompt and context design, tool use, non-deterministic behavior, and evaluation of generated outputs.
- Strong understanding of distributed systems and continuous integration at scale, including concurrency, state, timing, and reliability.
- Demonstrated ability to influence teams and drive adoption of tools or practices without direct management authority.
- Ability and willingness to understand the product deeply enough to build effective quality solutions.
- Sound engineering judgment balancing quality, maintainability, delivery speed, and business needs.
- Strong collaboration and communication skills in distributed, cross-functional environments.
Nice to have
- Experience with LLM evaluation or benchmarking, open-source development, developer tooling, or platform engineering.
- Experience with Java, TypeScript, Python, Kubernetes, or Helm.
Benefits
- Fully remote, flexible work environment, subject to applicable employment arrangements.
- Home office budget and co-working space support.
- Flexible time off.
- Healthcare benefits and global mental wellbeing support, tailored locally.
- Lifestyle spending account supporting wellbeing and personal needs.
- Retirement and pension plans, with company contributions where applicable, plus life and disability insurance where available.
- Up to $1,000 annually for professional development.
- Company gatherings, team offsites, local meetups, and budgets for in-person connection.
- Equity through a Virtual Stock Option Plan where applicable.