L
Senior Inference Systems Engineer – KV Cache Optimization
HybridFull-timeSystems Software DevelopmentEnterprise SoftwareData Infrastructure
A technology company developing software-defined storage and data platforms is hiring a Senior Inference Systems Engineer to design, implement, and optimize key-value cache systems for large-scale AI inference. The hybrid role is based in Kfar Saba, Israel, with partial work-from-home flexibility.
Responsibilities
- Design and implement KV cache systems for large-scale inference workloads.
- Analyze system performance, profile inference pipelines, and tune cache strategies.
- Optimize latency, throughput, reliability, caching layers, storage systems, and network I/O.
- Collaborate with software, hardware, and DevOps teams to integrate caching improvements into production.
- Troubleshoot complex system and performance issues.
- Improve observability and monitoring for inference and caching components.
- Contribute to systems architecture for evolving AI and data workloads.
- Document designs and participate in code and design reviews.
- Mentor team members on systems engineering and performance optimization practices.
Requirements
- Strong systems engineering and systems design experience with high-performance inference and KV cache architectures.
- Hands-on experience with Linux, containers, and production distributed systems.
- Advanced troubleshooting skills for performance bottlenecks, latency issues, and reliability problems.
- Experience profiling and optimizing caching layers, storage systems, and network I/O for AI or data platforms.
- Programming experience in C, C++, Rust, or Go, plus scripting with Python or Bash.
- Familiarity with inference frameworks and serving systems such as TensorRT, ONNX Runtime, or Triton.
- Knowledge of distributed systems concepts including consistency, partitioning, and replication.
- Knowledge of Kubernetes, microservices, and observability technologies.
- Bachelor’s or master’s degree in computer science, electrical engineering, or a related field, or equivalent practical experience.
- Clear communication, cross-functional collaboration, and ownership of complex technical challenges.