N
Senior Software Architect, AI Inference
Not specifiedFull-timeSoftware ArchitectureElectronics Manufacturing
Design and improve large-scale LLM inference systems running across GPU clusters. Work across software and hardware domains to shape production infrastructure for generative AI.
Responsibilities
- Design and evolve scalable multi-node LLM inference architectures across GPU clusters
- Build infrastructure to improve production model-serving latency, throughput, and cost efficiency
- Collaborate with model, systems, compiler, and networking teams on high-performance solutions
- Prototype KV cache management, tensor and pipeline parallelism, and dynamic batching
- Evaluate and integrate technologies for load balancing, telemetry, congestion control, and application integration
- Work with internal teams and external partners to turn architecture into reliable, high-performance systems
- Write design documents, technical specifications, and technical blog posts, and contribute to open-source projects when appropriate
Requirements
- Bachelor’s, master’s, or doctoral degree in computer science, electrical engineering, or equivalent experience
- At least 8 years building large-scale distributed systems or performance-critical software
- Strong understanding of deep learning systems, GPU acceleration, AI model execution, or high-performance networking
- Software engineering experience in C++ and/or Python; familiarity with CUDA or similar platforms is preferred
- System-level understanding of memory, networking, scheduling, and compute orchestration
- Strong communication and cross-functional collaboration skills
Nice to have
- Experience with LLM training or inference pipelines, transformer optimization, or model-parallel deployments
- Experience profiling and optimizing LLM system performance
- Knowledge of AI accelerators, distributed communication, congestion control, or load balancing
- Experience optimizing complex systems deployed at scale
- Experience driving complex initiatives from planning through implementation
Benefits
- Competitive salary
- Comprehensive benefits package