N
Senior AI Software Engineer, LLM Inference
לא צויןמשרה מלאהAI EngineeringSemiconductors
Develop and optimize GPU-based inference infrastructure for large language models, improving performance, efficiency, and scalability.
תחומי אחריות
- Design, implement, and optimize GPU inference pipelines for large language models
- Profile and tune execution across schedulers, kernels, and model runtimes
- Develop memory management approaches to improve bandwidth and cache efficiency
- Implement techniques such as speculative decoding, context caching, and FP8/INT4 quantization
- Build and maintain benchmarks and tests for latency, utilization, and efficiency
דרישות
- Bachelor’s, master’s, or higher degree in computer engineering, computer science, applied mathematics, or a related computing field, or equivalent experience
- At least 5 years of relevant software development experience
- Strong Python programming, software design, and software engineering skills
- Experience with deep learning frameworks such as PyTorch and Hugging Face
- Experience profiling and debugging performance across Python, PyTorch, and GPU utilization
- Knowledge of current LLM architectures and inference techniques
- Excellent written and spoken English
יתרון
- Contributions to inference frameworks such as TensorRT-LLM, vLLM, or SGLang
- Expertise in performance modeling, memory optimization, distributed model execution, or GPU workflows
- Experience with NVIDIA profiling tools, including Nsight Systems and PyTorch Profiler
- Strong understanding of inference trade-offs involving compute, memory, scheduling, batching, latency, and throughput
הטבות
- Competitive salary
- Comprehensive benefits package