All jobs

Sr. Principal Machine Learning Software Engineer - Inference Serving Engine

Ampere Computing

Portland, OR · RemoteFull-time$225,500 – $338,500 / yearly
Apply now

Job details

Pay

$225,500 – $338,500 / yearly

Job type

Full-time

Work setting

Remote / Hybrid

Location

Portland, OR

Posted

Posted today

Benefits

Pulled from the full job description.

401(k)Dental insuranceVision insurancePaid time off

Full job description

Ampere is a semiconductor design company developing SoCs and platform solutions focused on high-performance, energy-efficient AI compute, part of the SoftBank Group.

Lead the architecture and development of high-performance AI inference-serving software for Ampere accelerators - working across model serving, frameworks, compilers, runtimes, and optimized kernels to enable efficient deployment of modern AI workloads including generative AI and LLMs. Hands-on technical leadership role.

What you'll achieve:

  • Architect and build an inference-serving engine that optimizes state-of-the-art AI models on Ampere deep learning accelerators
  • Drive end-to-end serving performance: throughput, latency, tail latency, memory efficiency, scalability, and cost per inference
  • Develop batching and scheduling, KV-cache management, model loading, memory planning, quantization, and multi-device/distributed inference
  • Optimize model execution via graph compilation, operator fusion, precision optimization, and efficient attention
  • Influence future accelerator architecture through hardware/software co-design
  • Mentor engineers and lead technical decisions across cross-functional initiatives

Hybrid role in Portland, OR. Benefits include premium medical/dental/vision, 401K, unlimited flextime, and 10+ paid holidays.

Qualifications

  • Experience with LLM, transformer, multimodal, or generative-AI inference
  • Experience with PyTorch, oneDNN, SYCL, OpenCL, OpenMP, MPI, CUDA, ROCm, or similar
  • Experience with ML compilers, graph optimization, code generation, kernel development, or device runtimes
  • Familiarity with distributed inference, cloud-native serving, containers, Kubernetes, observability
  • Experience with mixed precision, INT8/INT4/FP8 quantization, sparsity, or efficient-attention methods
  • Contributions to open-source AI, compiler, runtime, or HPC projects

Requirements

  • Bachelor's in CS/CE/Math/EE or related field with 12+ years of relevant experience; master's with 8+ years; or Ph.D. with 5+ years
  • Experience developing or optimizing production inference runtimes, serving engines, deployment toolkits, or high-performance AI systems
  • Experience with vLLM, llama.cpp, TensorRT/TensorRT-LLM, ONNX Runtime, OpenVINO, Triton Inference Server, TVM, XLA, MLIR, or comparable
  • Expert proficiency in modern C++ and strong C experience, including concurrent programming and systems-level performance optimization
  • Strong understanding of AI inference performance: batching, scheduling, latency/throughput tradeoffs, KV-cache behavior, quantization
  • Demonstrated ability to lead complex cross-functional technical initiatives
SemiconductorTech

Ready to apply?

Apply on the company site

You’ll be taken to an external site to complete your application.