Overview

We are looking for an experienced ML Engineer to help build, deploy, and optimize machine learning systems in production. In this role, you will focus on model serving, inference performance, and scalable deployment of large language models, working closely with platform and infrastructure teams to deliver reliable and efficient AI solutions.

Responsibilities:
  • Design, deploy, and optimize large language model (LLM) inference systems for production environments
  • Evaluate, benchmark, and improve serving performance across latency, throughput, memory utilization, and cost-efficiency metrics
  • Implement and optimize model-serving solutions using frameworks such as vLLM, TensorRT-LLM, SGLang, Triton Inference Server, or similar technologies
  • Develop and maintain model conversion, deployment, and evaluation workflows across the inference stack
  • Apply and validate quantization techniques to improve performance while preserving model quality
  • Profile GPU workloads, identify bottlenecks, and drive performance improvements using industry-standard observability and profiling tools
  • Collaborate with Platform, Infrastructure, and Product Engineering teams to deploy and operate scalable AI services
  • Contribute to architecture decisions related to distributed inference, model serving, caching strategies, and resource utilization
  • Improve reliability, monitoring, and operational excellence of production inference systems
  • Stay current with emerging advancements in LLM serving, inference optimization, and AI infrastructure technologies, and help evaluate their adoption
Required Qualifications:
  • 3+ years of experience in ML engineering with a strong focus on model deployment, serving, and inference optimization
  • Hands-on production experience with modern LLM serving frameworks such as vLLM, TensorRT-LLM, SGLang, or similar technologies
  • Deep understanding of transformer architectures and LLM inference internals, including attention mechanisms, KV caching, autoregressive generation, prefill vs. decode phases, and performance bottlenecks
  • Experience implementing and validating model quantization techniques (FP8, INT8, AWQ, GPTQ, or similar), with a strong understanding of accuracy, latency, and hardware trade-offs
  • Strong performance engineering and profiling skills, including experience with tools such as Nsight Systems, DCGM, nvidia-smi, or equivalent
  • Advanced Python programming skills for benchmarking, model conversion pipelines, evaluation frameworks, and automation
  • Working knowledge of PyTorch and model export workflows, including ONNX, TorchScript, and TensorRT
  • Proven ability to collaborate effectively with platform, infrastructure, and product teams in production environments
Nice To Have:
  • Understanding of GPU architecture and performance characteristics, including HBM bandwidth, tensor cores, NVLink, and compute throughput
  • Experience working with modern NVIDIA accelerators such as B200, H100, L40S, or similar
  • Hands-on experience with Triton Inference Server
  • Familiarity with Kubernetes and deploying ML workloads in containerized environments
  • Knowledge of distributed inference systems, including tensor parallelism, NCCL tuning, RDMA/InfiniBand, and GPUDirect
  • Experience implementing speculative decoding approaches such as Medusa, EAGLE, or draft-model architectures
  • Operational experience with prefix caching strategies and multi-tenant serving environments
  • Experience serving and managing LoRA adapters, including multi-adapter hot-swapping configurations
Note:

✨ Our intelligent job search engine discovered this job and republished it for your convenience.
Please be aware that the job information may be incorrect or incomplete. The job announcement remains the property of its original publisher. To view the original job and its full details, please visit the job's URL on the owner’s page.

Please clearly mention that you have heard of this job opportunity on https://ijob.am.