Hiring: Senior ML Systems Engineer, Frameworks & Tooling Company: Cohere Location: San Francisco, CA Job Posted Time: 2026-09-11 10:17:39 Employment Type: Full-time / Remote Target Skills & Keywords : CV, Data Pipeline, Docker, Kubernetes, LLM, Node.js, PyTorch, TensorRT, vLLM About the job Required Skills: •Building a high-performance data loading and caching pipeline. •Implementing performance profiling across the ML systems stack •Developing internal metrics and monitoring for training runs. •Building reproducibility and regression testing infrastructure. •Developing a performant fault-tolerant distributed checkpointing system. •Build and own the training framework responsible for large-scale LLM training. •Design distributed training abstractions (data/tensor/pipeline parallelism, FSDP/ZeRO strategies, memory management, checkpointing). •Improve training throughput and stability on multi-node clusters (e.g., GB200/300, AMD, H200/100). Qualifications: •Strong engineering experience in large-scale distributed training or HPC systems. Deep familiarity with JAX internals, distributed training libraries, or custom kernels/fused ops. •Comfort debugging performance issues across CUDA/NCCL, networking, IO, and data pipelines. •A track record of building tools that increase developer velocity for ML teams. •Excellent judgment around trade-offs: performance vs complexity, research velocity vs maintainability. •Strong collaboration skills — you’ll work closely with infra, research, and deployment teams. •Contributions to ML frameworks. •Operational familiarity with evaluation and serving frameworks (vLLM, TensorRT-LLM, custom KV caches). •Background in performance engineering, profiling, or low-level systems. •: paper at top-tier venues (such as NeurIPS, ICML, ICLR, AIStats, MLSys, JMLR, AAAI, Nature, COLING, ACL, EMNLP). Compensation: •Flexible work environment (work from home / hybrid options) Interested candidates, please apply directly through the job posting on company's career page or try via AI auto apply on this platform. Don't miss this opportunity to join a forward-thinking team!