Principal DevOps Engineer
NBCUniversal - New York, NY
Hiring: Principal DevOps Engineer Company: NBCUniversal Location: New York, NY Job Posted Time: 2026-09-16 16:11:29 Employment Type: Remote Target Skills & Keywords : AWS, ArgoCD, Bash, CI/CD, Configuration Management, DNS, EC2, EKS, Envoy, Epic Systems, GitHub Actions, GitOps, Grafana, Helm, IAM, JWT, Kubernetes, Node.js, OAuth, OpenTelemetry, PKI, PostgreSQL, PowerShell, Prometheus, Puppet, RBAC, REST, S3, SQLite, Systems Engineering, VPC, Webhooks, k6 About the job Experience: •10+ years of experience designing, building, and operating production infrastructure and cloud-native platforms at enterprise scale. Required Skills: •Drive the technical vision of the platform. You will write production Go services, architect cloud-native solutions, and establish the engineering standards our teams rely on. This role requires deep fluency across Kubernetes internals, cloud infrastructure, Go systems programming, and a passion for mentoring engineers. •Platform Architecture & Vision: Architect a Kubernetes-native platform that models broadcast infrastructure as custom resources. Lead the technical strategy leveraging Crossplane compositions and custom Go functions to automate provisioning across multi-account AWS environments and on-prem control rooms. •Infrastructure & Networking Delivery: Lead the design of cloud networking, DNS strategies, and cross-account connectivity across hybrid environments, automating VPC topology and dynamic network routing. •Technical Leadership: Serve as a technical authority for the team. Write RFCs, drive architectural decisions, mentor engineers, and establish high-confidence CI/CD pipelines, testing strategies, and GitHub Actions automation. •Access & Security Strategy: Own the platform's authorization model, designing hierarchical RBAC systems, resource identifier schemes, and identity integrations that enforce fine-grained access control. •Operational Excellence: Drive GitOps-based continuous delivery (Flux, Kustomize, Helm) and manage configuration-as-code for compute fleets using Puppet. Ensure deep operational visibility by designing comprehensive observability and alerting stacks. •VDI Integration: Oversee the integration of remote desktop/VDI connectivity solutions, focusing on session authentication, credential management, and gateway routing. Qualifications: •Software Engineering: Strong proficiency in Go (systems-level programming, API servers) and deep experience building Kubernetes controllers/operators using patterns like controller-runtime and kubebuilder. •Kubernetes Internals: Expert-level knowledge of the Kubernetes ecosystem, including CRD/XRD generation, operators, informers, admission webhooks, and RBAC. •Crossplane & Control Planes: Deep production experience with Crossplane, including composite resources, composition functions, and specifically developing custom Crossplane providers in Go to integrate external enterprise platforms. •AWS & Networking: Extensive production experience with AWS multi-account architectures, cross-account networking patterns, and identity federation. Requires depth across EKS, EC2, VPC, IAM, STS, SSM, Secrets Manager, Route 53, and S3. •GitOps Delivery: Production experience with GitOps tooling, specifically Flux (HelmRelease, Kustomization) or ArgoCD for continuous delivery on Kubernetes. •Configuration Management: Hands-on experience with Puppet, including module development, PuppetDB, Hiera, and r10k. •APIs & Security: Experience designing REST APIs with middleware patterns and modern authentication (OAuth/JWT). Keen eye for information security, including cross-account IAM trust chains, least-privilege policies, JWT token lifecycles, and secrets abstraction. •Observability: Strong background in designing telemetry platforms using Grafana, Prometheus/Mimir, Loki, OpenTelemetry, and metrics collection agents (Alloy, Prometheus Node Exporter). •Databases: Working knowledge of PostgreSQL, SQLite or similar relational databases, encompassing schema design, migrations, and query optimization. •Communication: Excellent problem-solving skills with a proven ability to present architectural decisions to executives, engage with vendors, and write clear technical documentation. Compensation: •$180,000 - $230,000 / year Interested candidates, please apply directly through the job posting on company's career page or try via AI auto apply on this platform. Don't miss this opportunity to join a forward-thinking team!