Hiring: Senior Data Engineer Company: Kai Location: San Jose, CA Job Posted Time: 2026-09-17 11:08:12 Target Skills & Keywords : AWS, Airflow, Azure, CosmosDB, Data Pipeline, Docker, Feature Store, Flink, GCP, Iceberg, Kafka, Kubernetes, MongoDB, Python, SQL, Spark, Terraform About the job Experience: •20 years of experience in the cybersecurity industry. Their proven expertise and vision drive our ambitious goals. •7+ years of experience in data engineering or data platform engineering •Over 20 years of experience in the cybersecurity industry. Their proven expertise and vision drive our ambitious goals. Required Skills: •Design and build scalable data pipelines for batch and real-time processing across Kai's agentic AI platform •Own and optimize high-volume data infrastructure handling hundreds of millions of entries with low latency and high reliability •Build and maintain data models and storage systems optimized for large-scale, high-throughput security data workloads •Identify bottlenecks in the current architecture and drive optimization — reduce processing time, improve reliability, and make the customer experience better •Lead the Terraformization of data pipelines to enable cloud-agnostic deployment across Azure, AWS, and GCP •Integrate and manage cloud data services, ensuring secure service principles, permissions, and cross-service connectivity •Collaborate closely with Backend Engineering teams on both the ingestion and consumption sides of the data pipeline •Ensure data quality, consistency, and reliability across all pipelines Qualifications: •Strong proficiency in Python and SQL — these are how our systems are written •Strong data modeling skills — you can design schemas and storage systems that hold up at scale •Proven experience designing and building large-scale distributed data pipelines in both batch and streaming modes •Applied hands-on capability in Flink, Kafka, Spark, or similar stream and batch processing frameworks •Infrastructure experience — Terraform, Kubernetes, and Docker are expected, not aspirational •Cloud platform expertise — deep hands-on experience in at least one major cloud platform (Azure, AWS, or GCP); Azure experience strongly preferred •Strong communication skills — you work cross-functionally and can explain complex systems clearly •DataOps experience — ability to own data infrastructure decisions independently, reducing dependency on DevOps for pipeline deployment, permissions, and service integration •Startup or high-growth experience — you have operated in a fast-paced environment where things change quickly, and ownership is expected Compensation: •Generous compensation: We offer highly competitive salaries, equity options, and a supportive work environment Interested candidates, please apply directly through the job posting on company's career page or try via AI auto apply on this platform. Don't miss this opportunity to join a forward-thinking team!