Hiring: Data Engineer Company: ATC Location: New York, United States Job Posted Time: 2026-09-16 15:19:06 Target Skills & Keywords : AWS, Airflow, Apache, Azure, BigQuery, CI/CD, CloudFormation, Data Pipeline, Data Warehouse, Databricks, Delta Lake, Docker, ELT, ETL, GCP, Git, Iceberg, Infrastructure as Code, Kafka, MLOps, PySpark, Python, Redshift, SQL, Snowflake, Spark, Terraform About the job Experience: •5+ years of experience in designing, developing, and optimizing scalable data pipelines and cloud-based data platforms. The ideal candidate will have strong expertise in Python, SQL, Databricks, Apache Spark, and ETL/ELT development, along with hands-on experience in at least one cloud platform (AWS, Azure, or Google Cloud Platform). •5+ years of experience as a Data Engineer. Required Skills: •Design, develop, and maintain scalable ETL/ELT data pipelines. •Build and optimize data ingestion, transformation, and integration workflows using Databricks and Apache Spark. •Develop and maintain data lakes and cloud-based data warehouses. •Create scalable batch and real-time data processing solutions. •Optimize data pipelines for performance, reliability, and scalability. •Develop reusable data engineering frameworks and automation solutions. •Partner cross-functionally with data analysts, data scientists, and business stakeholders to deliver high-quality data solutions. •Implement data quality, governance, monitoring, and security best practices. Qualifications: •Bachelor’s or Master’s degree in Computer Science, Information Technology, Engineering, or a related field. •Strong programming skills in Python. •Strong experience with Apache Spark (PySpark preferred). •Applied hands-on capability in at least one cloud platform (AWS, Azure, or Google Cloud Platform). •Knowledge of relational and NoSQL databases. •Operational familiarity with Apache Iceberg or Apache Hudi. Interested candidates, please apply directly through the job posting on company's career page or try via AI auto apply on this platform. Don't miss this opportunity to join a forward-thinking team!