数据工程师

Data Engineer (Python, SQL, ETL, Airflow, Snowflake & BigQuery) – Remote
Position Type: Full-Time, Remote
Working Hours: U.S. Business Hours
About the Role
At Pavago, one of our clients is hiring a highly technical Data Engineer to build, maintain, and optimize scalable data pipelines, cloud data infrastructure, and analytics-ready datasets.
You’ll be responsible for the systems that move, transform, validate, and organize data across the business — ensuring analysts, data scientists, engineering teams, and leadership have access to accurate, reliable, and timely data.
This is a hands-on engineering role focused on ETL/ELT development, data warehousing, SQL optimization, orchestration, data quality, and cloud infrastructure.
If you enjoy building robust data systems, solving complex pipeline problems, and designing infrastructure that scales, this role is for you.
What You’ll Own
ETL / ELT Pipeline Development

• Build and maintain scalable ETL/ELT pipelines using Python and SQL.

• Ingest and process data from:

• APIs

• SaaS platforms

• Relational databases

• Cloud applications

• Streaming systems

• Develop reliable extraction, transformation, loading, and validation workflows.

• Build reusable connectors and pipeline components.

• Troubleshoot pipeline failures and data inconsistencies.

Workflow Orchestration & Automation

• Build and manage workflows using Apache Airflow, Prefect, Dagster, Luigi, or similar tools.

• Monitor pipeline health, scheduling, dependencies, and failed jobs.

• Implement automated retries, alerts, and failure handling.

• Improve pipeline reliability and reduce manual intervention.

• Maintain dependable data freshness across critical datasets.

Data Warehousing & Modeling

• Design and optimize cloud data warehouses using:

• Snowflake

• BigQuery

• Redshift

• Build analytics-ready data models and warehouse structures.

• Develop star and snowflake schemas where appropriate.

• Optimize SQL queries and warehouse workloads.

• Improve performance through partitioning, clustering, indexing, and efficient data modeling.

• Monitor and optimize warehouse costs.

Data Quality & Governance

• Implement automated data validation and quality checks.

• Build monitoring for anomalies, missing data, and transformation failures.

• Maintain logging, lineage, and auditability across pipelines.

• Use tools such as dbt and Great Expectations.

• Establish consistent naming conventions and transformation standards.

• Support governance and compliance requirements, including GDPR, HIPAA, or industry-specific standards where applicable.

Streaming & Real-Time Data

• Build and maintain streaming or event-driven data pipelines.

• Work with technologies such as:

• Kafka

• Kinesis

• Pub/Sub

• Support real-time ingestion and low-latency analytics use cases.

• Ensure streaming workflows remain reliable and scalable.

Cloud Infrastructure & DevOps

• Containerize data services using Docker.

• Support Kubernetes-based environments where applicable.

• Build and maintain CI/CD workflows using GitHub Actions, Jenkins, GitLab CI, or similar tools.

• Support infrastructure-as-code using Terraform or CloudFormation.

• Improve deployment reliability, scalability, and automation across the data platform.

Cross-Functional Collaboration

• Partner closely with Data Analysts, Data Scientists, BI teams, Product, and Engineering.

• Deliver curated datasets for:

• Dashboards

• Business intelligence

• Analytics

• Machine learning

• Operational reporting

• Support BI platforms including Tableau, Looker, and Power BI.

• Maintain clear documentation for pipelines, schemas, workflows, and data definitions.

Required Experience & Skills

• 3+ years of professional Data Engineering or backend engineering experience.

• Strong proficiency in:

• Python

• SQL

• Hands-on experience with at least one modern cloud data warehouse:

• Snowflake

• BigQuery

• Redshift

• Experience building and maintaining ETL/ELT pipelines.

• Familiarity with Airflow, Prefect, or similar orchestration platforms.

• Strong understanding of data modeling and warehouse architecture.

• Experience optimizing SQL queries and data workloads.

• Understanding of cloud-based data infrastructure.

• Strong troubleshooting and problem-solving skills.

• Excellent written and verbal English communication.

Nice to Have

• Experience with dbt and Great Expectations.

• Experience with Kafka, Kinesis, or Pub/Sub.

• Familiarity with AWS Glue, GCP Dataflow, or Azure Data Factory.

• Experience with Docker and Kubernetes.

• Infrastructure-as-code experience using Terraform or CloudFormation.

• Experience building CI/CD pipelines.

• Experience with data lineage and governance tools.

• Background in healthcare, fintech, or other regulated environments.

• Experience optimizing large-scale warehouse performance and cloud costs.

• Scala experience.

What Makes You a Great Fit

• You care deeply about data accuracy, reliability, and quality.

• You enjoy debugging complicated pipelines and infrastructure problems.

• You think beyond getting a pipeline to work — you consider scalability, maintainability, observability, and cost.

• You write clean, efficient SQL and Python.

• You proactively identify data quality issues before they reach end users.

• You understand how engineering decisions affect analysts and business teams downstream.

• You can communicate technical concepts clearly across technical and non-technical teams.

What a Typical Day Looks Like
You may:

• Review Airflow or Prefect pipeline health and investigate failures.

• Build a new connector for an API or SaaS platform.

• Develop or improve ETL/ELT workflows.

• Optimize SQL queries and warehouse performance.

• Work with analysts on analytics-ready datasets.

• Improve automated validation and monitoring.

• Troubleshoot data quality or freshness issues.

• Review warehouse usage and identify cost-saving opportunities.

• Document pipelines, schemas, and data definitions.

• Collaborate with engineering teams on infrastructure improvements.

In short: you build and maintain the data infrastructure that powers analytics, reporting, automation, machine learning, and business intelligence.
Key Metrics for Success

• Pipeline uptime of 99%+.

• Data freshness consistently maintained within defined SLAs.

• Zero critical data quality issues reaching production.

• Faster and more efficient SQL and warehouse performance.

• Reduced warehouse and infrastructure costs.

• Reliable and scalable ETL/ELT infrastructure.

• Strong monitoring and rapid resolution of pipeline failures.

• High confidence in data from Analytics, BI, and leadership teams.

Why This Role Stands Out

• Work with modern cloud-native data infrastructure.

• Build scalable pipelines and analytics systems from end to end.

• Gain exposure to data warehousing, orchestration, streaming, and cloud technologies.

• Solve meaningful scalability and reliability challenges.

• Collaborate directly with engineering, analytics, and data science teams.

• High technical ownership in a fully remote environment.

• Strong career growth opportunities into:

• Senior Data Engineer

• Analytics Engineer

• Data Platform Engineer

• Data Architect

Interview Process

• Initial Phone Screen

• Video Interview with Pavago Recruiter

• Technical Assessment — ETL Pipeline or SQL Optimization Exercise

• Client Interview with Engineering/Data Team

• Offer & Onboarding

What Happens After You Apply
Right after you apply, you’ll receive an email invitation from Spark Hire to record your Intro Video. This short, self-recorded video completes your application and can be recorded whenever it’s convenient.
Instead of repeating yourself across multiple screening calls, you’ll introduce yourself once, and your video will be shared with the hiring team. You can record your video as many times as you’d like before submitting it — only your final submission will be reviewed.
Please keep an eye on both your inbox and spam folder for your Spark Hire invitation.
Apply Now
If you’re a Data Engineer who loves building scalable pipelines, optimizing data infrastructure, and ensuring data stays accurate and reliable, we’d love to hear from you.
Apply today and help build the data systems that power better business decisions.
#DataEngineer #Python #SQL #ETL #Airflow #Snowflake #BigQuery #DataEngineering #DataPipelines #CloudData #RemoteJobs #DataWarehouse #RemoteWork
Originally posted on Himalayas