Databricks Engineer-Canada
在 V4C,我们赋予企业组织充分的权力来解锁其数据的全部价值。作为一个快速增长的技术咨询公司，我们的 Databricks 与行业领导者合作，设计现代数据平台，构建高性能数据管道，并加速企业人工智能和分析举措。我们正在寻找一位专注于生产的高级数据工程师，拥有卓越的沟通技能和可靠的数据管道提供经验证实记录。在这个角色中，你将拥有对我们的数据平台的端到端生产所有权，确保了高数据质量和强大的系统性能。理想的候选人优先考虑实际解决问题和实践执行高层次架构。你擅长调试复杂数据问题，并可以通过直接，影响驱动方法明确传达技术决议。关键责任 ·管道开发：使用 SQL,dbt 和 Python 建立，优化和维护强大的数据管道。· 数据质量与测试：在 dbt 内实施严格的数据质量检查和自动化测试框架以主动地发现异常。·生产所有权：监控生产工作流，管理警报，并快速从根到永久修复管道故障。· 数字转换导致：写出高效的 PySpark 转型来处理我们湖人环境中的大规模数据集。·协作与通信：将复杂的数据问题转化为技术队友和业务利益相关者双方明确，简洁的行动计划。需要技能和经验核心技术专业知识 · SQL & dbt:专家级 SQL 技巧和广泛实践经验在 dbt 中构建模块模型，快照和定制 Python 方案测试。· Python:编写清洁可维护脚本的强能力，用于数据操纵，自动化和 API 整合。· PySpark: 适合使用 PySpark 来实现数据转换和基本性能调整 （例如处理连接，DataFrame API）. · Lakehouse 知识：对 Delta Lake 或类似的 Lakehouse 技术的功能理解，包括 ACID 交易和 MERGE 操作。执行&软技能 ·强大的沟通能力：卓越的书面和口头交流技巧。直接及简洁地解释问题和结果的能力。· debugging & Problem Solving:优秀的故障解决技巧，具有方法式方法来识别根源原因和验证纠正。· 生产
查看英文原文
About V4C
At V4C, we empower enterprise organizations to unlock the full value of their data. As a fast-growing technology consultancy, our Databricks practice partners with industry leaders to architect modern data platforms, build high-performance data pipelines, and accelerate enterprise AI and analytics initiatives.
We are seeking a production-focused Senior Data Engineer with exceptional communication skills and a proven track record of delivering reliable data pipelines. In this role, you will have end-to-end production ownership of our data platforms, ensuring high data quality and robust system performance. The ideal candidate prioritizes practical problem-solving and hands-on execution over high-level architecture. You excel at debugging complex data issues and can clearly
communicate technical resolutions using a direct, impact-driven approach.
Key Responsibilities
- Pipeline Development: Build, optimize, and maintain robust data pipelines using SQL, dbt, and Python.
- Data Quality & Testing: Implement rigorous data-quality checks and automated testing frameworks within dbt to proactively catch anomalies.
- Production Ownership: Monitor production workflows, manage alerts, and rapidly debug pipeline failures from root cause to permanent fix.
- Data Transformation: Write efficient PySpark transformations to process large-scale datasets within our Lakehouse environment.
- Collaboration & Communication: Translate complex data issues into clear, concise action plans for both technical teammates and business stakeholders.
Required Skills & Experience
Core Technical Expertise
- SQL & dbt: Expert-level SQL skills and extensive hands-on experience building modular models, snapshots, and custom schema tests in dbt.
- Python: Strong proficiency in writing clean, maintainable Python scripts for data manipulation, automation, and API integrations.
- PySpark: Decent, practical experience utilizing PySpark for data transformation and basic performance tuning (e.g., handling joins, DataFrame API).
- Lakehouse Knowledge: Functional understanding of Delta Lake or similar Lakehouse technologies, including ACID transactions and MERGE operations.
Execution & Soft Skills
- Strong Communication: Exceptional written and spoken communication skills. Ability to explain problems and results directly and concisely.
- Debugging & Problem Solving: Superior troubleshooting skills with a methodical approach to identifying root causes and validating fixes.
- Production Mindset: Experience owning pipelines end-to-end, including CI/CD deployment, monitoring, and handling schema drift.
Originally posted on Himalayas