GenAI Data Engineer
泰格分析是一家快速增长的先进分析咨询公司。我们的顾问在数据科学，机器学习和人工智能领域带来了深入的专业知识。我们是几个财富100强企业的可靠分析合作伙伴，使他们能够从数据中产生商业价值。我们的商业价钱和领导力已被包括 Forrester 和 Gartner 在内的各种市场研究公司认可。我们正在寻找顶级人才，因为我们继续建立世界上最好的分析全球咨询团队。我们正在寻求一个有经验的数据工程师，拥有数据 iku 专业知识加入我们的数据团队。作为数据工程师，你将负责设计，构建和维护数据管道，数据集成流程和数据基础设施。你将与数据科学家，分析师和其他利益相关者密切合作，确保有效的数据流动并支持整个组织中基于数据的决策。要求 · 设计和实施强大的数据管道来摄入，处理和存储在 Snowflake 和 GCP 内部规模上的非结构化数据格式。· 利用 Snowflaque 的非结构性数据功能 （导向表，扫描 URL,雪地图）使"黑暗数据"可查询和操作。· 通过使用 BigQuery,Cloud Storage 和 Dataflow 构建和维护云原生 ETL/ELT 流程，确保 GCP 和 Snowflake 之间的无整合。· 而不是仅仅使用 LLM,你将将 AI 工具 （OCR,NLP 实体，文档 AI）集成到工程流中，以转化非结构化的小块成为结构化的见解。·调制复杂的 SQL 查询和基于 Python 的处理工作来有效地处理五字节规模环境。随着公司的增长而存在显著的职业发展机会。该职位提供了一个独特的机会，成为一个小型，具有挑战性的企业环境的一部分，并具有高度的个人责任。首次发布在喜马拉雅山脉
查看英文原文
Tiger Analytics is a fast-growing advanced analytics consulting firm. Our consultants bring deep expertise in Data Science, Machine Learning and AI. We are the trusted analytics partner for several Fortune 100 companies, enabling them to generate business value from data. Our business value and leadership has been recognized by various market research firms, including Forrester and Gartner. We are looking for top-notch talent as we continue to build the best analytics global consulting team in the world.
We are seeking an experienced Data Engineer with expertise in Dataiku to join our data team. As a Data Engineer, you will be responsible for designing, building, and maintaining data pipelines, data integration processes, and data infrastructure. You will collaborate closely with data scientists, analysts, and other stakeholders to ensure efficient data flow and support data-driven decision making across the organization.
Requirements
- Design and implement robust data pipelines that ingest, process, and store unstructured data formats at scale within Snowflake and GCP.
- Leverage Snowflake’s unstructured data capabilities (Directory Tables, Scoped URLs, Snowpark) to make "dark data" queryable and actionable.
- Build and maintain cloud-native ETL/ELT processes using BigQuery, Cloud Storage, and Dataflow, ensuring seamless integration between GCP and Snowflake.
- Instead of just using LLMs, you will integrate AI tools (OCR, NLP entities, Document AI) into the engineering flow to transform unstructured blobs into structured insights.
- Tune complex SQL queries and Python-based processing jobs to handle petabyte-scale environments efficiently.
Benefits
Significant career development opportunities exist as the company grows. The position offers a unique opportunity to be part of a small, challenging, and entrepreneurial environment, with a high degree of individual responsibility.
Originally posted on Himalayas