Pyspark & Databricks认证开发工程师
Pyspark & Databricks certified Developer
关于职位
在PradeepIT加速你的职业发展
PradeepIT是全球知名的大型IT咨询公司，将印度经过严格筛选的人才团队连接到全球客户。
我们总部位于班加罗尔，印度的硅谷。PradeepIT的客户包括SAP实验室、博世、劳斯莱斯、大金、戴姆勒和强生，以及数百家财富500强企业和快速发展的初创公司。
通过持续的努力工作并选择远程办公，PradeepIT被认证为值得信赖的工作场所！我们受到全球领先品牌和财富500强公司的信任，已取得以下成果：
6+ 年经验
580+ 开源技术顾问
120+ SAP顾问
40+ Salesforce顾问
60+ Adobe顾问
100+ 移动性顾问
890+ 客户覆盖亚太、欧洲、中东和美国地区
我们的理念
PradeepIT相信连接全球的人，并为他们提供远程工作的机会。作为以员工为先的组织，PradeepIT不断寻找那些不仅能够跟上步伐，还能突破新领域、使用前沿技术并借助PradeepIT学院由垂直领域主管和高级架构师精心设计的课程自由提升技能的个人。
职责
- 使用Azure Databricks从多个来源设计和实现数据摄入管道
- 开发可扩展且可重用的数据集摄入框架
- 整合端到端数据管道——将数据从源系统传输到目标数据存储，确保数据质量与一致性始终如一
- 使用基于事件/流式技术进行数据摄入和处理
- 与其他项目团队成员合作，支持其他项目组件（API接口、搜索）的交付
- 评估多种工具在客户要求下的性能和适用性
- 要求
- 本科学位，至少一年使用Databricks设计、开发、部署和/或支持数据管道的经验
- 在云解决方案（如Azure或AWS）上设计和部署数据应用的专业知识
- 在Databricks环境中进行性能调优和优化代码的实际经验
- 精通Pyspark和Python等编程语言
- 熟悉SQL、T-SQL和/或PL/SQL
- 具有分析和解决问题的能力，特别是在大数据环境中的应用
- 愿意根据需要现场或远程工作
查看英文原文
About the job
Accelerate your career with PradeepIT
PradeepIT is one of the largest, globally recognized IT Consulting firm to connect India's deeply vetted talent team to global customer.
Were headquartered in Bengaluru, Silicon Valley of India. PradeepITs customers include SAP Lab, Bosch, Rolls-Royce, Daikin, Daimler and J&J and hundreds of other Fortune 500 companies and fast-growing startups.
With continuous hard work and working remotely by choice, PradeepIT is certified as a Great Place to Work! Trusted by leading brands and fortune 500 companies from around the world, we have achieved:
6+ Years of Experience
580+ Open source technology Consultant
120+ SAP Consultant
40+ Salesforce Consultant
60+ Adobe Consultant
100+ Mobility Consultant
890+ Clients in APAC, EMEA & USA
Our Beliefs
PradeepIT believes in connecting people across the globe and provide them an opportunity work on remotely. Being a people-first organization, PradeepIT constantly strives for individuals who won't just keep up, but break new ground, work with cutting edge technology and ramp-up their skills with course created by our Vertical Heads, Senior Architect for freely with help of PradeepIT Academy.
Responsibilites
- Designing and implementing data ingestion pipelines from multiple sources using Azure Databricks
- Developing scalable and re-usable frameworks for ingesting of data sets
- Integrating the end to end data pipleline - to take data from source systems to target data repositories ensuring the quality and consistency of data is maintained at all times
- Working with event based / streaming technologies to ingest and process data
- Working with other members of the project team to support delivery of additional project components (API interfaces, Search)
- Evaluating the performance and applicability of multiple tools against customer requirements
Requirements
- Bachelors degree and at least one year of experience designing, developing, deploying and/or supporting data pipelines using Databricks
- Expertise in designing and deploying data applications on cloud solutions, such as Azure or AWS
- Hands on experience in performance tuning and optimizing code running in Databricks environment
- Proficient in programming languages like Pyspark and Python
- Good understanding of SQL, T-SQL and/or PL/SQL
- Demonstrated analytical and problem-solving skills particularly those that apply to a big data environment
- Willingness to work on-site or remote, as needed
Originally posted on Himalayas