数据工程师（网页抓取）
Data Engineer - Web Scraping
这是一个远程职位。替代路径正在寻找熟练的软件开发人员与资产管理公司合作，以帮助客户项目。在这个角色中，你将与各个公司部门的个人合作，为我们的平台塑造和创新新的产品和功能，增强现有的。你将获得大量的独立性和信任，但不会孤立；工程团队领导，产品团队领导以及其他技术团队成员都支持你。这是加入团队首要的权并帮助发展在替代路线内的企业家群体的机会。你将被要求做出贡献，获得所有权，并会希望让你的声音得到听觉。角色总结：使用各种剪辑技术进行 Web 缩，然后利用 Python Panda 图书馆来清理和操纵数据。然后将数据输入数据库/仓库中，并使用 Airflow 或其他工具安排剪辑器。需求角色概述 The Web Scraping Team at Alternative Path 正在寻找一个创造性和细节导向开发人员来为客户项目作出贡献。团队为客户组织内的各个团队制定基本应用程序，数据集和警报，支持他们的日常投资决策。任务是通过提供高质量的专有数据集，及时通知和卓越服务来保持运营优势。我们正在寻找一个自动化，自足，对造的热情和爱于自动化的人才。在这个角色中，你将与分析师合作了解和预测需求。·为各种替代数据集设计，实施和维护 Web 剪辑器。·在废弃数据上进行数据清理，探索，转换等工作。· 与跨功能团队合作理解数据要求并实现有效的数据处理工作流程。·作者 QC 检查验证数据可用性和完整性。·维护警报系统，调查时间敏感的数据事件以确保日常运作顺利。·设计并实施产品和工具来增强 Web 选平台。必须拥有 ·计算机科学或任何相关领域的学士/士学位。· 2-4年软件开发经验 · 强大的 Python 和 SQL/数据库技能 · 使用 Pandas 图书馆 (Python) 的强大专业知识是必需的 · 网络经验
查看英文原文
This is a remote position.
Alternative Path is seeking skilled software developers to collaborate on client projects with an asset management firm. In this role, you will collaborate with individuals across various company departments, to shape and innovate new products and features for our platform, enhancing existing ones. You will have a large degree of independence and trust, but you won't be isolated; the support of the Engineering team leads, the Product team leads, and every other technology team member is behind you. This is an opportunity to join a team-first meritocracy and help grow an entrepreneurial group inside Alternative Path. You will be asked to contribute, given ownership, and will be expected to make your voice heard.
Role SummaryPerforming Web Scraping using various scraping techniques and then utilizing Python’s Pandas library for data cleaning and manipulation. Then ingesting the data into a Database/Warehouse, and scheduling the scrapers using Airflow or other tools.
Requirements
Role OverviewThe Web Scraping Team at Alternative Path is seeking a creative and detail-oriented developer to contribute to client projects. The team develops essential applications, datasets, and alerts for various teams within the client's organization, supporting their daily investment decisions. The mission is to maintain operational excellence by delivering high-quality proprietary datasets, timely notifications, and exceptional service. We are seeking someone who is self-motivated, self-sufficient, with a passion for tinkering and a love for automation.In this role, you will,
· Collaborate with analysts to understand and anticipate requirements.
- Design, implement, and maintain Web scrapers for a wide variety of alternative datasets.
- Perform data cleaning, exploration, transformation, etc., on scraped data.
- Collaborate with cross-functional teams to understand data requirements and implement efficient data processing workflows.
- Author QC checks to validate data availability and integrity.
- Maintain alerting systems and investigate time-sensitive data incidents to ensure smooth day-to-day operations.
- Design and implement products and tools to enhance the Web scraping Platform.Must Have
- Bachelor's/master’s degree in computer science or in any related field.
- 2-4 years of software development experience
- Strong Python and SQL/Database skills
- Strong expertise in using the Pandas library (Python) is a must
- Experience with web technologies (HTML/JS, APIs, etc.)Proven work experience in working with large data sets for Data cleaning, Data transformation, Data manipulation, and Data replacements.
- Excellent verbal and written communication skills
- Aptitude for designing infrastructure, data products, and tools for Data Scientists
Preferred
- Familiarity with scraping and common scraping tools (Selenium, Scrappy, Fiddler, Postman, XPath)
- Experience containerizing workloads with Docker (Kubernetes a plus)
- Experience with build automation (Jenkins, GitLab CI/CD)
- Experience with AWS technologies like S3, RDS, SNS, SQS, Lambda, etc.
Originally posted on Himalayas