网络爬虫负责人 - 全球团队（印度）
Web Scraping Lead - Global Team (India)
关于公司：
YipitData 是颠覆性经济领域领先的市场研究和分析公司，最近从卡尔沃特集团（The Carlyle Group）融资高达 4.75 亿美元，估值超过 10 亿美元。
我们每天分析数十亿个替代数据点，为拼车、电子商务市场、支付等领域提供准确、详细的洞察。我们的按需洞察团队使用专有技术来识别、授权、清洗和分析许多世界最大投资基金和企业所依赖的数据。
连续三年，我们被列为 Inc 最佳工作场所之一。我们是一家快速增长的技术公司，由卡尔沃特集团和诺威斯风险投资公司（Norwest Venture Partners）支持。我们的办公室位于纽约、奥斯汀、迈阿密、丹佛、山景城、西雅图、香港、上海、北京、广州和新加坡。我们培养以员工为中心的文化，注重精通、所有权和透明度。
职位描述：
我们正在寻找一位经验丰富的网络爬虫经理，领导我们的网络爬虫工程团队，并负责我们爬虫基础设施的设计、可扩展性、可靠性和性能。您将领导工程师，推动技术架构和最佳实践，解决复杂的爬虫挑战，并与工程、数据、分析和产品团队合作，交付可靠、高质量的数据。
作为我们的网络爬虫经理，您将：
- 领导、指导并发展网络爬虫工程师团队。
- 负责爬虫系统的架构、路线图、可扩展性和可靠性。
- 指导使用 Playwright、Puppeteer、Selenium 等工具开发和维护爬虫。
- 解决涉及动态网站、浏览器自动化、指纹识别、会话、Cookie、请求头和代理的复杂挑战。
- 建立监控、日志记录、警报和数据质量实践。
- 推动性能、可靠性和基础设施改进。
- 与跨职能团队合作，优先处理爬虫需求并交付高质量数据。
- 参与招聘、技术评审和工程规划。
这是一个基于印度的全远程职位。标准工作时间为 IST 时间上午 11 点至晚上 8 点，但这里有一定的灵活性。
如果您具备以下条件，您很可能成功：
- 7 年以上软件工程经验，具有强大的网络爬虫经验。
- 具有工程管理/团队领导经验者优先。
- 在 Playwright、Puppeteer 或 Selenium 方面有扎实的实践经验。
- 对 HTTP、REST 有深入理解。
查看英文原文
About Us:
YipitData is the leading market research and analytics firm for the disruptive economy and recently raised up to $475M from The Carlyle Group at a valuation over $1B.
We analyze billions of alternative data points every day to provide accurate, detailed insights on ridesharing, e-commerce marketplaces, payments and more. Our on-demand insights team uses proprietary technology to identify, license, clean and analyze the data many of the world’s largest investment funds and corporations depend on.
For three years and counting, we have been recognized as one of Inc’s Best Workplaces. We are a fast-growing technology company backed by The Carlyle Group and Norwest Venture Partners. Our offices are located in NYC, Austin, Miami, Denver, Mountain View, Seattle, Hong Kong, Shanghai, Beijing, Guangzhou, and Singapore. We cultivate a people-centric culture focused on mastery, ownership, and transparency.
About The Role:
We are looking for an experienced Web Scraping Manager to lead our web scraping engineering team and own the design, scalability, reliability, and performance of our scraping infrastructure. You’ll lead engineers, drive technical architecture and best practices, solve complex scraping challenges, and collaborate with Engineering, Data, Analytics, and Product teams to deliver reliable, high-quality data.
As Our Web Scraping Manager You Will:
- Lead, mentor, and grow a team of web scraping engineers.
- Own the architecture, roadmap, scalability, and reliability of scraping systems.
- Guide the development and maintenance of scrapers using Playwright, Puppeteer, Selenium, etc.
- Solve complex challenges involving dynamic websites, browser automation, fingerprinting, sessions, cookies, headers, and proxies.
- Establish monitoring, logging, alerting, and data-quality practices.
- Drive performance, reliability, and infrastructure improvements.
- Partner with cross-functional teams to prioritize scraping requirements and deliver high-quality data.
- Participate in hiring, technical reviews, and engineering planning.
This is a fully-remote opportunity based in India. Standard work hours are from 11am to 8pm IST, but there is flexibility here.
You Are Likely To Succeed If:
- 7+ years of software engineering experience, with strong web scraping experience.
- Previous engineering management/team leadership experience preferred.
- Strong hands-on experience with Playwright, Puppeteer, or Selenium.
- Deep understanding of HTTP, REST APIs, HTML/DOM, browser rendering, and TLS/SSL.
- Strong experience with fingerprinting, anti-bot challenges, cookies, headers, sessions, and proxy rotation.
- Experience building scalable, reliable scraping infrastructure.
- Strong knowledge of monitoring, logging, metrics, and production troubleshooting.
- Excellent communication and stakeholder-management skills.
- Strong ownership, leadership, and mentoring abilities.
What We Offer:
Our compensation package includes comprehensive benefits, perks, and a competitive salary:
- We care about your personal life, and we mean it. We offer flexible work hours, flexible vacation, parental leave, team events, wellness budget, learning reimbursement, and more!
- Your growth at YipitData is determined by the impact that you are making, not by tenure, unnecessary facetime, or office politics. Everyone at YipitData is empowered to learn, self-improve, and master their skills in an environment focused on ownership, respect, and trust. See more on our high-impact, high-opportunity work environment above!
- The final offer may be determined by a number of factors, including, but not limited to, the applicant's experience, knowledge, skills, abilities, as well as internal team benchmarks.
We are committed to equal employment opportunity regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, marital status, disability, gender, gender identity or expression, or veteran status. We are proud to be an equal-opportunity employer.
Job Applicant Privacy Notice
<img height="1" width="1" style="display:none;" alt="" src="https://px.ads.linkedin.com/collect/?pid=4341228&conversionId=10486642&fmt=gif" />