自动化工程师 - 全球远程
Automation Engineer - Worlwide Remote
Invisible Technologies 为快速扩展的创新型公司提供独特的服务（外包与自动化的结合）。每家公司对我们的服务都有不同的理解和使用方式。我们亟需能够通过视觉吸引人且高度沟通性的素材，向用户群体传达信息。你将帮助我们以文字无法做到的方式解释我们产品的优势。
职位职责：
使用 Python 模块 [scrapy、selenium、requests、BeautifulSoup 和 splash] 为每个源 URL 创建爬虫
创建和维护 scrapy 管道和中间件，以管理爬虫的输出
为所有类型的网站创建爬虫，无论技术障碍如何
管理爬虫以克服技术挑战，如 IP 封禁、地理位置封禁、验证码和反机器人服务
使用 Python 模块如 sqlalchemy 编写 SQL 查询来管理数据库操作
将 Python 脚本/爬虫部署到基于 Linux 的 AWS 服务器
技能要求 必须具备：
在 Python 编程方面有扎实的实际经验
熟悉 Requests、BeautifulSoup、Selenium 和 Scrapy 等爬虫库
有 3 年以上在 Web 开发框架（如 Flask/Django/FastAPI/Tornado/Pandas）上的实际经验
了解使用 REST 构建 API 和服务
熟悉任何 RDBMS 并具备强大的 SQL 知识
清楚理解面向对象的概念
出色的故障排除能力
熟练掌握代码版本控制工具如 git
基本了解前端技术，如 JavaScript、HTML5 和 CSS3
了解可扩展应用程序的基本设计原则
加分项：
熟悉 UI 框架如 Angular/ReactJS
较强的单元测试和调试能力
对数据工程平台如 Airflow 有深入了解
有 Google Cloud、Docker 使用经验
熟悉 Agile 和 Scrum。如果曾在 Jira 和 confluence 中工作过更佳
有 ETL 经验
优秀的沟通能力和与多元化团队合作的能力
所需技术技能：
· Python
· JavaScript
· PostgresSQL
· 使用 Requests、BeautifulSoup、Selenium 和 Scrapy 进行网页爬取
· 框架如 Flask/Django/FastAPI/Tornado/Pandas
· 使用 REST 构建 API 和服务
https://jobs.lever.co/invisible/e3f21175-7605-4b30-8790-5ddbd97ab3b5
查看英文原文
Invisible Technologies offers unusual services (a combination of outsourcing and automation) to fast-scaling, innovative companies. Each company understands and uses our services in a variety of ways. It’s critical we can communicate to segments of users with visually appealing, and highly communicative assets. You’re helping us explain the benefits of our product in a way that words alone never could.
Job Responsibilities:
Create crawlers for each source URL by using Python modules [scrapy, selenium, requests, BeautifulSoup and splash]
Create and maintain scrapy pipelines and middlewares to manage the output from the crawlers.
Create crawlers for all types of websites irrespective of the technical roadblocks.
Manage the crawlers to overcome technical challenges like IP ban, geolocation ban, captcha and bot blocking services.
Write SQL queries to manage database operations using Python modules like sqlalchemyDeploy the python scripts / crawlers to Linux based AWS servers
SkillsMust Haves:
Strong hands-on experience in Python programming
Good experience with scraping libraries such as Requests, BeautifulSoup, Selenium and Scrapy.
Proven 3+ experience working on web development frameworks such as Flask/Django/FastAPI/Tornado/PandasKnowledge of building APIs and services using REST.
Experience with any RDBMS and strong SQL knowledge.
Clear with Object-oriented concepts.
Excellent troubleshooting skills.
Proficient understanding of code versioning tools like git.
Basic understanding of front-end technologies, such as JavaScript, HTML5 and CSS3Understanding of fundamental design principles behind a scalable application.
Nice to have:
Familiar with UI frameworks like Angular/ReactJS
Strong unit test and debugging skills.
Strong knowledge of data engineering platforms such as Airflow.
Experience with the Google CloudDockerAgile and Scrum. Even better if have worked in Jira and confluence.
Experience with ETLCeleryExcellent interpersonal skills and ability to to work with a diverse team
Required: Technical Skills
· Python
· JavaScript
· PostgresSQL
· Web Scraping using Requests, BeautifulSoup, Selenium and Scrapy
· Frameworks such as Flask/Django/FastAPI/Tornado/Pandas
· Building APIs and services using REST
https://jobs.lever.co/invisible/e3f21175-7605-4b30-8790-5ddbd97ab3b5