高级站点可靠性工程师
Senior Site Reliability Engineer
职位描述
SunCore Digital 正在寻找一位实战型高级站点可靠性工程师,评估我们系统的当前可靠性和可扩展性,识别风险,并实施必要的技术变更以解决这些问题。
这不是一个仅负责监控或提供建议的职位。SRE 将调查现有的应用程序和基础设施,建立可靠性基准,开发可观测性和测试能力,并在 SRE 领域内直接实施可靠性缓解措施。当缓解措施需要特定于应用的代码或业务流程变更时,系统所属团队将在 SRE 的指导下实现并维护这些变更。
由于平台尚未经过真实客户流量的验证,容量和可扩展性将被视为开放风险,直到测试提供相反的证据。
职责
可靠性评估与修复
- 评估网页应用、Flutter 移动服务、API、后端系统、基础设施、数据库、队列和第三方集成的可靠性。
- 识别单点故障、脆弱依赖、手动操作流程和故障模式。
- 区分已确认的问题与疑似风险以及尚未评估的领域。
- 在 SRE 领域内设计并实现共享的可靠性能力和改进措施。
- 定义特定于应用的可靠性变更,并与系统所属团队合作实施;这些团队对其代码和服务保留责任。
- 在解决方案经过测试并加固后,将特定服务的监控、操作手册和持续维护责任转移给相应的系统负责人。
- 跟踪已识别的风险直至实施和验证,而不是止步于建议。
- 性能、负载和容量
- 建立实际可行的性能和容量测试计划。
- 与 QA 和产品相关方合作,识别关键工作流和现实使用场景。
- 建立基准响应时间、吞吐量、并发性及资源消耗。
- 设计并执行负载、压力、耐久性、可扩展性和故障测试。
- 识别涉及应用、数据库、网络、队列、缓存、基础设施和外部服务的瓶颈。
- 实现共享的 SRE 缓解措施,并与系统所属团队协调特定于应用的缓解工作,这些团队对其代码和服务保留责任。
- 在变更后重复测试
查看英文原文
About the Role
SunCore Digital is seeking a hands-on Senior Site Reliability Engineer to assess the current reliability and scalability of our systems, identify risks, and implement the technical changes required to address them.
This is not a monitoring-only or advisory position. The SRE will investigate existing applications and infrastructure, establish reliability baselines, develop observability and testing capabilities, and directly implement reliability mitigations within the SRE domain. When a mitigation requires application-specific code or business-workflow changes, the system-owning team will implement and maintain those changes with SRE guidance.
Because the platform has not yet been validated under real customer traffic, capacity and scalability will be treated as open risks until testing provides evidence otherwise.
Responsibilities
Reliability assessment and remediation
- Assess the reliability of web applications, Flutter mobile services, APIs, backend systems, infrastructure, databases, queues, and third-party integrations.
- Identify single points of failure, fragile dependencies, manual operational processes, and failure modes.
- Distinguish confirmed issues from suspected risks and areas that have not yet been evaluated.
- Design and implement shared reliability capabilities and improvements within the SRE domain.
- Define application-specific reliability changes and work with system-owning teams to implement them; those teams retain responsibility for their code and services.
- Transfer service-specific instrumentation, runbooks, and ongoing maintenance responsibilities to the appropriate system owners after the solution is tested and hardened.
- Track identified risks through implementation and validation rather than stopping at recommendations.
Performance, load, and capacity
- Establish a practical performance and capacity-testing program.
- Work with QA and product stakeholders to identify critical workflows and realistic usage scenarios.
- Establish baseline response times, throughput, concurrency, and resource consumption.
- Design and execute load, stress, endurance, scalability, and failure tests.
- Identify bottlenecks involving applications, databases, networks, queues, caches, infrastructure, and external services.
- Implement shared SRE mitigations and coordinate application-specific mitigation work with system-owning teams, which retain responsibility for their code and services.
- Repeat testing after changes to verify results.
- Document tested capacity, observed constraints, and remaining unknowns.
- Define safe operating limits and early-warning indicators.
Performance and load testing have not yet been completed across the platform. Establishing this capability will be an early priority.
Observability
- Assess current logging, metrics, tracing, health checks, dashboards, and alerting.
- Establish consistent observability standards across services.
- Implement and harden shared health-checking and observability capabilities, transfer shared components to the designated long-term owner, and work with system-owning teams on service-specific instrumentation that they will maintain after handoff.
- Define meaningful service-level indicators and initial reliability objectives with system owners and engineering leadership.
- Build shared reliability dashboards and initial service-specific views, then train system owners to maintain their service-specific dashboards and alerts.
- Ensure alerts are actionable, routed to accountable owners, and tested.
- Identify monitoring blind spots.
- Improve application instrumentation in collaboration with developers.
- Ensure logs and telemetry do not expose sensitive information.
Incident readiness
- Help establish the initial on-call and escalation model.
- Create and test incident-response and troubleshooting runbooks.
- Define severity levels and technical escalation paths.
- Lead or support incident investigation.
- Improve detection, diagnosis, mitigation, and restoration capabilities.
- Facilitate technically focused post-incident reviews.
- Track corrective actions and recurring failure patterns.
- Conduct controlled failure exercises where appropriate.
SRE automation
- Automate repetitive reliability-engineering work, including health checks, diagnostics, alert enrichment, incident triage, capacity checks, and evidence collection.
- Develop safe automated remediation or self-healing for clearly defined and well-tested failure conditions.
- Reduce manual diagnostic, maintenance, incident-response, and reliability-validation steps within the SRE function.
- Define and implement the reliability checks, test logic, and SRE automation that should run through CI/CD. Work with DevOps to integrate them into the shared delivery framework, and with system-owning teams to maintain application-specific configuration after handoff.
- Create reusable reliability tools and patterns, harden and document them, transfer shared components to the designated long-term owner, and train application teams to operate the service-specific portions they own.
- Document automation ownership, safeguards, limitations, rollback behavior, and conditions requiring human intervention.
Collaboration
- Work with the Disaster Recovery and Resilience Engineer on service and data recovery dependencies.
- Work with Security Operations on the security review of SRE implementations before production adoption.
- Provide reliability requirements for deployment safeguards and environment health, and work with DevOps and platform staff on shared integration. SRE does not own deployment automation or routine application deployments.
- Work with QA to define realistic user scenarios and post-mitigation validation.
- Work with application teams on system-specific code changes and instrumentation.
- Provide factual technical findings to engineering leadership without presenting unverified assumptions as confirmed conclusions.
Initial Priorities
- Inventory critical services and their dependencies.
- Assess reliability risks and potential single points of failure.
- Establish system-health and latency visibility.
- Define critical workflows for performance testing.
- Build the initial performance, load, and capacity-testing capability.
- Establish technical baselines.
- Identify and implement the highest-priority reliability and scaling mitigations.
- Create initial runbooks and on-call recommendations.
- Identify SRE processes that should be automated, including diagnostics, alert handling, capacity validation, incident response, and safe remediation.
- Document tested behavior, unresolved risks, and unknowns.
Required Qualifications
- Five or more years of experience in site reliability, production engineering, infrastructure engineering, DevOps, or a comparable role.
- Hands-on experience supporting cloud-hosted applications and distributed systems.
- Experience implementing reliability improvements rather than only producing assessments.
- Experience with performance, load, stress, or capacity testing.
- Experience diagnosing application, database, network, and infrastructure bottlenecks.
- Strong knowledge of monitoring, logging, metrics, alerting, and incident response.
- Experience with reliability automation and scripting.
- Experience working with CI/CD systems.
- Experience troubleshooting APIs, backend services, databases, queues, caches, and external integrations.
- Understanding of timeout, retry, rate-limit, circuit-breaking, and graceful-degradation patterns.
- Ability to work directly in unfamiliar codebases and environments.
- Strong technical documentation and communication skills.
- Ability to distinguish confirmed findings, suspected risks, and unknowns.
- Comfort working in a small organization where operational practices are still being established.
Bonus Points
- Experience preparing a platform for its first external users.
- Experience establishing an SRE capability in a startup or small engineering organization.
- Experience with financial, digital-asset, telemetry, equipment-monitoring, or operational platforms.
- Experience working with Flutter-backed mobile services.
- Experience with chaos or fault-injection testing.
- Experience with data-intensive or event-driven systems.
- Experience in a remote, asynchronous environment.
What Success Looks Like
- Critical services and dependencies are inventoried.
- Reliability risks are visible and prioritized.
- Performance and capacity baselines exist for critical workflows.
- High-priority reliability and scaling mitigations are implemented and tested.
- Critical services have meaningful health checks, dashboards, alerts, and runbooks.
- The company can identify and respond to failures without relying entirely on one person.
- Application teams understand and maintain the reliability improvements made to their systems.
- Remaining capacity and reliability risks are documented clearly.
Compensation
• Negotiable based on location
• Deferred completed Demo
• Role transitions into a permanent hire thereafter
Why Join SunCore Digital
• High-impact role shaping the security foundation of a rapidly scaling digital company
• Fully remote team with async flexibility and optional collaboration in London or Bellevue
• Direct partnership with executive leadership and top-tier engineering, marketing, and design partners
• Competitive compensation with performance upside
Originally posted on Himalayas