Senior Engineer – Observability Platform
Posted 3hrs ago
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
Senior Observability Platform Engineer building CVS Health’s large-scale telemetry platform for healthcare applications. Designing OpenTelemetry pipelines and backend services across cloud and on-prem environments.
Responsibilities:
- Develop custom software for the observability platform using Java Spring Boot, Python, and Go
- Design, build, and operate core observability platform services
- Lead enterprise-wide adoption of OpenTelemetry, including client libraries, semantic conventions, instrumentation patterns, and Collector/agent strategy
- Design, architect, and scale high-throughput, fault-tolerant telemetry pipelines for logs, metrics, and traces
- Develop self-service observability capabilities for onboarding, troubleshooting, and adoption
- Implement end-to-end monitoring, SLOs, health checks, and alerting for the observability platform
- Establish reliability standards, error budgets, and incident response practices with SRE, Platform, and Cloud teams
- Participate in on-call rotations and lead incident mitigation, root-cause analysis, and post-incident reviews
- Automate operational workflows through tooling, CI/CD enhancements, and platform automation
- Secure telemetry pipelines using mTLS, secrets management, and zero-trust design patterns
- Produce and maintain technical documentation, standards, and best practices
- Gather requirements, influence roadmap prioritization, and deliver platform improvements
- Provide technical leadership through mentorship, design reviews, architectural guidance, and cross-team collaboration
Requirements:
- 5+ years of experience in Software Engineering
- 3+ years of experience with observability practices, including SLIs/SLOs/SLAs, alerting, and incident management
- 3+ years building production-grade backend services in Go and/or Java
- 3+ years implementing and operating OpenTelemetry, including OTLP, semantic conventions, and instrumentation patterns
- 3+ years with cloud-native and containerized platforms, including Docker, Kubernetes, and Argo CD
- 3+ years working with public cloud platforms, including AWS, GCP, or Azure
- 2+ years designing and scaling distributed, high-volume data pipelines
- 2+ years of experience with Infrastructure as Code tools such as Terraform or CloudFormation
- 2+ years of experience with Helmcharts, Kustomize, or similar tools
- 2+ years working with Grafana OSS or comparable observability backends such as Grafana, Loki, Tempo, or Mimir
- 2+ years with relational databases such as PostgreSQL or MySQL
- Bachelor’s degree from an accredited university or equivalent work experience
- Alternatively, high school diploma plus 4 years of relevant experience
- Preferred experience with service meshes, Envoy, Istio, commercial observability platforms, on-call scheduling tools, streaming/data platforms, time-series/NoSQL/analytical databases, cost optimization, chaos engineering, security-aware platform design, mentoring, 24x7 production systems, regulated environments, and cross-team technical communication
Benefits:
- CVS Health bonus, commission or short-term incentive program in addition to base pay
- Medical coverage
- Dental coverage
- Vision coverage
- Paid time off
- Retirement savings options
- Wellness programs
- Other resources supporting physical, emotional, and financial well-being
- Comprehensive benefits package for colleagues and their families, based on eligibility


















