Senior Software Engineer II – LLM Observability
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
Senior engineer building LLM and agent observability for Honeycomb’s developer tools platform. Shipping full-stack AI monitoring features from data ingestion through customer-facing visualizations.
Responsibilities:
- Own end-to-end development of AI observability capabilities, from ingestion to in-product visualizations
- Architect and evolve the data model and infrastructure for observing agents and their ecosystems across the Honeycomb product
- Build systems that analyze and surface AI-specific signals, including cost, token usage, errors, and anomalies
- Collaborate with product and design partners to create usable, intuitive features and meaningful user workflows
- Teach best practices for exploring and understanding AI system behavior across conversations and workloads
- Iterate through fast feedback loops and experimentation, sharing wins and failures
- Work in a small, cross-functional engineering group with a dedicated product manager and designer
- Ship features customers use in production
- Participate in on-call rotation after about 90 days
Requirements:
- Significant software engineering experience, with a record of shipping and owning features end to end
- Experience shipping customer-facing features at a SaaS company, beyond internal tooling
- Full-stack proficiency in backend development with Go and/or Python and frontend development with TypeScript and React
- Track record of working closely with product managers and designers
- Regular, hands-on use of AI tools in day-to-day engineering work
- Experience being on call; joining the rotation after about 90 days
- Track record of navigating ambiguity in a fast-moving field where requirements shift and experimentation is core
- Bonus: Experience building LLM or agent-powered features in production
- Bonus: Hands-on experience with observability tooling, such as OpenTelemetry
- Must verify identity and eligibility to work
- Honeycomb cannot currently sponsor or support visa transfers
Benefits:
- Generous equity with employee-friendly stock program
- Transparent pay based on levels relative to experience
- Unlimited PTO
- Distributed-first mindset and culture
- Home office, co-working, and internet stipend
- Full benefits coverage for employees, with additional coverage available for dependents
- Up to 16 weeks of paid parental leave, regardless of path to parenthood
- Annual development allowance
- Reasonable accommodations during the hiring process
Similar Jobs
Senior Frontend Developer – Tech Lead
Technical Lead building React and Next.js frontends for complex interactive graphics. Owning architecture, performance, integrations and mentoring engineers.
Senior Frontend Developer – Tech Lead
Technical Lead building React and Next.js interfaces with interactive graphics. Owning frontend architecture, performance, API integration, and team mentoring.
Senior Frontend Developer – Tech Lead
Frontend tech lead owning React and Next.js architecture for complex interactive graphics. Guiding performance, API integration, testing and technical mentoring.
Senior Software Engineer – Função System
Engenheiro de Software Sênior supervisionando sistemas bancários e soluções de crédito no PagBank. Avaliando projetos, APIs, homologações e melhorias de processos.
Senior Software Engineer II – LLM Observability
Senior engineer building AI observability features for Honeycomb’s developer tools platform. Owning data infrastructure, LLM signals, and customer-facing visualizations.
Software Engineer I – LLM Observability
Software Engineer building AI observability features for Honeycomb’s developer tools. Developing data infrastructure and product experiences to help customers debug LLM agents.
Software Engineer I – LLM Observability
Software Engineer building AI observability for Honeycomb’s developer tools platform. Developing systems that surface agent costs, token usage, errors, anomalies, and conversations.
Senior Software Engineer
Engenheiro(a) de Software Sênior desenvolvendo aplicações backend, APIs e microsserviços. Projetando sistemas escaláveis e evoluindo pipelines de CI/CD em São Paulo.
Software Engineering Specialist
Especialista em engenharia de software evoluindo sistemas críticos do mercado de capitais na Depositária. Criando agentes inteligentes, automações e soluções de IA para clientes.
Senior Software Engineer
Senior Software Engineer improving LanceDB’s distributed data infrastructure and open-source Lance format. Building high-performance integrations across big data and AI ecosystems.
Senior Software Engineer
Engenheiro de Software Sênior na Flash, ecossistema de soluções inteligentes para RH e financeiro. Projetando sistemas escaláveis e acelerando entregas com agentes de código de IA.
Software Engineering Internship Program, 2027
Software engineering intern contributing to Wikimedia’s open-source tools for Wikipedia and free-knowledge projects. Learning through mentored project work across apps, data, infrastructure, and editing teams.
Senior Android Software Engineer, Consumer Engineering
Senior Android engineer building scalable Kotlin experiences for Reddit’s community platform. Leading consumer product development, experimentation, architecture, and cross-functional initiatives.
Senior Software Engineer
Senior Software Engineer building DigiCert’s certificate management, TLS, vulnerability monitoring, and security platform. Owning full-stack TypeScript services, infrastructure, and production operations.
Staff Software Engineer – SRE
Staff SRE scaling AWS Kubernetes, databases, and crypto nodes for Phantom’s global self-custody wallet. Improving reliability, observability, and infrastructure automation across blockchain systems.
Staff Software Engineer
Staff Software Engineer architecting Go and Kubernetes systems for vCluster Labs’ AI infrastructure platform. Building open-source tenant-isolation technology for global AI cloud providers.
Senior Software Engineer
Senior Software Engineer leading scalable cloud software development at Hudl, which helps sports teams capture video, analyze data, and share highlights. Championing agentic AI and mentoring engineers.
Software Development Engineer – Azure CLI, Azure PowerShell
Software Development Engineer building Azure CLI and Azure PowerShell developer tools for Centific’s cloud infrastructure and automation projects. Developing Python or C# features with global engineering teams.
Principal Engineer – Consumer & Merchant Experience
Principal Engineer shaping architecture for InPost’s customer-facing e-commerce and parcel-delivery products. Leading cross-domain technical initiatives across international markets.
AI Solutions Engineer, Full-stack Developer
AI solutions engineer and full-stack developer building telephony, CRM, billing, and scheduling systems. Automating operations for a storage and logistics company serving Australian businesses.