Builder Product Manager I
Posted 2hrs ago
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
Product Manager defining AI-agent missions and evaluation frameworks for ECI’s ERP software serving lumberyards, building-material dealers, and hardware stores.
Responsibilities:
- Define missions that product squads execute instead of traditional PRDs
- Guide customer-value-driven enhancements and products with the Industry PM and Value Stream PM
- Translate complex ERP business logic into precise, executable agent missions
- Interview subject matter experts and encode knowledge into system prompts and agentic constraints
- Work with UX Research to define AI-agent interface behavior and generative UI experiences
- Co-build evaluation datasets and rubrics to verify agent output against ERP business logic
- Prototype prompt chains rapidly to validate automation concepts before build cycles
- Analyze usage data, support volume, and codebase complexity to prioritize legacy modernization work
- Define mission specifications, agent goals, tool constraints, success evaluations, and failure conditions
- Collaborate with Builders, QA Builder, UX Research, Industry PM, Value Stream PM, and subject matter experts
Requirements:
- Experience using LLMs to synthesize user research, generate acceptance criteria, and stress-test assumptions
- Ability to specify and pressure-test autonomous agent behavior
- Understanding of LangGraph state machines
- Ability to define measurable acceptance criteria, quantitative evaluations, and ground-truth definitions
- Ability to translate accounting regulations and SME knowledge into system prompts, agentic constraints, and evaluation rubrics
- Ability to define behavioral specifications for AI-agent interfaces, including tone, escalation logic, and failure modes
- Experience co-owning evaluation frameworks and datasets
- Experience using AI for user research and synthesizing support tickets, usage telemetry, and interview transcripts
- Understanding of Python, SQL, LangChain/LangGraph, OpenAI/Anthropic APIs, DSPy, GitHub, and Weights & Biases
- ERP business logic and customer workflow understanding
- Must be able to work remotely in the United States

















