Data Operations Lead
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
Strategic Data Operations Lead for TRACTIAN managing data annotation programs and ensuring quality at scale. Oversee workflows, vendor relations, and maintain operational data integrity.
Responsibilities:
- End-to-End Ownership: Define program objectives, timelines, and deliverables for multiple data labeling projects simultaneously.
- Workflow Design: Create scalable SOPs (Standard Operating Procedures) and guidelines. You will decide when to use a "Consensus" model (multiple labelers per item) versus a "Single-Pass" model based on cost/quality trade-offs.
- Risk Mitigation: Proactively identify bottlenecks (e.g., ambiguity in guidelines, tool downtime) and implement "Risk Mitigation Strategies" before they impact model training schedules.
- Crowd/Team Oversight: Recruit, train, and manage a distributed team of annotators. Monitor "Throughput" (items/hour) and "Efficiency" to ensure productivity targets are met.
- Vendor Relations: Act as the primary interface for external data vendors. Negotiate timelines, track budget utilization, and hold vendors accountable to accuracy SLAs (e.g., 98% quality on Gold Sets).
- Performance Coaching: Implement data-driven feedback loops. If an annotator's quality drops, you will analyze their errors and provide targeted retraining materials.
- Gold Set Management: Maintain a "Gold Set" (master answer key) to blindly test annotators.
- Metric Analysis: Track and report on key quality metrics: Inter-Annotator Agreement (IAA), Accuracy, and Precision/Recall of the human labels.
- Root Cause Analysis: When model performance dips, you will investigate the training data to determine if the issue stems from "Labeler Bias," "Guideline Drift," or "Edge Case Ambiguity."
- Platform Operations: Help set up the UI/UX and configurations for the internal platforms used for labeling and annotation.
- Reporting: Generate weekly executive dashboards using Excel/Google Sheets (Pivot Tables, VLOOKUP) to visualize "Spend vs. Output" and "Quality Trends" for stakeholders.
Requirements:
- 3+ years in Data Operations, Program Management, or QA for Machine Learning/AI.
- Familiarity with the AI lifecycle (Training vs. Validation vs. Test sets).
- Experience writing technical documentation/guidelines that leave no room for interpretation.
- Advanced proficiency in Excel/Google Sheets. (SQL experience is a strong plus).
- Hands-on experience with annotation platforms (e.g., Labelbox, Scale AI, Appen Global, CVAT).
Similar Jobs
Director, Solutions Strategy – Operations
Director of Solutions Strategy & Operations responsible for operational foundations in fast-growing functions. Building systems for technical teams with a focus on scalable solutions.
Product/Process Engineer, Broadband Operations
Product/Process Engineer leading the manufacturing lifecycle of optoelectronic products at Ciena. Ensuring production readiness, reliability, and performance for high-performance networks globally.
Associate, Market Operations
Associate, Market Operations ensuring integrity of DraftKings’ prediction markets. Manage market activities, provide operational support, and conduct surveillance in a dynamic regulatory context.
Senior Manager – Testing Operations
Sr. Manager Testing Operations at Luminare Health managing testing strategy and planning meetings. Leads testing programs ensuring business outcomes by developing consultative teams and engaging methodologies.
Community Operations Intern
Intern role in community operations directly engaging users through various interactions. Focusing on user behavior understanding and relationship building for growth opportunities.
Client Operations Associate
Remote Client Operations Associate role at Blyss Journeys, responsible for client requests and documentation in a flexible scheduling environment.
Lead, Executive Operations
Lead, Executive Operations to structure execution for CEO and leadership team. Collaborate with teams to translate priorities into actionable plans for improved clarity.
Lead, Executive Operations
Lead, Executive Operations focusing on building simple structures for the CEO and leadership team. Coordinating across functions to improve clarity and reduce friction in high-priority work.
Operations Executive – Junior Production Coordinator, 1–3 Years Experience
Operations Executive supporting residential interior fit-out projects and coordinating with internal teams for smooth execution. Ensuring positive customer experience and adherence to design standards.
Senior Operations Analyst
Senior Operations Analyst at Argano overseeing project data and delivery processes. Driving operational excellence and strategic initiatives with cross-functional collaboration and mentorship.
Lead IT Operations Engineer
Lead IT Operations Engineer managing cloud IT infrastructure and operations, focusing on Microsoft 365 and Azure configurations while ensuring security and efficiency.
Cloud Cost Analyst – Cloud FinOps
Cloud Cost Analyst optimizing cloud spend for the National Institute of Allergy and Infectious Disease. Collaborating with clients on financial operations and cloud economics best practices.
Country Study Operations Manager – France
Country Study Operations Manager responsible for study management in clinical trials at Parexel. Leading study activities and vendor management with a focus on patient well-being.
Revenue Specialist III
Revenue Specialist III responsible for investigating and designing strategic initiatives to improve revenue cycle performance. Analyze data trends and support billing processes in healthcare Finance sector.
Operations Director
Operations Director at Solenis managing operational performance across nine manufacturing sites in North America. Leading safety, compliance, quality, and continuous improvement efforts in a complex manufacturing environment.
Operations Associate
Operations Associate overseeing pre and post-trade processing for broker-dealer operations. Collaborating across teams to optimize financial operations and regulatory compliance.
Operations Associate
Operations Associate supporting the operational infrastructure of a globally distributed crypto wallet team. Managing internal coordination systems and driving cross-functional accountability.
Senior Manager – RCM – Coding Operations
Sr. Manager overseeing revenue cycle management and coding operations at US Anesthesia Partners. Providing leadership and expertise in coding edits and denials while improving operational performance.
Staking Operations Lead
Lead Staking Operations for network health at Solana Foundation. Focus on decentralization and maintain relationships with validators and stakers in the Solana ecosystem.
Global Operations Quality Manager
Global Operations Quality Manager at Syneos Health responsible for leading quality management in clinical trials. Ensuring compliance and collaboration within study teams to achieve quality goals.