Senior/Staff AI Model Engineer
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
AI model engineer owning evaluation, safety, and reliability for Clear Street’s AI copilot. Improving model quality across trading workflows on a cloud-native brokerage platform.
Responsibilities:
- Own the reliability and quality bar for an AI copilot embedded in a trading platform used by sophisticated investors
- Design and build evaluation systems measuring correctness, safety, latency, and regression risk across market analysis, portfolio/risk reasoning, and trading workflows including order placement
- Develop and maintain benchmarks including curated golden sets, scenario suites, stress/adversarial cases, and refreshed market/regime-based test corpora
- Build automated quality gates and regression workflows that block releases when key metrics degrade
- Partner with engineering and product to define safe tool/action contracts with deterministic previews, confirmations, and auditability
- Own model improvement loops tied to evaluations, including data collection/labeling strategies, error taxonomy, prompt/tooling changes, and appropriate fine-tuning or preference optimization
- Design and operate AI monitoring and incident response, including telemetry, alerting, root-cause analysis, and fix-forward processes
- Develop deep understanding of trading concepts such as margin, shorting, portfolio margin, risk, and execution, and express them accurately to users
- Work with the technology stack: Rust, TypeScript, Postgres, React, React Native, observability/telemetry tooling, LLM APIs, model serving, and evaluation/training pipelines
Requirements:
- At least Eight (8) years of experience shipping production software
- Strong proficiency with any programming language
- Strong knowledge of computer science fundamentals, testing methodology, and systems design
- Experience building evaluation frameworks, test harnesses, and benchmark suites for complex systems (LLMs/agents/search/retrieval/ranking/recommenders)
- Experience running model improvement cycles: dataset curation, labeling/QA, offline experimentation, and deploying changes with measurable impact on benchmarks
- Ability to define metrics, build measurement pipelines, and drive engineering/product decisions from data
- Comfort working across the stack: debugging model/tooling failures, instrumenting services, and partnering with frontend/product on UX patterns that improve safety and trust
- High degree of self-motivation and willingness to jump into unfamiliar areas to solve problems
- Bonus: Experience with fine-tuning, preference optimization, distillation, or prompt/compiler-style techniques for improving tool-use reliability
- Bonus: Experience creating domain-specific benchmarks and adversarial suites for high-stakes applications
- Bonus: Deep experience with trading across asset classes, margin types, etc.
- Bonus: Experience with Rust and performance-sensitive services
- Bonus: Experience designing incident response and SLOs for ML/AI systems
Benefits:
- Company equity
- 401k matching
- Gender neutral parental leave
- Full medical, dental and vision insurance
- Lunch stipends
- Fully stocked kitchens
- Happy hours
- Great location
- Amazing views
- Equal opportunity workplace
Similar Jobs
AI Safety Specialist
AI safety specialist supporting Mercor’s short-term AI safety project. Applying red-teaming and related domain expertise during a fast-paced remote sprint.
AI Safety Expert – English, Marathi
AI safety red teamer probing conversational AI models for Mercor, which partners with AI labs and enterprises to train frontier systems. Generating reproducible vulnerability data and attack cases.
AI Safety Expert – English, Tamil
AI safety red teamer probing conversational AI models for Mercor, which partners with leading labs to train frontier systems. Generating vulnerability data and reproducible attack artifacts.
AI Safety Expert, English, Punjabi
AI red-teaming experts probing conversational models for Mercor’s frontier AI safety projects. Generating vulnerability data, attack cases, and reproducible reports for AI labs and enterprises.
AI Safety Red Teamer
AI Safety Red Teamer stress-testing frontier AI models for Mercor. Identifying vulnerabilities and improving model alignment, robustness, and safety through adversarial evaluation.
AI Safety Red Teamer
AI Safety Red Teamer stress-testing frontier AI models for Mercor. Identifying vulnerabilities and improving model alignment, robustness, and safety.
Trainer for CAD and Artificial Intelligence in Engineering
CAD- und KI-Trainer:in für eine Qualifizierung zur modernen digitalen Produktentwicklung. Vermittlung von Generative Design, KI-gestützten CAD-Workflows und technischer Bewertung.
Trainer – CAD, Artificial Intelligence & Engineering
Trainer:in für CAD und KI im Engineering bei einem digitalen Weiterbildungsanbieter. Vermittlung praxisnaher KI-gestützter Konstruktions- und CAD-Workflows an Maschinenbau-Fachkräfte.
Senior Director Analyst – AI Technology Economics
Senior Gartner analyst shaping AI economics research across technology markets. Developing thought leadership and advising executives through data-driven economic insights.
AI Safety Expert – English, Kannada
AI Safety Experts red-teaming conversational models for Mercor, which partners with AI labs to train frontier systems. Probing vulnerabilities and producing reproducible safety data.
AI Safety Expert – English, Kannada
AI safety red teamer probing conversational AI models for Mercor, which partners with leading AI labs to train frontier systems. Creating vulnerability data, attack cases, and reproducible reports.
AI Safety Expert – English, Marathi
AI safety red-teamer probing Mercor’s frontier AI models with adversarial English and Marathi inputs. Generating vulnerability annotations, attack cases, datasets, and reproducible reports.
AI Safety Red Teamer
AI Safety Red Teamer designing adversarial prompts and uncovering vulnerabilities in Mercor’s frontier AI projects. Evaluating model behavior across high-risk domains and supporting AI safety research.
Staff AI Builder
Staff AI Builder designing and shipping production GenAI systems for Robots and Pencils’ enterprise clients. Leading agentic AI, RAG, AWS architecture, reliability, and technical mentoring.
AI Benchmark Engineer, Native Language Specialist – Chinese
AI benchmark engineer building Chinese-language Terminal-Bench tasks for Lilt’s multilingual AI platform. Validating coding agents, reference implementations, and deterministic verifiers.
AI Benchmark Engineer, Native Language Specialist – Chinese
AI benchmark engineer creating Chinese-language Terminal-Bench tasks for Lilt’s multilingual AI platform. Validating coding agents, verifier scripts, and multilingual software robustness.
AI Benchmark Engineer, Native Language Specialist – Arabic
Arabic AI benchmark engineer designing Terminal-Bench tasks for Lilt’s multilingual AI and human-verified language services. Building multilingual assets, verifiers, and calibration workflows for coding agents.
AI Benchmark Engineer, Native Language Specialist – Arabic
Arabic-speaking benchmark engineer designing Terminal-Bench tasks for LILT’s multilingual AI evaluation suite. Building datasets, verifiers, and calibrated tests for coding agents.
AI Safety Experts – English, Telugu
AI safety red-teamer probing conversational models for Mercor’s human-data AI training projects. Generating vulnerability datasets, attack cases, and reproducible reports to strengthen frontier AI systems.
AI Safety Expert – English, Tamil
AI red-team expert probing conversational models for Mercor, which partners with AI labs to train frontier systems. Generating adversarial data, vulnerability reports, and reproducible attack cases.