AI Evaluations Engineer – School of Government and Policy
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
Remote AI evaluations engineer managing sensitive data, audits, queries, and reports. Supporting Johns Hopkins’ School of Government and Policy with data processes and user training.
Responsibilities:
- Provide data support for the team, end-users, and senior management
- Validate data and completion or routing of reporting projects
- Handle data correction requests, escalate issues, and execute approved corrections
- Guide clients and business partners on data and standardized business processes
- Maintain accurate records and audit data entry for accuracy and compliance
- Prepare and present data metrics
- Create standard and ad-hoc reports for local, state, and federal reporting compliance
- Participate in creating queries and reports
- Initiate technology service requests for data-processing modules and reports
- Support data requests, data availability, request feasibility, ad-hoc reporting, and technical support
- Provide end-users with guidance and informal training
- Create internal training tools and process guides
- Prepare and present workshops explaining system requirements and procedures
- Respond to technical issues and recommend changes to the data platform
- Perform other duties as assigned
Requirements:
- High School Diploma or graduation equivalent
- Three years of administrative/academic related experience
- Additional education may substitute for required experience under the JHU equivalency formula
- Ability to enter, maintain, audit, and process sensitive data
- Ability to prepare and process ad-hoc and scheduled reports
- Ability to resolve data anomalies and inconsistencies
- Ability to follow confidentiality and safety policies
- Successful candidates must complete a pre-employment background check
- Faculty, staff, and students must receive the seasonal flu vaccine, subject to stated exceptions
Benefits:
- Total rewards package supporting health, life, career, and retirement
- Seasonal flu vaccine required, with religious or medical exceptions
- Required vaccinations for applicable positions provided at no cost through Occupational Health
- Workplace accommodations and accessibility support
- Equal opportunity employment
- Pre-employment background check process
- Remote work arrangement
Similar Jobs
AI Safety Red Teamer
AI Safety Red Teamer stress-testing frontier AI systems for Mercor. Designing adversarial prompts and documenting vulnerabilities to improve model safety and alignment.
Research Staff, Voice AI Foundations
Research Staff pioneering latent-space models, neural audio codecs, and speech systems at Deepgram. Advancing scalable, efficient voice AI for real-time human-machine interaction.
Research Staff – Voice AI Foundations
Research Staff developing latent-space models for Deepgram’s voice AI platform. Advancing neural audio codecs, generative speech, multimodal systems, and hardware-efficient inference.
Research Staff, Voice AI Foundations
Research Staff pioneering latent-space models for codecs, speech generation, and multimodal voice AI. Advancing Deepgram’s real-time speech platform through scalable, efficient foundational research.
AI Operations Engineer
AI Operations Engineer building and supporting enterprise AI agents, chatbots, and integrations. Enabling secure, governed AI operations for Turtle’s electrical distribution business.
Business Logic Expert – AI Community
Business Logic Expert reviewing AI-generated business content for professional accuracy and etiquette. Helping TELUS Digital improve AI systems through remote, flexible freelance work.
Head of Sales – Custom Software & AI
Head of Sales für Sidestreams individuelle Software- und KI-Lösungen. Verantwortung für Neukundengeschäft, Teamführung, Abschlüsse und Upsell in Deutschland.
AI Safety Expert – English, Finnish
AI safety experts red-teaming conversational AI models for Mercor, which partners with leading labs to train frontier systems. Producing reproducible vulnerability data and attack cases.
Lead Analyst, Commercial Data – AI
Lead Analyst enabling commercial data, CRM analytics, and AI for Johnson & Johnson’s orthopaedics business. Building trusted datasets, KPI frameworks, and responsible GenAI solutions.
Senior Sales Coordinator, AI Factory
Sales Coordinator supporting complex AI infrastructure deals at Armada, the edge AI infrastructure hyperscaler. Managing sales operations, Salesforce data, NDAs, travel, events, and customer coordination.
OCM Manager – AI Fluency, Transformation
OCM Manager helping Unisys transform enterprise work through AI fluency, adoption, and governance. Building learning programs, operating models, and measurable AI-enabled change.
Senior Consultant – Grants Innovation, AI Solutions
Senior Consultant modernizing federal grants management for Guidehouse’s federal clients. Evaluating AI solutions, requirements, policies, and quality assurance across the grants lifecycle.
Senior Commercial – AI Solutions Partner
Enterprise commercial partner selling AI-driven software delivery transformations for Yopeso, a software product development company. Structuring outcome-based deals and building strategic partnerships across European markets.
Insurance Expert – AI Training, Evaluation
Insurance experts creating underwriting and claims tasks for Weekday's AI training project. Evaluating advanced models and explaining errors with rigorous insurance judgment.
Director, AI Data Operations
Director building Bill Gosling Outsourcing’s AI Data Operations practice. Developing scalable AI services, partnerships, client solutions, and revenue streams for global delivery.
Behavioral Health Expert – AI Safety, Model Evaluation
Behavioral health experts evaluating sensitive AI conversations for leading AI labs. Defining safe, neutral responses and improving model judgment.
AI Safety Experts, English – Gujarati
AI safety red-teamer probing frontier AI models for Mercor. Testing vulnerabilities, annotating failures, and producing reproducible datasets and attack cases.
Senior Manager, AI Business Modernization
AI modernization leader advancing Hershey’s U.S. snacks business through AI products, agentic solutions, and measurable productivity gains. Driving adoption, governance, and value realization across functions.
Engineer III, AI SDLC Engineer
AI SDLC Engineer building agentic workflows for CrowdStrike’s cybersecurity platform. Improving developer productivity through LLM infrastructure, observability, model routing, and benchmarks.
Working Student, Sales Operations & AI
Working student supporting PTV Logistics, a transportation logistics software company, with AI-driven sales operations. Building analytics, improving CRM data quality, and optimizing business processes.