AI Agent Power User
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
Part-time AI workflow consultant evaluating ChatGPT and Claude across realistic business processes. Scoring outputs, documenting model behavior, and providing actionable feedback for 24-MAG’s AI projects.
Responsibilities:
- Run complex, multi-step professional scenarios using AI assistants such as ChatGPT and Claude
- Recreate realistic business workflows involving analysis, communication, documentation, research, and operational tasks
- Assess AI performance across each stage of a workflow
- Evaluate whether AI-generated outputs are useful, complete, and practically actionable
- Score AI-generated outputs using defined rubrics, quality criteria, and assessment standards
- Review responses for completeness, relevance, accuracy, clarity, and professional usefulness
- Identify quality issues, inconsistencies, and recurring model-performance patterns
- Write clear, constructive, and actionable feedback
- Document workflow steps for transparency and reproducibility
- Explain where and why AI performance succeeds or breaks down
- Maintain detailed written records for AI research and product teams
- Connect AI assistants with workplace platforms such as Google Drive, Gmail, Slack, and Notion
- Test connected, multi-tool business processes
- Assess realistic professional AI adoption and identify opportunities to improve reliability and usability
Requirements:
- Minimum of 5 years of professional experience in a business function involving technology-enabled problem solving
- Completed Bachelor's degree or higher in any discipline
- Consistent, hands-on use of ChatGPT and/or Claude for real professional work
- Strong familiarity with AI-assisted workflows, prompt-based work, and practical business use cases
- Experience applying, creating, or reviewing evaluation rubrics, QA scorecards, grading criteria, or content-review guidelines
- Academic evaluation, quality assurance, hiring assessment, annotation, or similar structured-review experience is advantageous
- Comfortable integrating AI tools with workplace software and productivity platforms
- Excellent written English
- Strong attention to detail, analytical judgement, and process-improvement skills
- Ability to work independently within structured remote and asynchronous project environments
- Based in the United States, Canada, United Kingdom, Ireland, Australia, or New Zealand
- Authorised to undertake contract work in the relevant country
- No prior formal experience in AI research or model training is required
Benefits:
- Part-time independent contractor engagement
- Fully remote work
- Compensation of $30–$90/hour
- Flexible project scope, workload, timing, and duration depending on project requirements
Similar Jobs
Especialista en Inteligencia Artificial Agente, Banca
Ingeniero de software diseñando soluciones escalables con agentes inteligentes, APIs y automatización. Empresa tecnológica especializada en activos digitales, staffing y fábrica de software.
AI Evaluation Specialist
AI Evaluation Specialist assessing AI-generated outputs, reasoning, and tool use for 24-MAG LLC’s remote consulting platform. Providing rubric-based quality evaluations and actionable written feedback.
AI Safety Red Teamer
AI Safety Red Teamer stress-testing frontier AI models for Mercor. Designing adversarial prompts, identifying vulnerabilities, and improving model safety with leading AI researchers.
AI Safety Red Teamer
AI Safety Red Teamer adversarially testing frontier AI systems for Mercor. Identifying jailbreaks, hallucinations, and safety failures across high-risk domains.
AI Safety Red Teamer
AI Safety Red Teamer identifying vulnerabilities in frontier AI models for Mercor. Designing adversarial prompts and evaluating high-risk AI behavior.
AI Safety Red Teamer
AI Safety Red Teamer stress-testing frontier AI models for Mercor. Designing adversarial prompts and documenting vulnerabilities across high-risk domains.
Adjunct Instructor, Generative AI
Adjunct Generative AI Instructor teaching adult learners through competency-based online education. Providing AI expertise, personalized support, feedback, and mentorship at Calbright College.
AI Visual Artist
Freelance AI Visual Artist experimenting with generative image, video, music, and text tools. Creating innovative visual concepts for Paintgun’s performance advertising campaigns.
Senior Marketing Strategy, AI Evaluation Specialist
Senior marketing strategist designing evaluation tasks and rubrics for generative AI models. Assessing AI-generated marketing content using brand, growth, and performance expertise.
AI Evaluation Guidelines & Rubric Specialist
AI evaluation specialist developing precise rater guidelines and scoring rubrics for generative AI programmes. Translating complex requirements across finance, legal, retail, insurance, and sports domains.
U.S. Licensed Dermatologist – AI Dermatology Research
U.S.-licensed dermatologist reviewing skin images and AI-generated assessments for 24-MAG LLC’s dermatology research. Establishing clinical guidelines and improving medically grounded AI systems.
AI Safety Red Teamer
AI Safety Red Teamer identifying vulnerabilities in frontier AI systems for Mercor. Designing adversarial prompts and evaluating high-risk model behavior with AI researchers.
AI Safety Red Teamer
AI Safety Red Teamer adversarially testing frontier AI models for Mercor, which partners with AI labs to train and enhance them. Documenting vulnerabilities across high-risk domains.
AI Safety Red Teamer
AI Safety Red Teamer stress-testing frontier AI models for Mercor. Designing adversarial prompts and documenting vulnerabilities to improve AI alignment and safety.
AI Data & Insights Officer
AI Data & Insights Officer turning event and luxury-travel business data into reliable intelligence. Building RAG, dashboards and agent-performance insight for Worldwide Events Group.
AI Data & Insights Officer
AI Data & Insights Officer transforming Worldwide Events Group’s MICE and luxury travel data into reliable intelligence. Building RAG knowledge systems, dashboards and measurable agent performance insights.
Data & AI Analyst
Data & AI Analyst transforming insurance marketing data into actionable insights, dashboards, and AI-powered automation. Building scalable solutions with SQL, BigQuery, Tableau, Python, and emerging AI tools.
AI Safety Red Teamer
AI Safety Red Teamer stress-testing frontier AI models for Mercor. Designing adversarial prompts and documenting vulnerabilities across high-risk domains.
AI Safety Expert, English, Finnish
AI red team expert testing Mercor’s frontier AI models through jailbreaks, prompt injections, and adversarial data generation. Producing reproducible vulnerability reports that improve AI safety.
AI Director
AI Director leading production AI architecture, governance, and client strategy at Futurice, a data-and-AI consultancy. Growing UK capability through consulting, business development, mentoring, and hiring.