Software Engineer
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
Part-time Software Engineer evaluating AI-generated technical responses and documentation for 24-MAG’s remote consulting platform. Scoring accuracy, clarity, and usefulness with detailed written rationales.
Responsibilities:
- Evaluate and compare AI-generated responses to technical prompts for correctness, clarity, tone, and usefulness
- Review technical documentation, error messages, code comments, architectural explanations, instructions, and related software-engineering content
- Determine whether generated responses accurately address underlying technical questions or requirements
- Identify factual, logical, or technical inaccuracies in fluent or convincing outputs
- Assess whether technically correct responses are clearly and appropriately communicated to their intended audience
- Review technical writing, design documents, API references, specifications, and comparable technical materials for precision, completeness, structure, and instructional value
- Identify unclear explanations, ambiguous wording, misleading statements, inconsistencies, and missing information
- Assign analytical quality scores to AI-generated technical responses
- Write detailed rationales explaining evaluation decisions
- Document subtle errors, inconsistencies, or ambiguities precisely and consistently
- Provide actionable written feedback to support improvements in AI model performance
- Work independently within structured remote evaluation workflows
- Collaborate asynchronously with project managers, reviewers, and other expert contributors
- Follow established evaluation guidelines, rubrics, and project-specific quality standards
- Complete assigned work within agreed project requirements and defined timeframes
Requirements:
- Professional experience as a Software Engineer working with a modern technology stack
- Native-level English fluency and exceptional written communication skills
- Proven experience authoring or reviewing technical documentation, design documents, API references, specifications, or similar materials
- Experience conducting code reviews, design reviews, or providing structured feedback on technical writing
- Strong ability to assess technical communication for correctness, audience appropriateness, and instructional value
- Excellent analytical skills and meticulous attention to detail
- Ability to identify subtle factual inaccuracies, logical inconsistencies, ambiguous explanations, and misleading technical language
- Strong independent-working skills and comfort operating within remote, asynchronous project environments
- Familiarity with AI model evaluation, annotation, RLHF, or related structured evaluation workflows is advantageous but not required
- No prior experience in AI is required
- Work must be completed without using confidential or proprietary information belonging to any employer, client, institution, or other third party
Benefits:
- Part-time independent contractor engagement
- Fully remote
- Compensation of $90–$140/hour
- Flexible project scope, workload, timing, and duration depending on project requirements
Similar Jobs
Member of Technical Staff, Finance Research
Finance AI researcher developing benchmarks and evaluation frameworks for advanced financial reasoning and agentic systems. Conducting applied research across AI models, enterprise workflows, and quantitative finance.
Software Engineer
Contract Software Engineer contributing code, debugging, optimization, and documentation across major programming languages. Reviewing technical submissions to improve AI training data quality.
Junior Sales Manager – Plant Engineering, Industrial Plant Design & Digitalization
Junior Sales Manager supporting sales of digital engineering, CAD/BIM, and software solutions for plant design. Presenting 3D modeling, laser scanning, and database solutions to customers.
Interface Developer, Pentaho
Interface Developer building Pentaho ETL, APIs, and enterprise integrations for CGI’s government ERP platform. Supporting secure data exchange and modernization initiatives for public-sector organizations.
Associate Developer
Associate Developer building software for HealthStream, a healthcare workforce solutions company. Designing, troubleshooting, and maintaining products using SQL, CI/CD, and development tools.
COBOL Developer
COBOL Developer maintaining and modernizing legacy mainframe applications for federal government agencies. Supporting COBOL, JCL, production operations, testing, compliance, and system releases.
Software Engineer
Software Engineer developing and maintaining C++ Windows CAM software for GibbsCAM, an established manufacturing platform. Debugging mature codebases and expanding specialized machining capabilities.
Software Engineer
Advanced Software Engineer building .NET, AWS, and data-driven applications for Agilent’s scientific instruments and laboratory systems. Developing APIs, analytics, ETL pipelines, and scalable enterprise solutions.
Senior Energy Risk Engineering Consultant
Senior risk engineering consultant assessing energy casualty risks for Zurich Insurance customers. Providing safety consultation, mitigation strategies, and underwriting support across the U.S.
iOS Mobile Developer
iOS developer building resilient Swift applications for Accenture Federal Services, a technology company serving the US federal government. Leading mobile development teams through production delivery.
Assembler Developer – SDE I
Software Development Engineer I developing and testing mainframe assembler software at CSG, which helps companies connect customers with valued services. Supporting debugging, QA, and Agile delivery.
Senior Manager, AI Forward Deployed Engineering
AI FDE leader scaling Databricks’ enterprise GenAI delivery teams. Driving customer AI transformations, strategic accounts, product influence, and production AI solutions.
Tooling Developer
Microsoft tooling manager leading Power Platform, Azure, and DevOps services for Veterans Health Administration operations. Improving IT governance, data management, and strategic decision-making.
Software Engineer
Software Engineer building AWS serverless services for Bring a Trailer’s classic and enthusiast vehicle auction platform. Modernizing WordPress and PHP workflows through event-driven architecture.
Engineer, Merger & Acquisition Engineering
M&A engineer integrating acquired companies into ABACUS’s Microsoft cloud environment. Migrating Microsoft 365 tenants, identities, storage, and communication tools with minimal downtime.
Technical Developer Specialist – Contract
Technical Developer Specialist troubleshooting Roblox beta projects and improving developer resources. Supporting creators building interactive 3D experiences with Roblox Studio and Lua/Luau.
Materials and Engineering Specialist
Materials and engineering expert creating scorable tasks for Mercor’s AI-model training projects. Testing model reasoning through domain-authentic prompts, data rooms, and grading criteria.
Materials and Engineering Specialist
Materials and engineering expert creating objectively graded tasks for Mercor’s AI-lab and enterprise model-training projects. Testing, analyzing, and refining model performance with domain experts.
Desarrollador/a Senior en SISnet
Desarrollador/a Senior SISnet desarrollando e integrando soluciones para el sector asegurador de Inetum. Mantenimiento de módulos core, APIs, bases de datos y procesos CI/CD.
Director of Engineering – Integration Platform
Engineering Director leading Intermedia’s cloud communications integration platform and Unite Integrations Builder. Driving APIs, integrations, architecture, and engineering teams.