Prompt Engineer – LLM Systems, Evals & Safety
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
Designing high-quality prompts and improving LLM features for webook.com, Saudi’s #1 event ticketing platform. Partnering with engineers to integrate safe and effective prompts into production.
Responsibilities:
- Design high-quality prompts, system instructions, and tooling that make our LLM features accurate, safe, and cost-effective.
- Own evaluation, prompt versioning, and continuous improvement.
- Author, refactor, and chain prompts (system/tool/policy) for varied tasks.
- Create offline/online evaluation harnesses (rubrics, golden sets, metrics).
- Build prompt libraries with versioning, A/B testing, and telemetry.
- Reduce hallucinations via verification, constrained decoding, and tool use.
- Implement safety: jailbreak/prompt-injection tests, content policy checks, PII handling.
- Partner with engineers to integrate prompts into production features.
Requirements:
- Demonstrated prompt design across multiple task types and models.
- Experience building eval datasets and automated scoring (e.g., accuracy, faithfulness, utility, cost/latency).
- Familiarity with retrieval-augmented generation concepts and tool/function calling.
- Strong scripting (Python/TypeScript) for data prep, evals, and analysis.
- Clear writing; ability to translate business goals into measurable prompt specs.
- Nice-to-Haves
- Experience with LangChain/LLM orchestration, vector stores, and rerankers.
- Knowledge of safety tooling and red-teaming techniques.
- Experiment platforms (feature flags, A/B tests), analytics.
Similar Jobs
Power Systems Study Engineer – III
Power Systems Study Engineer III at DLB Associates working on power system studies for mission critical projects. Collaborating within the Engineering Team to optimize performance.
Cloud Operation Engineer
Cloud Operation Engineer responsible for designing, implementing, and maintaining server infrastructure at CSG Systems, Inc. Ensuring stability and security for enterprise systems in both on-premise and cloud environments.
Senior I&C Engineer – Energy
Senior I&C Engineer Energy designing and maintaining systems for nuclear facilities. Delivering technical documentation and ensuring compliance with nuclear safety regulations.
DFT Engineer
Senior DFT Engineer leading end-to-end DFT strategy for complex SoCs at Niobium Microsystems. Overseeing architecture, integration, and validation processes within the team.
External Lighting Engineer
External Lighting/Street Lighting Engineer at Fixatex performing installations and maintenance of street and external lighting across London. Ensuring compliance with safety standards and delivering quality service in social housing and local authority sectors.
Thermal Hydraulics Engineer 3
Thermal Hydraulics Engineer developing thermal systems analysis for NuScale Power’s microreactor technology. Collaborating on modeling, analysis, and software selection within a dynamic team.
Interconnection Engineer
Interconnection Engineer managing engineering items for clean energy solutions at WECTEC Staffing Services. Collaborating with corporate teams and Transmission Owners to ensure interconnection agreements are met.
Sandbox Agentic Engineer
AI Agent Engineer developing LLM-based AI solutions for cybersecurity at CrowdStrike. Leading AI agent development and optimizing performance with modern frameworks and technologies.
Critical Power Field Technician, UPS Engineer
Critical Power Technician maintaining critical power systems in Nashville, TN. Role involves technical support, customer service, and mentoring crew members in UPS systems.
Forward Deployed Engineer
Forward Deployed Engineer at Truelogic Software integrating AI solutions with clients. Collaborating on impactful projects to enhance digital transformations in businesses.
LLM Evaluation Engineer
Develop evaluation systems for AI behavior in enterprise environments using LLMs and real-time data. Collaborate on guardrails and enforcement strategies for AI compliance.
Chief Engineer – Container Vessel
Chief Engineer leading Engine room operations for Columbia Shipmanagement's fleet, ensuring equipment functionality and safety. Requires strong experience with specific engine types and good communication skills.
Technical Engineer
Technical Engineer for the Wendy's engineering team focusing on mobile development and problem solving.
Staff IT/MSP Engineer
Staff IT Engineer responsible for technical health of MSP's client base, designing deployment standards. Bridging business goals with technical execution, reporting to CTO or Director of Engineering.
Customer Delivery Engineer
Customer Delivery Engineer at Tiger Data implementing PostgreSQL projects and optimizing customer experiences with AI tools during migrations and production rollouts.
Senior Identity Access Management Engineer
Senior IAM Engineer improving Identity and Access Management processes at U.S. FinTech. Responsible for automation, documentation and overseeing AWS IAM solutions for security compliance.
Senior Software V&V Engineer
Senior Software V&V Engineer ensuring thorough verification and validation of software in cancer diagnostics. Collaborate with cross-functional teams for bioinformatics pipeline testing and regulatory compliance.
Project Engineer
Project Engineer working on maintenance and development of automation systems. Involves offshore operations and all stages of automation lifecycle support in Rio de Janeiro.
Project Engineer
Project Engineer maintaining automation systems for customer facilities including offshore operations. Engaging in all stages of systems lifecycle from requirements gathering to startup.
Systems Monitoring Engineer
Systems & Monitoring Engineer architecting monitoring stack for Hedera’s DLT ecosystem. Blending NOC practices with Web3 challenges to ensure uptime and compliance.