AI Prompt Engineer

Vagas Externas — Remotar · remote · CLT · pleno
Descrição

You will design and refine prompts, system instructions, and evaluation rubrics to improve LLM outputs for production tasks such as summarization, classification, extraction, and customer support. You will run prompt evaluation experiments, analyze failure modes (hallucinations, toxicity, policy violations), and collaborate with engineers to integrate prompts into tools, agents, and retrieval-augmented generation (RAG) workflows. You will contribute to RLHF-ready artifacts including preference comparisons, prompt-response pairs, and prompt templates aligned to annotation guidelines compliance and training data quality.

Key Responsibilities

- Responsibilities include: building reusable prompt libraries and style guides; 

- performing prompt testing, red-teaming, and regression suites; 

- defining evaluation criteria and scoring rubrics for QA evaluation; 

- supporting RLHF and prompt evaluation workflows with clear labeling instructions; 

- iterating on prompts to reduce hallucination and improve factuality; 

- partnering with NLP and platform teams to ship prompt templates; 

- documenting experiments, metrics, and learnings; 

- contributing to content safety labeling, policy compliance checks, and guardrail prompts when required.

Required Qualifications

- Qualifications include: experience delivering prompt engineering or LLM evaluation work in production or large-scale programs; 

- strong writing and structured reasoning skills; 

- familiarity with NLP concepts and LLM failure modes; 

- ability to design experiments and interpret results; 

- experience with prompt optimization patterns (few-shot, chain-of-thought style structuring where permitted, tool instructions, and constrained generation); 

- ability to write clear annotation guidelines and QA checklists; 

- comfort working remotely with cross-functional stakeholders.

- Preferred Qualifications

- Nice to have: experience with RLHF pipelines, preference ranking, or human-in-the-loop evaluation; 

- experience with RAG prompting, tool calling, or agentic workflows; 

- familiarity with content safety, policy, and adversarial prompting; 

- experience with named entity recognition and information extraction tasks; 

- comfort with Python/SQL for analysis, prompt telemetry, and experiment tracking; 

- exposure to computer vision annotation or multimodal evaluation.

Processo seletivo inclui:
Selecionamos as principais informações da posição. Para conferir o descritivo completo, clique em "acessar"
Candidatar-se

Esta vaga é agregada de Vagas Externas — Remotar. A candidatura é feita no site da fonte.
Candidatar-se no site da fonte