How can you compare two prompt versions fairly?
In the documented Agenta 1.0 playground, give two versions the same question and change only one thing, such as the instructions. Save useful examples as a test set so you can repeat the comparison later.
When should a person judge the answer?
In Agenta 1.0, an exact match can check a known label, and a person can decide whether a help response makes sense to a new reader. A model-based evaluator needs selected data to score the answer. Check which fields it receives and remove private material first.
What should you check before running a larger batch?
Agenta 1.0 documents exporting human evaluation results. Confirm that your version supports the specific testing and export steps you need. Agenta 2.0 has separate cloud pricing, with credits for included model usage and agent runtime. If you run Agenta yourself, budget for your own computing, storage and any connected model calls.
Did a shorter answer still include the next step?
Write two fictional support replies for someone changing an appointment. Give both the same time and account situation.
- Keep the input constantChange only the reply style. Check whether each answer still tells the person where to go next.
- Save the criterionRecord the required next action and what a human reviewer would consider unclear.