On November 15, Anthropic launched a feature in the developer console to directly improve prompts and manage examples. Try it out: https://console.anthropic.com/dashboard
Let me show you the step-by-step results I used. I had a prompt that helped me learn English before, and I asked for modifications.
Step 1: Submit the original prompt
Step 2: Understand the intent of the prompt and develop a plan
Step 3: Draft (initial optimization)
Step 4: Optimize the plan
Step 5: Write the final prompt
Step 6: Generate the final version
Step 7: Test in Workbench
Step 8: Generate Response
Core functions and effects
The prompt optimization tool enhances existing prompts through the following methods, making the model's generation more accurate and reliable:
: Adding a dedicated reasoning section in the prompt to guide the Claude system to think about problems systematically, thereby improving the accuracy and consistency of responses. ：Convert to a consistent XML format ：Add a coherent reasoning section to align with the new structure of the prompt ：Optimize its structure, correct minor errors in grammar or spelling, and make the expression clearer and more understandable. : Add a pre-filled section of Assistant messages to the prompt, which guides Claude's actions and ensures that the output conforms to the specified format.
Dynamic feedback, continuous improvement:
After generating new prompts, users can provide feedback to Claude, pointing out which parts are effective and which need improvement. This iterative optimization mechanism further enhances the applicability and performance of the prompts.
Test results: Significant improvement in model performance
In internal testing, the prompt optimization tool demonstrated the following significant results:
Multi-label classification tasks: Accuracy improved by 30%. Summary generation tasks: Word count control achieved a 100% compliance rate.
Multi-example management function: A powerful tool for improving response quality.
Adding examples to prompts is one of the most effective ways to improve model response quality, especially when requiring Claude to strictly adhere to a specific output format. These examples can be managed in a structured format directly in the Workbench, making it easy to add, edit, and optimize them to further enhance response quality.
Key Features:
: Add new examples or edit existing ones in a clear input/output pair format through Workbench. : Utilize Claude's example generation feature. Claude will automatically create synthetic inputs and provide you with output drafts, greatly simplifying the process of adding examples. : Through the visual interface in the Anthropic console, you can intuitively view, manage, and adjust prompt examples, ensuring that every step of the operation is clear and precise.
Advantages brought by adding examples
: Reduce the model's misunderstanding of instructions and ensure the accuracy of task completion. : The output format always meets expectations, no matter how complex the task is. : Significantly improves Claude's ability to handle complex tasks.
Prompt Evaluation and Ideal Output: Creating More Efficient Prompts
Anthropic provides a prompt evaluation tool that allows users to test the effectiveness of prompts in various scenarios. To help developers benchmark and improve prompt performance, an optional "ideal output" column has been added in the Evaluations tab, assisting new users in effectively evaluating and scoring model outputs.
Function Details:
: Developers can define expected results in the ideal output column and then score the quality of the model's output on a scale of 5. This method helps you quantitatively assess the effectiveness of the prompts. ：By testing new prompts, further feedback can be provided to the prompt optimization tool, indicating which areas still need improvement. The optimization tool will adjust the prompts and example content based on the feedback to ensure that the output quality is closer to expectations. ：The prompt optimization tool supports modifying the output format according to specific needs. For example, you can request to convert XML formatted output into JSON format.