Anthropic workbench

The Anthropic Workbench

Author: Jacob Forward, Thursday, November 28th, 2024

Starting out:

The best place to start your prompt-writing journey on the Anthropic Workbench is the 'Dashboard' tab, which includes such options as writing a prompt from scratch or automatically generating a prompt. I'd highly recommend the automatic prompt generation feature. If you clearly explain to the model what you hope to achieve with your prompt, what inputs you expect to give, and what output you would like, then it's likely to write you a helpful first draft of a prompt.

The Workbench:

The first prompt you have either written or generated can now be improved in the 'workbench' itself by either directly typing the changes you would like or by using the automatic prompt-improvement feature which can help you iterate through a cycle of testing and improvement until you get the full functionality you expect from your prompt. Here are a few key features of the workbench that may be useful to know about:

1. Model Settings: in the top right of the Workbench you can find the model settings button. This allows you to specify which model you want to use (they each have their strengths and weaknesses- more can be read about them in the 'Docs' which I highly recommend everyone read in detail!). In model settings, you can also adjust the 'temperature' which essentially means how creative/independently minded the model will be when writing its response. For most of our use cases, a highly deterministic model with temperature set to 0 is best. Here you can also specify the ideal length of Claude's response, which can be useful if it's cutting off its responses too soon.

2. Variables: here you can input an example of the data that the model will receive, according to all of the variables defined in your prompt. This means when you click 'run' the model will have some example data to run the prompt on.

3. Get Code: this is super handy if you're building a script and need to input your prompt in the relevant coding language (see below on APIs). Just copy this code and input it into your script as the prompt to be sent to the model.

On the left of the screen in the Workbench you will see your prompt. Here is some explanation of the different available features here:

1. System prompt: it is often a good idea to assign a role to the model when you are prompting it, such as an expert in your subject area, as this helps condition the generated output to use the appropriate terminology. Any role assignment should be short and clear and should go in the 'system prompt' section. Note that there are cases in which assigning the role of an expert in your domain can cause problems. For instance, an expert in history might read in your data the word 'slave' and reproduce it in the more academically correct form 'enslaved', which could be problematic if your task is accurately extracting original language from the sources. Try assigning roles like, 'you are a faithful, diligent, and accurate transcriber' or something like that to better suit your task.

2. Examples: offering one or two clear examples of ideal input-output can really help Claude remain true to your prompt instructions, especially in terms of producing the correct output format, CSV for instance. Examples should also explore edge cases so that Claude knows how to respond if the task can only partially be completed (for instance there is data missing for one of the fields you have requested).

3. Tools: this is a much more advanced functionality that transforms Claude into an agent which can interact with digital tools that you've built, independently deciding which tools to activate, and using the returned data to generate the final response. A classic tool example might be a calculator, if you need accurate calculations in its response (LLMs are notoriously bad at maths on their own).

4. User: this is where the main part of your prompt should be, with all variables and instructions clearly defined. Careful formatting is your best friend in prompt writing, for clearer 'chain of thought' instructions.

APIs:

For those with a little more programming experience, or those willing to try it out and realise just how easy it is, the API keys for API access to Anthropic can be found in the 'Dashboard' area. An API key is a bit like a password that can grant you access to Anthropic remotely, from a program you have written, for instance. It can be really handy to write a simple program for your work with Anthropic, this will enable you to do things like batch-processing your data instead of manually copying and pasting inputs into the Workbench. As with everything I'd recommend asking either ChatGPT or (even better for coding tasks I find) Anthropic's Claude Sonnet to help you write and run a script.

I would suggest that you load any data you may have into your Google Drive and then open a new 'colab' file in your Google Drive. Colab offers a great free to the user cloud-based coding environment to test out and run programs.

Here's an example of a prompt you could give to Claude to get started with a simple script to work with an API. My use of double brackets indicates where you should adapt this prompt to your needs:

"Hi Claude, today you are a helpful and knowledgeable coding coach. Please help me write a simple script to automate the processing of my data via API access to Anthropic. The script should be written in Python and will be executed in Colab. The script should do the following:

1. Mount colab to my Google Drive where the data is stored

2. Install all necessary dependencies for this script to run in colab

3. Open a dialogue box where I can input my Anthropic API key

4. Take in the path to ((a file or folder where your data is stored, specify the file type, csv, docx, pdf, etc))

5. Send the data in batches of ((either single file, or paragraph, or whatever you think is appropriate)) to Anthropic for processing using the prompt below, and the Haiku 3.5 model ((or change model according to your needs)).

6. All responses from the Anthropic model should be concatenated and written to a file entitled 'Anthropic responses' in my Google Drive.

7. Here is the prompt that the model should run for each batch of data:

((here paste the prompt that you have built and designed in Anthropic Workbench))."

As default, Claude should describe step by step how you go about running the code. If you encounter any errors simply paste them into your chat with Claude or describe the problems you encounter and it will work with you to resolve them. This is just a starting point to get you up and running with API access for the first time. APIs allow you to do very exciting and sophisticated things, for instance by making API calls in one program to multiple tools or databases (such as Pinecone for vector data storage, Voyage AI for embedding models, or indeed archives). Gaining confidence in working with APIs is a real superpower in this information age. Best of luck!