New Research Reveals the Art and Science of AI Prompting

A new report sheds light on the complex world of AI prompting, offering valuable insights for anyone looking to improve their interactions with large language models.

The Prompt Report, a comprehensive 76-page study from AI education company Learn Prompting and researchers at OpenAI, Microsoft, Stanford, and other leading institutions, surveyed 1,500 papers on prompting and examined over 200 different prompting techniques.

While the report doesn't offer a silver bullet for perfect prompting, it does provide crucial insights that could help you level up your AI game. 

What are the most important takeaways from the report?

I got the scoop from Marketing AI Institute founder and CEO Paul Roetzer on Episode 106 of The Artificial Intelligence Show podcast.

What types of prompting work best?

The research highlighted two prompting techniques that tend to perform well:

1. Few-shot prompting: Providing at least a few examples when you prompt.
2. Chain of thought reasoning: Encouraging the model to show its reasoning step by step.

These are by no means the only effective prompting methods (the report looks at dozens of them). But they are ones that are commonly used and seem to perform well.

However, the researchers emphasize that performance can vary significantly depending on the task and the specific model being used.

6 tips for better few-shot prompting

In an interview, Learn Prompting CEO Sander Schulhoff broke down six pieces of advice contained within the report that you can use to improve your few-shot prompting results:

1. More is often better: Including more well-selected examples typically improves performance.
2. Randomize example order: This can reduce the model's tendency to bias towards certain examples.
3. Balance your examples: Ensure a balanced representation of different outcomes you're trying to achieve.
4. Label correctly: Be as accurate as possible when labeling your examples.
5. Maintain consistency: Use a common, consistent format for your examples.
6. Stay relevant: When possible, use examples that are similar to the specific task you're asking about.

What it means for you

Roetzer says the research emphasizes an important point about AI models:

“The models are weird,” he says. “We still don't really know why they work. Some of the leading researchers continue to be surprised that you can just give these models all this data and they somehow make all these predictions about the next token or word in a sequence and they do it in this really comprehensive and intelligent way.”

That makes prompting part science and part art. 

“We have to realize this is all still pretty new—us humans interacting with these machines and learning how to talk to them.”

And learning how to interact and talk with machines via prompting is, right now, a highly relevant skill that can give any knowledge worker a significant competitive advantage.

Here’s the great news: You don’t have to get a PhD in prompting to learn how to interact and talk with machines better.

“If you take 30-60 minutes to read and understand this report, you’re going to be ahead of 99.9% of the professionals in your field,” says Roetzer.

Beyond the report: resources for better prompting

Roetzer recommends several additional resources for those looking to improve their prompting skills:

• Vendor-specific guides: Many AI companies, including Anthropic, Google, and OpenAI, offer their own prompting guides. These can provide valuable insights into the nuances of each platform.
• Workshops and training: Consider professional training options, like the prompting workshop offered by SmarterX, Marketing AI Institute’s sister company.
• Hands-on experimentation: Build custom GPTs or projects in platforms like ChatGPT or Anthropic to see firsthand how different prompting techniques affect outputs.

The future of prompting

As AI models continue to evolve, the art and science of prompting will likely change with them. However, the insights from this research provide a solid foundation for anyone looking to improve their AI interactions today.

"You have to experiment with these things," Roetzer emphasizes. "You cannot just go in and click the ‘help me write’ button in Gemini or in Copilot and think you're going to get value."

For businesses investing in AI tools, the message is clear: 

Simply providing access isn't enough. To truly unlock the value of these powerful technologies, companies need to invest in education and training that helps employees master the art of prompting.