About     Design and AI     Writings     SxD Blog

Prompts and Prompting

The art and science of prompts and prompting is known as prompt engineering. Given how much language skill is involved, however, it might better be characterized as prompt design. The prompts used to obtain responses from generative language models are a combination of natural language requests and instructions, and topically-specific references. The more users understand about how best to prompt AI, the more they will get out of them. But the more that designers can furnish behind-the-scenes instructions to language models, the more AI will be able to do with what users prompt them to do.

As generative AI becomes more sophisticated and is embedded into more software tools and systems, designers use meta prompts to instruct models for the purposes of function calls, issuing instructions, articulating behaviors, forcing reasoning steps, calling agents, and more. Prompting is a hybrid of algorithmic instructions, code, lexically-specific terminology, and natural language. Prompt engineering has not been claimed by the user experience design community, but there is a lot of user experience to prompting. Design research could be conducted on prompting best practices, techniques, tips and tricks, and outcomes. In professional use cases, prompts will need to be captured, documented, iterated on, replicated, and more for traceability purposes. The more valuable and useful the output of an AI, the more the prompt could be given credit or held to account.

Multi-agent systems combine separate models, or use a single model in multi-agent roles, to employ a variety of techniques on generative tasks. These can range from conversational question answer to recommender systems, task management, tool use, planning, coding, and more. These AI architectures use prompts internally to pass context and responses from one agent to the next, in cases subjecting responses to reasoning and reflection for factual accuracy and verification. The success and failure of these systems can come down to the prompting used; as can the cost of additional search and reasoning. Prompting is the imperfect science at the heart of many of these architectures. As models evolve, prompting too evolves. There can be a lot more of trial and error in prompting than should be the case.