The AI boom has created a new career: being an expert at talking to machines

Pau Martí Felip, 23, graduated with a degree in audiovisual communications. Two years ago, he started working as a video editor in a digital company. Over time, he saw his job change.

“That’s how I became a prompt engineer, which is a creative and technological mix of giving instruction to AI, so that it can give you an answer,” he explains.

The new artificial intelligence applications – used to create images, text and soon video or music – need to receive text requests for what we humans want. That request can range from a simple phrase (”create some 10-minute stretches for people over 70″) to an intricate eight-line instruction that includes details about an image’s colors and background.

“People will need to understand the machine,” says Martí Felip. “It’s like talking to an animal with whom most people don’t know how to communicate. We’re learning that language.”

On LinkedIn, a search for “prompt engineer” and “Spain” yields only six profiles and one job offer. The offer is precisely from Martí Felip’s company – Raona – which is dedicated to digital transformation in companies.

This unique job offer for “prompt engineers” in Spain is proof of its exploratory nature. Back in February, an offer for a prompt engineer from a large startup in the US (with an enviable salary) went viral.

The job offer is to help Martí Felip. He’s well-aware that he’s searching for candidates in deserted land.

“At the moment, most resumes don’t have AI skills. If you ask four questions about ControlNet or something more advanced, you’ll realize this,” he sighs. But he’s willing to settle:

“Someone who wants to learn and is very creative is worth it.”

Although it may seem striking, Raona is not looking for more computer scientists at the moment. “We’re full of programmers… what we need is more creative people.”

The appearance of the job of prompt engineer has sparked a debate about the profession’s feasibility and future. Even the most skeptical experts admit two things: it’s impossible to foresee anything in the AI field today and, in the short-term (two or three years), such work will be necessary.

“The job offers for prompt engineers seem temporary to me,” says Javi López, an investor in AI startups. “During a transition period, many specific people will need to be trained. But, in two or three years, [the job] will be trivial and the concept of prompt, as such, won’t exist. Even a five-year-old child will be able to ask a machine for things out loud,” he assures EL PAÍS.

But López himself has, so far, obtained more than $27,000 in funding from 612 patrons towards an illustrated novel (made with AI) that will include a guide on how to get started and the descriptions (prompts) that he has used. That figure multiplies López’s initial forecast by almost five; there’s certainly interest in exploring this world. Perhaps one day the requests will be simple… but for now, López’s examples include bits like this string of code: “Ultra-realistic, low angle, on a wooden table in a beautiful kitchen, Canon 5D, DSLR, portrait of 50mm, DOF, V-ray rendering, 8k, ray tracing, Goldenhour lighting, uplight, hard edge lighting.”

López’s skepticism comes from his belief that professionals in each field will know how to handle intelligent machines. A new “engineer” won’t be needed.

“There were also Excel experts once… [but] now all accountants are expected to know the tools of their trade. In the end, what will remain is not a prompt engineer per say, but rather someone who was a designer or artist before. The barriers to entry are going to be lowered: it will no longer take so much technique (drawing, lighting, photography), but rather, direction and selection,” he opines.

Pablo Moreno-Muñoz – a researcher at the Technical University of Denmark – also believes that the models will tend to be simplified over time, at least in three ways: “One, training the tools with more data (images and text); two, hours of engineering to build models with greater capacity (number of parameters, size of neural networks); and three, training time and money spent on supercomputers, where AI will find more and more relationships between data, which will then allow it to generate better results from prompts.”

The key distinction in this debate boils down to a simple question: will AI be able to understand what we mean – like Google search does today – or will the complex needs of a future video, image, tune or story still require specialists?

Perhaps a simple use of these tools will coexist with something more sophisticated. Today, you can search on Google, or be an SEO expert… or you can be an expert in Photoshop, or simply apply a filter.

Simon Willison – a programmer – is an advocate of prompt engineering. He believes that prompt engineers will require – in addition to strong communication skills – something from a variety of disciplines to understand the deep learning that drives these models: linguistics, psychology, art history, computer security and philosophy.

“How can we teach a language model the difference between truth and fiction? What is, in fact, the truth?” Willison wonders.

Silviu Pitis – a researcher at the University of Toronto – also sees a clear future for this profession: “As the models become stronger, we will still need humans to interact with them [and] teach them to communicate.”

Martí Felip’s specific task is twofold: helping engineers to test the model they’re programming with – for example, ChatGPT – and then using it to obtain optimal results, or helping the client take advantage of it.

Training is another key issue. Martí Felip missed out on having learned some coding skills at university. But most of his training is a result of being self-taught.

“I’ve always been interested in AI. I’ve been learning with tutorials on YouTube, Twitter and TikTok. In college, we should have done a bit more coding. I know programming basics, but I missed out on some more advanced Python. Although it’s [useful] to have the creative part for the design element,” he explains.

Jessica Gutiérrez – an administrative assistant from Gijón – is one of the six people in Spain who have added “prompt engineer” to their LinkedIn bio. Aside from her day job, she spends time writing for websites. The next step was logical.

“If you’re not learning to generate text, your copy work is going to become obsolete,” she notes. “I realized that – four months after I started [looking into this] – in the end, you find an ally in artificial intelligence… although I’ve come a long way to be able to position myself [in the field] as early as possible.”

Her training also took place on the internet. “I’m self-taught, through trial and error, based on watching videos on YouTube, or on Twitch,” she adds. Now, she sees a lot of new possibilities in the future.

“There’s a lot of work needed to lay the foundations for job profiles. Participating in this environment is very useful to explore business and educational possibilities,” she says – although those around her don’t see it that way.

“People laugh. They don’t think this [career] has a future. And they don’t think the tool will generate everything I ask for. Between their laughs, I made this book,” Gutiérrez jokes, referring to the fact that she has prepared a recipe book: all the text and illustrations were created with AI.

“I got the recipes in one afternoon. It took me longer with the illustrations,” she details. If everything works out for her, she’ll produce a children’s version.

One of the wonders of this new kind of engineering is that knowing code is secondary… a detail that lowers the barriers to entry. Andrej Karpathy – a respected programmer who led Tesla’s AI department before rejoining OpenAI – once tweeted: “The most fashionable programming language is English.”

It’s possible, says Ignacio Peis, a researcher at the Carlos III University in Madrid, “to classify English as a new programming language, since a few lines of text generate programming code routines. You might think that this is a new level of abstraction. However, the programming relationships between the spoken language and the generated code are not defined, they are not universal.”

“The same text entry can generate different codes, since we’re talking about probabilistic models,” he clarifies.

English, therefore, doesn’t always work as a programming language. But that doesn’t prevent it from relating to the machine. “In a certain way, it is programming,” sums up Pitis.

“As a requirement for strong AI, computers must be able to communicate with humans through natural language. If that interface becomes strong enough, we can teach them by talking to them normally,” he adds.

Although English is a possible programming language, machines can beat humans at that as well. A recent study has tested this, with automated requests trying to improve a human request. In certain questions, the AI succeeds.

In the study, the authors note that “we’re going to solve this step-by-step, to make sure that we have the correct answer.”

The race to achieve the perfect request has only just begun, says Gutiérrez, who frequently conducts his own tests.

“You have to be very sure of the prompt. I write it down two or three times… if I don’t have all the details, I keep adding. I won’t ask the machine until it’s perfect.”

Sign up for our weekly newsletter to get more English-language news coverage from EL PAÍS USA Edition