Writing an appropriate prompt is a skill. Many (most?) people using LLMs aren’t using prompts that give them what they actually want, in many cases because they don’t understand the thing they’re trying to ask for, but also in many cases because they don’t know how to ask for it while avoiding the many possible pitfalls that come with uncareful prompts (including, but not limited to, hallucination and bias).
Personally I haven’t bothered learning that skill yet, largely because I don’t want that crap shoved in my face all the damned time, and have little use for it anyway. But it’s still a skill.
It’s really not for most people. You ask a question, the LLM answers. You ask it to do a task, it does it. Most people are not using these things for anything advanced or technically difficult.
I work in a big corporate office, if you exclude the IT staff and software devs, here’s a pretty exhaustive list of what everybody else is using LLMs for:
Organizing emails.
Drafting memos and auto-responses in Teams.
Organizing cluttered files.
Adding “artwork” to company fliers.
Doing web searches.
Summarizing emails and Teams messages and creating to-do lists from them.
Searching large documents for key terms.
It’s grunt work, that’s what these things excel at, and that’s how most people use them. That’s fine, but acting like most people are somehow not able to unlock the “true potential” of these LLMs unless they have special training and skills is a marketing ploy to sell overpriced “AI” courses to gullible and ignorant executives and upper managers.
It’s like those stupid DEI courses that corporations have everybody take so they can “learn to not be problematic.” But all the quizzes are filled with stupid stuff like:
Your manager, Sarah finishes briefing you and your coworker, Phil on an upcoming project. After she leaves, Phil says to you, “That Sarah sure is a real juicy piece of meat, I’d love to get taste of her!” What should you do? A. High-five Phil and tell him you call dibs on her secretary.
B. Tell Phil that his comment was not appropriate, then report it to HR.
It’s just there to check boxes for compliance and provide an excuse for other companies to charge outrageous prices to tell people that sexually assaulting your coworkers or pulling your eyelids tight and calling yourself “Ching Chong” isn’t appropriate office behavior.
It’s another way Capitalism poisons and hollows out substantive discussions about topics like systemic racism, multiculturalism, and gender relationships, in favor of standardized quizzes that cost thousands of dollars a year to administer.
The amount of “AI consultants” and “AI integration” firms that have popped up overnight is staggering. Just like with the crypto/blockchain craze a few years ago, everybody is trying to get rich quick off the hype.
Good point. I probably should have stated “using LLMs for technical tasks that anyone cares about the results of”.
I’m not going to claim LLMs are trustworthy, but I am saying that there are ways to ask things that either reduce the odds of it giving you false things or actively cause it to provide wrong answers with misstated or hallucinated context. People who know what they’re doing can get better results out of them. This doesn’t stop them from being the equivalent of a nepo-hire intern, but it could be said to change whether they’re a malicious, apathetic, or semi-eager nepo-hire intern.
And one of the big risks with LLMs is staff who can recognise bullshit or questionable results being replaced with staff who unquestioningly accept whatever results they get. And people reading an “AI summary” of something and assuming it’s actually accurate.
People who know what they’re doing can get better results out of them.
I don’t think that’s true in the way you appear to mean (sorry if I misunderstood). People who are already experts on the subject matter might be able to use better keywords and discard hallucinatory material quicker, but I don’t believe that you can generically “be good at prompting” outside of an area where you have substantial domain knowledge.
And the only way to learn to recognise bullshit involves not using LLMs or other automated tools and working problems out for yourself, not to mention that they’re an “intern” who actually costs the same in computing power as two senior engineers’ salary.
I don’t think that’s true in the way you appear to mean (sorry if I misunderstood). People who are already experts on the subject matter might be able to use better keywords and discard hallucinatory material quicker, but I don’t believe that you can generically “be good at prompting” outside of an area where you have substantial domain knowledge.
There are ways of phrasing LLM requests that can actively induce them to give you made-up bullshit that confirms your pre-existing assumptions. Which is fine if that’s what you want it to produce, but often not what people actually want. For (possibly poor, given I don’t actually use LLMs) example “give me the transcripts for five court cases for assault with a deadly fish” would likely result in five-ish almost-certainly made-up court transcripts. Someone who knows what they’re doing could phrase the prompt to be more likely to say that no such cases exist (assuming no such cases actually exist) instead of just hallucinating them as per the request.
And you might be right about domain knowledge being required for good prompting. But it’s possible to have domain knowledge and still be terrible at asking for what you actually want it to produce.
not to mention that they’re an “intern” who actually costs the same in computing power as two senior engineers’ salary.
Don’t expect me to defend the moronic economic decisions of companies that have jumped on this stupid band-wagon without regard for the actual value (or lack thereof) to their business.
Writing an appropriate prompt is a skill. Many (most?) people using LLMs aren’t using prompts that give them what they actually want, in many cases because they don’t understand the thing they’re trying to ask for, but also in many cases because they don’t know how to ask for it while avoiding the many possible pitfalls that come with uncareful prompts (including, but not limited to, hallucination and bias).
Personally I haven’t bothered learning that skill yet, largely because I don’t want that crap shoved in my face all the damned time, and have little use for it anyway. But it’s still a skill.
It’s really not for most people. You ask a question, the LLM answers. You ask it to do a task, it does it. Most people are not using these things for anything advanced or technically difficult.
I work in a big corporate office, if you exclude the IT staff and software devs, here’s a pretty exhaustive list of what everybody else is using LLMs for:
It’s grunt work, that’s what these things excel at, and that’s how most people use them. That’s fine, but acting like most people are somehow not able to unlock the “true potential” of these LLMs unless they have special training and skills is a marketing ploy to sell overpriced “AI” courses to gullible and ignorant executives and upper managers.
It’s like those stupid DEI courses that corporations have everybody take so they can “learn to not be problematic.” But all the quizzes are filled with stupid stuff like:
Your manager, Sarah finishes briefing you and your coworker, Phil on an upcoming project. After she leaves, Phil says to you, “That Sarah sure is a real juicy piece of meat, I’d love to get taste of her!” What should you do? A. High-five Phil and tell him you call dibs on her secretary. B. Tell Phil that his comment was not appropriate, then report it to HR.
It’s just there to check boxes for compliance and provide an excuse for other companies to charge outrageous prices to tell people that sexually assaulting your coworkers or pulling your eyelids tight and calling yourself “Ching Chong” isn’t appropriate office behavior.
It’s another way Capitalism poisons and hollows out substantive discussions about topics like systemic racism, multiculturalism, and gender relationships, in favor of standardized quizzes that cost thousands of dollars a year to administer.
The amount of “AI consultants” and “AI integration” firms that have popped up overnight is staggering. Just like with the crypto/blockchain craze a few years ago, everybody is trying to get rich quick off the hype.
Good point. I probably should have stated “using LLMs for technical tasks that anyone cares about the results of”.
I’m not going to claim LLMs are trustworthy, but I am saying that there are ways to ask things that either reduce the odds of it giving you false things or actively cause it to provide wrong answers with misstated or hallucinated context. People who know what they’re doing can get better results out of them. This doesn’t stop them from being the equivalent of a nepo-hire intern, but it could be said to change whether they’re a malicious, apathetic, or semi-eager nepo-hire intern.
And one of the big risks with LLMs is staff who can recognise bullshit or questionable results being replaced with staff who unquestioningly accept whatever results they get. And people reading an “AI summary” of something and assuming it’s actually accurate.
I don’t think that’s true in the way you appear to mean (sorry if I misunderstood). People who are already experts on the subject matter might be able to use better keywords and discard hallucinatory material quicker, but I don’t believe that you can generically “be good at prompting” outside of an area where you have substantial domain knowledge.
And the only way to learn to recognise bullshit involves not using LLMs or other automated tools and working problems out for yourself, not to mention that they’re an “intern” who actually costs the same in computing power as two senior engineers’ salary.
There are ways of phrasing LLM requests that can actively induce them to give you made-up bullshit that confirms your pre-existing assumptions. Which is fine if that’s what you want it to produce, but often not what people actually want. For (possibly poor, given I don’t actually use LLMs) example “give me the transcripts for five court cases for assault with a deadly fish” would likely result in five-ish almost-certainly made-up court transcripts. Someone who knows what they’re doing could phrase the prompt to be more likely to say that no such cases exist (assuming no such cases actually exist) instead of just hallucinating them as per the request.
And you might be right about domain knowledge being required for good prompting. But it’s possible to have domain knowledge and still be terrible at asking for what you actually want it to produce.
Don’t expect me to defend the moronic economic decisions of companies that have jumped on this stupid band-wagon without regard for the actual value (or lack thereof) to their business.