I remember the first prompt I sent to the GPT-3 API. I asked it to explain how photosynthesis works as if explaining to a ten-year-old. The response came back in seconds and it was good. Clear, accurate, age-appropriate, with a nice analogy about leaves being tiny factories. I sent another prompt. Then another. An hour passed.
The experience was qualitatively different from anything I had used before. GPT-2 had been impressive but limited. GPT-3 was 175 billion parameters, more than 100 times larger, and the improvement was not linear. The model could follow complex instructions, write code, translate languages, answer questions, generate structured data from prose descriptions, and summarise long texts. All from the same model, with no fine-tuning.
What struck me most was the few-shot learning. You could show the model two or three examples of a pattern and it would generalise correctly. Give it three examples of converting a customer complaint to a structured JSON object and it would convert any customer complaint to JSON. This was not programming in any conventional sense but it produced real, useful output.
The implications took time to process. Any task that involved understanding and generating text could potentially be automated or augmented by this technology. Code generation, document summarisation, data extraction, translation, customer service, content creation. The list was long.
The practical limitations were real. GPT-3 made confident mistakes. It would invent facts, misremember details, and state falsehoods with the same confident tone as correct information. The hallucination problem that had been present in GPT-2 was still there. For any application where accuracy was critical, human review was necessary.
The cost was also significant. API access was priced per token and generating lots of text with GPT-3 was expensive enough that many applications were not economically viable. The economics of LLM applications were not yet right for most use cases.
But the directional change was unmistakable. The question was no longer whether AI could produce useful text output. It clearly could. The questions were now about cost, reliability, deployment, and the appropriate human-AI interface for different tasks. Those were engineering problems, not research problems, and that shift mattered.