LLMs Aren't Just Spooky
They're Liquid
Halloween seems like an appropriate time to write about LLMs. Nearly everyone who works with LLMs knows that they're a bit spooky.
The language around generative AI also betrays this strangeness: perplexity, hallucination, context, alignment. Nothing that you need to worry about when firing up Python to wrangle some nice tabular data and output some graphs.
The capabilities of frontier LLMs are at times absolutely astonishing. Who would have thought 3 years ago that by now you could ask an app on your phone to look at a picture of a math problem on a whiteboard, transcribe it, and solve it using Python. Back then, I couldn't even get Siri to reliably play the next song [1].
The mobile app for OpenAI uses your phone's haptic feature to make it feel as if a ghostly invisible hand is typing on your keyboard very, very quickly when its models return output. And the model improvements and capability changes from release to release are equally if not more spooky than the typing of the invisible hand.
Classical computer programs are hardened, have clear if-then logical paths, and computers execute code exactly as written. They resemble trains that follow tracks that switches are manipulated to guide them onto. That's the point.
Programming is creating a mathematical object, where an input yields an output following prescribed, fixed steps. If you put in one wrong period or write FRON instead of FROM in single line of your program or query, it all goes out the window.
If this kind of work resembles soldering a solid motherboard or building a solid railroad (or Rube Goldberg machine). LLMs are inescapably fluid. They can adapt to a huge variety of inputs, and their outputs can take an incredible variety of shapes. You can make misspellings in your prompts. You can ask them to prompt themselves. You can ask them to answer in the form of a picture, or rewrite the code (or text) they just generated in a totally different language. Fluid doesn't mean infallible, but it does mean flexible.
This liquidity has major implications for systems that can incorporate the best of generative AI and traditional engineering approaches, that can harness both fluidity and reliability and explain-ability. Their fluidity, and how it relates to the lurking strangeness behind classical programming is what motivates the first edition of this very infrequent newsletter.
[1] Actually, I still can't.