I tried running AI chatbots locally on my laptop-and they suck

Just because you can doesn't mean you should. There are still too many obstacles to overcome.

- BY MATTHEW S. SMITH

When DeepSeek-R1 released back in January, it was incredibly hyped up. This reasoning model could be distilled down to work with smaller large language models (LLMs) on consumer-grade laptops (fave.co/3qL4oHf). If you believed the headlines, you'd think it's now possible to run Al models that are competitive with ChatGPT right on your toaster.

That just isn’t true, though. I tried running LLMs locally on a typical Windows laptop and the whole experience still kinda sucks. There remain a handful of problems that keep rearing their heads.

PROBLEM 1: SMALL LLMS ARE STUPID

Newer open LLMs often brag about big benchmark improvements, and that was certainly the case with DeepSeek-R1, which came close to OpenAl's ol in some benchmarks.

But the model you run on your Windows laptop isn't the same one that's scoring high marks. It's a much smaller, more condensed model—and smaller versions of large language models aren't very smart.

Just look at what happened when I asked DeepSeek-R1-Llama-8B how the chicken crossed the road: see right.

This simple question—and the LLM’s rambling answer—shows how smaller models can easily go off the rails. They frequently fail to notice context or pick up on nuances that should seem obvious.

In fact, recent research suggests that less intelligent large language models with reasoning capabilities are prone to such faults. I recently wrote about the issue of overthinking in Al reasoning models (fave.co/3GeScYS) and how they lead to increased computational costs.