Opinion · Local AI
Local AI is not a cloud feature
The advertisements are right about one thing: where the model runs now matters. The rest of the pitch usually skips the part that decides whether it helps you.
“Local AI” has become a marketing phrase. A laptop maker says it. A router says it. A subscription service says it while sending every prompt to a data center. The words are doing a lot of work, and most of them are not about your computer.
A useful definition is narrower. Local means the model runs on hardware you control, with weights you chose, and the request does not have to leave that machine to get an answer. That is a real change, but it is only one part of a working setup. The rest is where your project lives, who holds the keys, and what still works when the connection drops.
What local actually changes
Four things change when inference moves onto your machine. They are easy to test and hard to fake.
- The request path. A prompt handled by a local server does not need a round trip to a provider. That is not the same as zero network use: the runner may still check for updates, and your editor may still use other services.
- The file path. A cloud assistant can read the files you attach or the context you send. A local setup can read the folder directly, which is a smaller boundary for the source itself.
- The credentials. Local inference usually does not need an API key. That removes one thing that can leak, and one account that can be rate limited.
- The failure mode. When the internet is down, a local model can still answer. It will be the model you already downloaded, with the speed your hardware can manage, not a newer one behind a login.
Notice that none of those points promise better answers. They are changes to where the work happens, and that is exactly why they are worth understanding.
The tradeoffs are real
A local model is a file on your disk, not an unlimited service. You need enough memory to hold the model and enough room for the conversation around it. A model that chats well may not call tools reliably, which matters for coding agents that edit files. Larger, more capable models can be too slow on a small laptop.
There is also a maintenance cost. Model files are separate from the app that runs them. Licenses differ. Quantized versions behave differently. A local setup rewards people who are willing to pick a model deliberately and test it on their own work.
None of this means local is worse. It means local is a trade: control and a predictable request path in exchange for hardware limits and your own tuning. For some projects that is a good deal. For others, a cloud model is the honest choice.
Where Lyre fits
Lyre is built around the computer that holds the project. The agents run there. The terminals run there. The app you are building runs there. That makes it a natural place to put a local model, because the model and the files are already on the same machine.
The setup is deliberately ordinary. Run Ollama, LM Studio or llama.cpp, point Lyre at the server, and choose the model in the conversation. Lyre scans for a local server and lets you add one manually when it uses a custom port. The local models guide walks through the whole path, including the memory and context checks that matter for agent work.
The same workspace still connects to the providers you already pay for. That matters more than the label. Some tasks are a good fit for a small model running quietly on your laptop. Others need a larger remote model. Keeping both in one project means you are not choosing an ideology every time you open a file.
A paired phone reaches that same host. It does not download a model or run one itself. It opens the conversation and the live app while your computer does the work. On your own network the connection can be direct, and the phone does not need a Lyre account for local use. Those details are in Connectivity and Continue the same work across devices.
Start smaller than the advertisement
The practical way to evaluate local AI is to give it one honest job. Pick a small model, open a real project, and ask it to summarize a file or explain a function before you let it edit anything. Then try a small change and review the diff.
Compare the result with the model you normally use. Keep notes on speed, memory and whether the model actually used the tools it was given. If it struggles, move down a model size or move the task back to a remote provider. That is not a failure. It is the evaluation working.
The reason to care about local AI is not that it wins every comparison. It is that it makes a different arrangement possible: your files, your hardware, your model, and a request path you can explain. A cloud feature cannot give you that, no matter what the billboard says.