Welcome to issue #3. I said weekly last time and here we are, one week later. Small victories.
This issue has a theme, and it's money. I guess I am not the only one who thinks that the generous Claude and Codex subscriptions people are enjoying right now are just to get you addicted to using AI.
And it's quite evident, isn't it? A big credit giveaway comes first. Then the models quietly get dumber. Then your credits start burning faster than they used to. Then the price goes up or the offer just ends.
If you've rebuilt your workflow, or worse, your company, on top of this "generosity", you may find yourself in a tough spot when the token price starts going up. The $20/$200,$2000 a month pricing you're planning today, won't give you the same thing in long run.
Which is exactly why the local AI is going to go big. Not every task needs a frontier model. A lot of them need a small model, running on hardware you already own, wrapped in a harness tuned to your work.
Before we get into it, thanks to Monid for supporting this edition.
Think of Monid as OpenRouter, but for agent tools. One base URL, one key, and an agent can reach 2,000+ tools across 72+ providers. The connector layer went open source under MIT, and because connectors are declarative, adding your own API is a pull request a coding agent can write for you.
🧪 On my bench: Buzz
I promised you an Unsloth verdict last issue and I owe you that. But I got distracted by Buzz, an open source communication tool built for agents and humans to work together. I actually liked Buzz. There is smartphone app too and thus staying connected in not an issue. It could replace Discord for team It's FOSS.
📥 Agents are getting an inbox
Two biggies independently landed on the same idea at alsmot the same time, and it's a good one. Chat is a bad interface for an agent that works while you're asleep.
Cloudflare open sourced agentic-inbox, a self-hosted email client with an AI agent built in, running entirely on Workers (so it is cloud, not your GPU). Incoming mail arrives through Email Routing, each mailbox lives in its own Durable Object with a SQLite database, and attachments go to R2. The agent reads your inbox, searches conversations, and drafts replies that you approve before they send.
AWS did something similar with Pizza Bot. It's an Apache 2.0, local-first inbox for long-running agents, built on DeepAgents and LangGraph. Finished work shows up as unread threads and anything needing your approval gets flagged.
Unlike agentic-inbox, this one runs on your machine with no telemetry, storing threads, checkpoints, memories and logs as SQLite files in a local folder. You pick the model provider, and Ollama is on that list. So you can run the whole thing locally, even offline.
🔍 Discover AI tools
Two finds this week, and they sit at opposite ends of the "what is AI for" question.
First, OpenPencil, an MIT-licensed, AI-native design editor that opens and writes native Figma .fig files. You can copy nodes between it and Figma. It ships a headless CLI, an MCP server so coding agents can read and edit your designs, and a roughly 7 MB Tauri desktop app that needs no account. Built on Vue 3, Skia for rendering and Yoga for layout. Local model support is on the roadmap.
open-pencil
The second AI tool is kind of anti-AI tool. Sounds weird, I know but hear me out. AI-based auto transcribers are part of almost all meeting tools. Not everyone wants to be heard by AI, specially that sends data to server for further training, probably.
Kalypta runs a small model locally on your device and reshapes your audio in real time so that AI transcribers can't make sense of what you said. The goal is to be inaudible to the note-taking bot in your meeting while staying perfectly clear to the humans. Interesting project to watch out for.
📡 Open Model News
The big one is Qwen-Image-2.1. It's a small text-to-image model and some people are already calling it the Nano Banana alternative. Not available on Ollama yet probably because weights ship under the Qwen Research License, which is non-commercial and requires you to email Qwen for a commercial licence. Another reminder that open weights and open source are not the same thing.
The other release worth your time is small in a different way. You've probably seen Jev everywhere for the past week, TypeSafe AI's "System One" model that doesn't generate text at all. The open source alternative to Jev is laya-mlx, an Apache 2.0 native MLX port of Laya, the typed decision model from Convai Innovations. Available only for Apple Silicon model for now.
👀 Big Tech Watch
Microsoft rewrote the GitHub Copilot agent runtime from TypeScript into more than 800,000 lines of production Rust, and one engineer drove most of it with a fleet of AI agents. It took 128 pull requests over about fourteen and a half weeks, shipped incrementally.
The whole migration cost around $120,000 in tokens. That's 136.3 billion tokens, of which 130.6 billion were cached input reads.
Think about what that means for the rest of us. That project only worked because 96% of the tokens were cache hits billed at roughly a tenth of the normal rate. Remove the caching and the whole thing will be far from a profitable migration. This is the token economics I was talking about earlier.
🗂 AI Jargon: Prompt caching
Since caching reduced the migration bill for Microsoft, let's learn more on it.
Here's the thing. Every time you send a request to a model, it has to read your entire prompt and build an internal representation of it before generating a single token. In a long agent session, that prompt is mostly the same every turn: the system prompt, the tool definitions, then the conversation so far, with a bit of new text on the end.
Prompt caching means the provider saves the intermediate computation for a prefix it has already processed. Send the same prefix again and it reuses that work instead of redoing it. Providers typically bill a cache hit at around a 90% discount, so a million input tokens might cost $2.00 fresh and $0.20 cached.
This is why agent harnesses are built to save on tokens. They keep a long, stable prefix and only ever append to it, because anything that changes the beginning of your prompt invalidates the cache and multiplies your bill by ten.
How does it impact local inference? Well, on your own hardware there's no per-token bill, so the equivalent optimisation is KV cache reuse. So you win on latency instead of money.
⚡ Quick Tip: Check whether Ollama is actually using your GPU
Last issue I told you how to keep models warm. Here's a similar check.
Run ollama ps while a model is loaded. Alongside the model name and size you'll see a PROCESSOR column telling you whether it's on GPU, CPU, or split between the two with a percentage.
If a model doesn't fit in VRAM, Ollama will quietly offload some layers to system RAM and keep working rather than failing. It runs; you get answers, but it will sluggish.
If you see anything other than 100% GPU and you expected it to fit, drop to a smaller quantisation or shorten your context window. A Q4 model fully on the GPU will usually beat a Q8 model that's splitting between CPU and GPU.
In the end...
I am not doomsaying, but I would like you to plan for the future beyond the cheap phase of frontier AI. Build your harness and use local models in your workflow wherever it makes sense. I understand that hardware is not cheap these days but there are smaller models that can fit niche scenarios.
I welcome your feedback and I'll see you next week (because it's a weekly newsletter)..
If this helped you, consider supporting It's FOSS
It's FOSS has been helping people use Linux for the past 14 years. Help us stay independent from big tech. Become a Plus member, enjoy ad-free reading and get 5 eBooks.
