mlx-community/ThinkingCap-Qwen3.6-27B-OptiQ-4bit · Hugging Face - I need to play around with this model, it promises better reasoning with fewer tokens and delivers good performance on my MacBook Pro - if it holds up, it could be quite interesting for local Agents.
hunk — review-first terminal diff viewer is a diff viewer designed to work specifically with Herdr: one terminal for the whole herd. An agent can generate diffs and integrate comments into them via a skill, which can then be displayed in Hunk. The user can also comment directly in the diff, enabling a classic review process that runs entirely locally within the terminal.
Herdr: one terminal for the whole herd - ich bin ja alter screen User, aber ich glaube es wird wirklich Zeit, dass ich mich mal mental modernisiere. Herdr ist screen auf steroiden. Es hat eine Client-Server Architektur, remote-Installation und Nutzung, und eine sehr gute Agent-Integration. Agent Sessions tauchen in der Übersicht auf, genauso wie Projektstatus (Branch etc.), alles schön in einem Sidebar. Tabs können geöffnet werden, Screens gesplittet werden und obwohl es eine Terminal-Anwendung ist, unterstützt es die Maus. Fühlt sich wirklich gut in der Benutzung an. Ich benutze im Moment gerne Supacode für meine Terminal-Sessions, das hat vields davon schon eingebaut in einer Mac Anwendung, aber natürlich nur auf dem Mac. Herdr funktioniert auch problemlos auf einem remote Server über ssh.
JustVugg/colibri: Run GLM-5.2 (744B MoE) on a 25GB-RAM consumer machine — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦 - one of the wildest projects recently, because it can run a frontier model on completely unsuitable hardware. With abysmal tokens/second, but it runs. And there are already Metal and CUDA backends, so there's already some acceleration.
567-labs/instructor: structured outputs for llms - and here the alternative to dottxt-ai/outlines: Structured Outputs, but it operates behind the LLM rather than during generation. Essentially, this is the classic feedback loop for LLM responses that you constantly build: parse, validate, report specific errors, adjust the prompt, and repeat until it works.
dottxt-ai/outlines: Structured Outputs - a highly promising tooling that integrates directly into vLLM, validates LLM responses against predefined schemas, and directly rejects invalid tokens during generation, ensuring the output strictly adheres to the expected formats. Particularly useful for smaller models that often struggle with structured output formats. It can be highly beneficial for strict workflows where you know exactly what the LLM is supposed to answer.
datalab-to/marker: Convert PDF to markdown + JSON quickly with high accuracy - an interesting solution for PDF preprocessing that turns PDFs into Markdown while preserving the structure and reading order. This is particularly crucial for document understanding, as the layout of tables carries a significant amount of information, and the tool can help preserve that.
jakobdylanc/llmcord: Make Discord your LLM frontend - Supports any OpenAI compatible API (OpenRouter, Ollama and more) is a Discord bot written in Python that integrates an LLM and makes it available as a chatbot on Discord. It runs as a simple Docker container. Easy to set up and straightforward.
Forest Shuffle is a pleasant tableau-builder with quite cool card combos. The game is relatively easy to learn and play; the only minor drawback is the somewhat limited card pool, which might slightly reduce replayability, but the expansions help to offset this.
LMCache – Building the foundation of AI memory tensor with KV Cache Infrastructure is a KV cache manager that can distribute the cache across GPU memory and SSDs and also helps when a model runs distributed across multiple machines. I primarily use it myself as an offloader for KV cache to disk for Qwen3.6 27B, because it has a slightly higher memory requirement for KV cache than the 35B model.
Supacode is another very nice terminal that I think cmux — The Terminal for Multitasking can replace for me. At first glance, it certainly looks very good, somehow more mature than cmux.
deepreinforce-ai/Ornith-1.0-35B · Hugging Face is a model based on Qwen3.5 35B 8-bit with post-training specifically designed to provide explicit support for coding tasks. It is expected to perform significantly better in coding compared to other 35B MoE models and even outperform Qwen 3.6 35B. Initial experiments I conducted show solid tool-use with few error scenarios, which is promising. I could easily imagine using it as a base model for a Hermes Agent, for example, simply because it is incredibly fast (over 80 tokens per second locally on my M5 Max MacBook Pro).
DietrichGebert/ponytail: Makes your AI agent think like the laziest senior dev in the room. The best code is the code you never wrote. - a very interesting plugin for various coding agents (including Pi and OpenCode, which I use most frequently, but also all major well-known agents from the commercial sector), with which unnecessary code can be identified. It directly targets the typical weaknesses of agent-generated code, which often contains unnecessary abstractions, indirections, and code duplicates that can be represented much more simply. It also catches re-implementations of things already included in the standard library, or unnecessary dependencies on external libraries that can easily be replaced with a few inlines and standard functions.
AbarthJoe/Qwopus3.6-27B-v2-oQ8-mtp · Hugging Face is a very interesting model: Qwen 3.6 27B – one of the best dense models for coding – with fine-tuning based on Opus Thinking Bubbles. The goal is to have a solid foundation for a coding model that can operate with relatively low memory usage. In this version, it is the MLX-converted variant with 8-bit quantization and all weights for native MTP. Even though it is not the absolute fastest runner, it is at least usable and performs quite well in my small tests when analyzing code.
antirez/ds4: DeepSeek 4 Flash and PRO local inference engine for Metal, CUDA and ROCm is a quite interesting project that enables DeepSeek (specifically Q2 quantizations) to run as efficiently as possible on various UMA notebooks. Supported are MacBook, DGX Spark, and ROCm Strix Halo machines. Quite cool if it works—DeepSeek V4 Flash is already quite good as a base model for various purposes, such as a main model for Hermes.
oMLX — LLM inference, optimized for your Mac is a bit like my MLX Server, but with a stronger focus on efficient execution and, as a gimmick, a persisted KV cache so that old prefixes can be revived. I could test it on my upcoming MacBook Pro to see if it’s interesting for me; in particular, the example with Qwen 3.5 122B A10B looks good, as that is currently my favorite model on my DGX.
MTPLX — Twice as fast on MLX bookmarked for later, when my new MacBook Pro arrives. Then I'll have the necessary hardware to run the model, and MTPLX will be extremely important for speed.
Livebook.dev is something like Jupyter Notebooks, but for Elixir. It looks very exciting and comes with many built-in Elixir features and extensions (e.g., Bumblebee for AI applications).
opencode - After the utterly miserable price hikes by various coding agent providers (Microsoft being the latest with massive increases), it is becoming increasingly important to look at alternatives. OpenCode Go is currently a seriously credible offering, featuring a mix of some quite good models that are also open-weight, allowing them to be operated by more than one provider. It has proven useful for my OpenClaw so far.
Dicklesworthstone/pi_agent_rust: High-performance AI coding agent CLI written in Rust with zero unsafe code is a quite interesting project that is a reimplementation of pi.dev in Rust, with a stronger focus on security, but still maintaining Pi's openness — so not thousands of tools and workflows already built into the core, but rather just a minimalist agent.
nearai/ironclaw: IronClaw is an OpenClaw inspired implementation in Rust focused on privacy and security could potentially become interesting at some point. Essentially OpenClaw but in Rust. I ran into a whole series of issues with OpenClaw. OpenAI subscriptions (the $20 per month one) are too small for an agent, the weekly limit fills up too quickly. Switching to cheaper open weight models revealed that OpenClaw really only tests the primary code paths – I ran into all sorts of things that broke that had worked before. On top of that, there have been a whole series of successful supply chain attacks against NPM – clearly NPM's security is more than inadequate. So I deleted the OpenClaw VPS for now. Life is too short for bad software.
stonerl/Thaw: Menu bar manager for macOS 26 - helps against the notch on Apple MacBooks. It's already embarrassing that you need an extra program to do this, and that the system doesn't support it out of the box.
Fission-AI/OpenSpec: Spec-driven development (SDD) for AI coding assistants. is a lighter-weight variant of juxt/allium: A language for sharpening intent alongside implementation.. Could also be quite interesting—the tooling is more developed, but the language is weaker. Allium is much more formal and oriented toward pseudocode.
juxt/allium: A language for sharpening intent alongside implementation. is a very exciting project. Specifically, a specification language for the behavior of software systems. A language for which there is no runtime environment or compiler, except the LLM. Implemented as Agent Skills. Super exciting because it also includes a Distill, with which you can analyze existing software and retroactively build specifications, or work out a specification in an interview process with the AI that is much more precisely understandable to an LLM than general English. A funny detail on the side: I tried Allium with Qwen3-Coder-Next, my current favorite model for local hosting, in pi.dev. I couldn't install the binary for Allium (a syntax checker and linter) with homebrew, so pi.dev simply downloaded the binary and installed it itself.
scitrera/cuda-containers: Scitrera builds of various CUDA containers for version consistency, starting primarily with NVIDIA DGX Spark Containers - I'm currently a big fan of eugr/vllm-node as a base package because it always provides up-to-date versions for vLLM, but if I want to play around with sglang sometime, this is probably the most similar project. I'm particularly interested in EAGLE-3 speculative decoding - basically, tokens are generated in parallel via a very small model and the main model checks what fits and takes it, or generates itself if necessary. This way you can often have a third of the tokens generated via a much faster simple model in the <3B range and only pass every third one through the large model.
thushan/olla: High-performance lightweight proxy and load balancer for LLM infrastructure. Intelligent routing, automatic failover and unified model discovery across local and remote inference backend might be the better choice following the LiteLLM debacle (hacked supply chain with data extractor in the package). For me, it's definitely interesting because I simply want to run two models and make them available under a single endpoint, and all the other packages are significantly overkill for that.
Running Mistral Small 4 119B NVFP4 on NVIDIA DGX Spark (GB10) - DGX Spark / GB10 User Forum / DGX Spark / GB10 - NVIDIA Developer Forums - lifesaver discussion in the NVIDIA forums. With what's in there, I got Mistral Small 4 running smoothly. And it runs cleanly with 150K context and 100 tokens/second in generation. Wow. This is the first time I've really noticed the power of this machine.
Zed: The Fastest AI Code Editor — Zed's Blog is a pretty cool editor that finally comes with a good VIM implementation. It supports Coding Agents and is open to open weight and self-hosted models. To be honest, I really like it and it's genuinely fast. And it's written in Rust, which excites me even more.
Introducing Mistral Small 4 | Mistral AI is another interesting candidate for the ASUS Ascent GX10｜｜ASUS Deutschland, especially since I don't need side-car models for vision there, because the model itself already comes with vision capabilities built-in. And as a MoE model, it should also deliver good speed performance.
nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4 · Hugging Face will likely be the first large model on the ASUS Ascent GX10｜｜ASUS Deutschland because it was trained optimized on NVFP4 - and thus does not experience any "dumbing down" due to quantization, but functions completely normally as expected. And it is optimized for agentic workflows, which should benefit OpenClaw, just as the 1M context, which can probably even be utilized in the model (different architecture than classic transformer-based models).
WireGuard® for Enterprise sounds funny on a purely private blog with that name, but they have pretty affordable plans for VPN-like constructs that make it very easy to reach your local home devices while on the go. There are certainly other alternatives, but I think if I run my agent locally, I'd rather put the subscription money into a proper VPN service instead, where I get more value overall. Update: it's free for private users up to 3 users. Nice.
ASUS Ascent GX10｜｜ASUS Deutschland is arriving in the next few days. AI powerhouse that will allow me to run larger models locally and, for example, operate an OpenClaw Agent autonomously at home without needing any subscriptions. I'm really looking forward to seeing what's possible with it.
Cognee is also something I’ll keep an eye on for later. Basically a knowledge graph controlled by an LLM to make memory available for another LLM. Certainly exciting to play around with when I have good local hardware to run larger models on. But for now, just a memory keeper.
Docker Model Runner Adds vLLM Support on macOS | Docker - at first just noticed, that could be interesting later because I can run models via Docker with vllm, while using Apple Silicon at the same time. Interesting here is that it comes as a Docker image ready to use, and I don't have to fiddle with setup. I'm currently working more with my own rfc1437/MLXServer: a simple MLX based server for small models to run locally simply because I only need it for offline operation, but vllm-metal could be very exciting later.
rfc1437/MLXServer: a simple MLX based server for small models to run locally is a tool that I built (with AI assistance) to run small models directly locally, without heavy overhead. It doesn't consume much memory, has a built-in local chat for personal experiments, and feels significantly more practical to me compared to the big alternatives—fewer knobs to adjust, but consequently less confusion. I just want to run a small model locally for my on-the-road blog.
mlx-community/Qwen3.5-9B-MLX-4bit · Hugging Face is another nice, small model — larger than the others, thus slightly more consistent in execution, but still pretty fast. And that's the upper limit of what you can run on a MacBook Air M4 with 16GB RAM without crashing the computer.
Google/gemma-3-4b-it · Hugging Face is a pretty nice model that has been trained for many European languages and is therefore well suited for local translations – it loads under 4G into memory and occupies approximately 6.5G in interference during operation. And it has Vision Capability, so it can also be used to get image descriptions. Ideal, for example, to be used locally with bDS when you want to be offline on the go. And significantly smaller than mlx-community/gemma-3-12b-it-4bit · Hugging Face – that was borderline on my Macbook Air.
Inferencer | Run and Deeply Control Local AI Models is an interesting tool that allows you to run LLMs locally. Of course, LM Studio or Ollama or vllm-mlx can do this as well. But Inferencer has a feature called "Model streaming" that's pretty cool: it can run models that are actually too large for memory. Of course, you're trading time for memory, but for a local model for image captioning or similar smaller tasks, you could definitely use it. However, I have the feeling that the model becomes somewhat more fragile this way - for example, it suddenly doesn't use tools correctly anymore (I tried it with gemma3 12b, which is just scratching the memory limit of my laptop).
Pagefind | Pagefind — Static low-bandwidth search at scale is a static search engine for statically generated HTML, like my blog. And it will soon power the search on this blog. No external dependencies, no server, no infrastructure complexity - just a few additional files that get uploaded. And of course active JavaScript in the browser.
pi.dev is a minimalist harness for agentic coding whose focus is not on features, but on extensibility. The underlying idea is solid: a very simple harness that can be extended through TypeScript plugins, so the harness can adapt to your own workflow and requirements. Maybe I'll take a closer look at it soon.
steveyegge/beads: Beads - A memory upgrade for your coding agent is a to-do list tool for agents. Essentially a memory system for projects that agents can use to manage themselves (storing tasks and features) and to coordinate more complex workflows where an agent needs to work through a series of issues and resolve them. Tasks can have dependencies, and agents only see the set of tasks that can actually be worked on right now. Interesting for projects where you want to run agents in loops to solve larger tasks.
dolthub/doltgresql: DoltgreSQL - Version Controlled PostgreSQL is the PostgreSQL-flavored partner of dolthub/dolt: Dolt – Git for Data and offers the same features, but with PostgreSQL syntax and a binary interface that is compatible with PostgreSQL.
dolthub/dolt: Dolt – Git for Data is a SQL database that internally uses mechanisms similar to git, thereby supporting data branches and merges and generally providing the ability to version data in the database and work with history. And like git, it also allows data changes to be distributed via version control. In principle, "slow transaction shipping".
paperclipai/paperclip: Open-source orchestration for zero-human companies is an approach to orchestrating and managing agents. What's interesting here is that it works with an organization modeled after a company structure and uses means to depict agent communication that could provide good transparency. I haven't tried it yet, but alongside gastown, it's one of the more interesting projects on this topic. The whole field of agent orchestration is still quite new, so there's still a large area of experimentation, but it's exciting to watch.
Ghostty is the foundation that cmux — The Terminal for Multitasking was built on. Generally also a very nice terminal that responds noticeably faster than the standard terminal and already works very well. What I didn't like was that tabs weren't automatically reopened in the appropriate directories when the program was closed. I simply have too many persistent sessions that I keep coming back to. cmux just does that better.
cmux — The Terminal for Multitasking is a pretty brilliant terminal program for CLI workflows, which are experiencing a resurgence especially through agentic coding. And what I love about it: it has clean persistence of open workspaces where you can have multiple tabs, so I don't have to keep my work environment open all the time. I've always liked having various directories open directly because I switch back and forth between several while programming, and this works much better with CMUX than with anything else.
NuGet Gallery | Photino.Blazor.CustomWindow 1.3.1 is a library for Photino Blazor that allows you to make windows chrome-less. That is, you can remove the title bar and standard decorations. The idea behind it is to gain more control over the look and feel and create more compact UIs. With this library, you get back basic functions like window resizing and other standard elements that users expect, but under full control of the application.
OpenClaw Memory Masterclass: The complete guide to agent memory that survives • VelvetShark - interesting compilation of the memory system and the pitfalls with compaction in Openclaw. The agent is meant to run for a long time, but there is always the risk that compaction will strike right in the middle of complex situations. And openclaw runs autonomously, so you want to be sure that it continues continuously.
unum-cloud/USearch - that's what it says. So a library that offers an index for vectors that can come from embeddings, for example, and can find semantically similar texts. Not text-similar, but semantically, i.e., content. Interesting topic, the models required for this are related to LLMs, but not large, but small - they don't need to fully understand and generate because they only create vectors that can then be compared against each other and the higher the similarity, the higher the similarity of the texts in the topic. Cool little feature for bDS.
waybarrios/vllm-mlx: OpenAI and Anthropic compatible server for Apple Silicon. I use this to run mlx-community/gemma-3-12b-it-4bit on my MacBook Air. It works very well, a small shell script to start the server and then I am autonomous. Not as comfortable as Ollama, but it perfectly supports Apple's MLX and thus makes good use of Silicon.