daily

By Dan Walsh at Gorey Library

The 16^th annual Gap Arts Festival at Ballythomas, Gorey, runs from Thursday to Saturday, August 6^th to 8^th inclusive and this year’s extensive programme of events was officially launched by Cllr Mary Farrell, Cathaoirleach Gorey Kilmuckridge Municipal District Council at a ceremony in Gorey Library.

The official opening is on Thursday with a photographic exhibition by Gap Camera Club in The Barn at The Gap Pub at 6.30pm which, incidentally, is running for the month.

On Friday at 6pm there is a showcase presentation of Lover, a new play by Joy Redmond read by Tara Quirke and directed by Garrett Keogh. Some sexual references in there so it is unsuitable for under 16’s!

Film and Theatre in Ballythomas School at 8pm and the late-night family movie is Under The Stars from the Jungle Book (1967 Disney) in the Community Field, Ballythomas.

Saturday from 11.30am to 1pm features talks at The Gap; Plant Medicine with Nikki Darrell and Titania’s Palace with Garrett Keogh and that’s in Ballythomas School.

1-4pm is the Drop-In Family Fun Day with face painting, music, a dog show, craft market and more. Theatre and film in Ballythomas School at 8pm.

The Gap Festival will conclude on Saturday night from 6pm with live rock music featuring GAPHEAD, Frankie Fingers, Ashley’s Country Stomp, Robin James Hurt and more playing in The Gap Pub Yard.

Some events are free and some have a charge, but all the details and full programme of events is available on www.GapArtsFestival.com

Booking is available on Eventbrite.ie and the festival is funded through Wexford County Council’s Arts Office Small Arts Festival Grant Scheme 2026 and Kilanerin Ballyfad Community Development Association.

Run 35B and 80B Qwen models on ordinary Apple devices, including iPhones. Swiftlet is a Swift + Metal runtime for the Qwen3-Next and Qwen3.5/3.6 MoE hybrid model family. It keeps only the small dense core of a model resident in memory and streams the routed Mixture-of-Experts weights from storage on demand. The result: | Model | Disk | Peak RAM | Decode speed (M5 Mac) | |---|---|---|---| | Qwen3.6-35B-A3B, 4-bit | 18 GB | 2.6 GB | 7 to 11 tok/s | | Qwen3-Next-80B-A3B, 4-bit | 42 GB | 4.3 GB | 4.5 to 5 tok/s | The 35B also runs on an iPhone 17 in about 2.5 GB of RAM, at about 1 tok/s today. As far as we know, that is the first time a model of this class has run natively on a phone. Status: working end to end. Both models generate correct, validated output. The current focus is kernel speed (the decode loop is dispatch bound, not IO bound, so there is clear headroom). One expectation to set honestly: only about 3B parameters are active per token, so these models chat and write like large models but recall facts like small ones. git clone https://github.com/leonickson1/Swiftlet.git && cd Swiftlet swift build -c release # Download the 35B container from Hugging Face (resumable): .build/release/swiftlet-repack \ --from-hf Leonickson/Qwen3.6-35B-A3B-qpack \ --output ~/models/qwen3.6-35b.qpack # Or the 80B (42 GB on disk, still only ~4.3 GB of RAM): .build/release/swiftlet-repack \ --from-hf Leonickson/Qwen3-Next-80B-A3B-qpack \ --output ~/models/qwen3-next-80b.qpack # Chat (applies the model chat template, disables the reasoning block, # keeps conversation state so follow-ups prefill only the new turn): .build/release/swiftlet chat ~/models/qwen3.6-35b.qpack \ "Who wrote One Hundred Years of Solitude?" "What language did he write it in?" # One-shot generation with stats: .build/release/swiftlet generate ~/models/qwen3.6-35b.qpack \ --gpu --chat --prompt "Explain expert streaming in one paragraph." # OpenAI-compatible server (loopback only): .build/release/swiftlet-server --model ~/models/qwen3.6-35b.qpack --port 8080 The same command also repacks raw MLX checkpoints (--from-hf mlx-community/... or --source /path/to/checkpoint ). Requirements: Apple Silicon, macOS 14+ or iOS 17+, free SSD space for the container (18 GB for the 35B, 42 GB for the 80B). The 35B runs on iPhone inside Priv AI on the App Store: open Settings, then Experimental Models, and download the model. It streams from storage and chats on-device with no server involved. The Experimental Models feature ships in the newest app version, which is still in App Store review, so it may not appear for a couple of days. If you want the phone experience today, build the app from source: the app is open source at leonickson1/localLLM. Clone this repo next to it as swiftlet , open the Xcode project, and run it on your iPhone. These models activate only about 3B of their parameters per token. Each layer routes every token to 10 of 512 experts (80B) or 8 of 256 (35B). Swiftlet: - keeps the dense weights resident: attention, DeltaNet projections, routers, shared experts, embeddings. About 1.3 GB (35B) or 2.5 GB (80B) at 4-bit; - repacks the tens of thousands of routed experts into fixed-stride blobs in a .qpack container, so fetching one expert is exactly onepread from SSD, no mmap and no page-cache thrash; - caches hot experts in a bounded pool with LFU plus recency eviction. Cache size barely affects speed (measured 43 to 70 percent hit rates at the same throughput), because Apple SSDs absorb the misses; - runs the whole forward pass on Metal with runtime-compiled shaders, so no Metal toolchain is needed at build time and the same code ships on iOS. 75 percent of the layers use Gated DeltaNet linear attention with a fixed-size recurrent state, so there is no growing KV cache for those layers at any context length. Swiftlet is a library first: - The Swift package. Add SwiftletCore to any macOS or iOS app and use SwiftletSession for chat with streaming deltas, conversation caching, sampling with repetition control, and memory-pressure handling built in. - The CLI. swiftlet chat andswiftlet generate for local use and benchmarking,swiftlet-repack to build containers from MLX checkpoints (including streaming straight from Hugging Face with resume). - The server. swiftlet-server speaks the OpenAI chat-completions API on loopback, so any chat UI that talks to OpenAI-compatible endpoints can use a streamed local model. - An app. Priv AI on iOS embeds SwiftletCore as its streamed-model engine. End users tap Download and chat. Nothing here is terminal-only. The app itself is open source at leonickson1/localLLM if you want to build it yourself (clone this repo next to it as swiftlet ). Every layer of the forward pass (Gated DeltaNet recurrence, gated GQA attention, sparse MoE routing) is validated against mlx-lm reference implementations with per-layer fixtures, in f32 and int4 quantized form. Incremental decoding is verified against whole-sequence processing. Metal kernels are tested against the exact CPU reference, and the fast and scalar GPU kernels are verified to produce identical outputs. Containers are byte-verifiable against their source checkpoints. Streaming placement never changes model semantics: an expert answers identically from cache or disk. swift test TurboFieldfare proved the expert-streaming thesis for Gemma on Macs, and Swiftlet adopts several of its published design lessons with gratitude: stream experts with pread into a bounded slot pool instead of mmap, evict with LFU plus recency, pack experts at fixed stride so one fetch is one read, install by routing downloaded bytes straight into their final container positions, and compile shaders at runtime. Everything else is built here, from scratch, in about 10k lines of Swift and Metal written against mlx-lm references rather than TurboFieldfare code: - support for a different model family with a fundamentally different architecture: the Qwen hybrid stack with Gated DeltaNet linear attention, gated GQA, and high-sparsity MoE with a shared expert (TurboFieldfare runs Gemma, a classical dense transformer); - MLX affine int4/int8 group quantization compute in Metal, byte-addressed kernels with 64-bit offsets for multi-gigabyte shards, a cooperative simdgroup GEMV fast path, and explicit hazard management; - a validated CPU reference implementation and the fixture infrastructure that gates every kernel change; - the .qpack container and repacker, the resumable Hugging Face streaming installer with stall recovery, and download cancellation; - the chat session layer: template handling for thinking and non-thinking Qwen variants, sampling with presence and frequency penalties and minimum-length and sentence-completion stopping, conversation caching with delta prefill, and iOS memory-pressure coordination; - iPhone support end to end, including the app engine integration. colibrì informed the caching and placement policy thinking. mlx-lm is the correctness reference throughout. Swiftlet was built in collaboration with Claude Code. Apache 2.0. Model weights are downloaded separately and remain governed by their own terms (Qwen models: Apache 2.0). See THIRD_PARTY_NOTICES.md.

Smaller, faster, safer: running Kimi and GLM at scale Workers AI runs inference for some of the best open models in the world on GPUs in Cloudflare data centers close to your users. Two of the most capable, and most demanding, are Moonshot's Kimi K-series and Z.ai's GLM. They are large, long-context, mixture-of-experts models, and they are wonderful to use. They are also very hard to serve efficiently because of memory constraints. We've written before about how we serve large models on Workers AI and about separating the prefill and decode phases of inference to get more out of each GPU. This post looks at three techniques we layer on top of that to fit these models into memory and keep them fast: quantizing the KV cache, compressing the model weights, and, because both of those pack more requests onto shared hardware, protecting the cache those requests share. These optimizations enable us to support more customers at lower costs, with no change in model accuracy. All our experiments and production traffic are running and benchmarked with SGLang, an open-source inference serving framework. We found that SGLang offers the best performance in the market, and we work closely with the SGLang team to upstream patches and new features to make our work available to the open-source community. Quantizing the KV cache As a model generates text, it stores the attention keys (K) and values (V) for every token it has already processed in a structure called the KV cache. The cache is what lets the model extend a long conversation without re-reading the entire context on every new token. For a long-context model, it grows quickly, and it is usually the KV cache, not the model's weights, that fills up GPU memory first. By default, the cache is stored in 16-bit precision (BF16). We store it in 8-bit floating point instead (FP8, e4m3), which halves its size. On Kimi K2.6, that raises the amount of context we can hold in memory from roughly 686,000 tokens to about 1.37 million, twice as much. It's worth being precise about where the benefit comes from, because it isn't raw speed. Quantizing the cache adds a small amount of work per token, since the FP8 attention kernel has to convert values as it reads them. What it changes is how many requests we can keep resident at once. The following measurements are for Kimi K2.6 decoding on a disaggregated H200 deployment, comparing the attention kernels directly: Concurrent requests | BF16 KV cache (tok/s) | FP8 KV cache (tok/s) | |---|---|---| 1 | 137 | 125 | 8 | 731 | 689 | 16 | 1,106 | 1,028 | 32 | 1,558 | 1,489 | 64 | Out of memory | 2,192 | At any single concurrency level, BF16 is a few percent faster per token. But BF16 runs out of cache at 32 concurrent requests and can't admit a 33rd, while FP8 keeps going to 64 and reaches 2,192 tokens per second, about 41% higher than BF16's peak, for roughly 30% less cost per token. Because we run prefill and decode as separate pools, we can apply this where it helps most: prefill is compute-bound rather than memory-bound, so there we leave the cache in BF16 and keep its slightly higher throughput. None of this would matter if it changed the model's answers, so we checked. Across our evaluation suite, FP8 and BF16 caches are indistinguishable: Benchmark | BF16 KV | FP8 KV | |---|---|---| GSM8K | 94.24 | 94.09 | ARC-Easy | 89.06 | 89.14 | ARC-Challenge | 66.72 | 67.49 | MMLU | 89.11 | 89.04 | MMLU-Pro | 80.29 | 79.29 | mcxams (internal benchmark) | 61 / 63 | 61 / 63 | Tool-call validity | 92.2% | 92.6% | Compressing the model weights The KV cache is one demand on GPU memory; the model's weights are the other. For GLM 5.2, we compress the weights from 8-bit floating point down to 4-bit integers (INT4) with no loss in accuracy. The checkpoint shrinks from 705 GB to 421 GB, about 40%, and per-GPU memory across an 8-way tensor-parallel deployment drops from roughly 88 GB to 52 GB, which leaves room for around 1.18 million tokens of KV cache on the same hardware. Across our evaluation suite, INT4 and FP8 weights are indistinguishable: Benchmark / Capability | Metric | FP8 | INT4 | |---|---|---|---| GSM8K | Exact match | 94.39% | 93.56% | GSM8K | Flexible | 94.24% | 93.48% | ARC-Easy | Accuracy | 86.62% | 86.15% | ARC-Easy | Acc (norm) | 84.51% | 85.19% | ARC-Challenge | Accuracy | 64.93% | 64.85% | ARC-Challenge | Acc (norm) | 67.24% | 66.64% | MMLU | Average | 86.60% | 86.54% | MMLU-Pro | Exact | 80.80% | 80.47% | mcxams (internal benchmark) | Passed | 62 / 63 | 62 / 63 | Smaller weights make the decode phase faster, and for a clear reason: generating each token means streaming the model's weights out of GPU memory, so decode speed is limited by memory bandwidth. Move less data and every token arrives sooner. The effect is largest at low concurrency, where per-request latency matters most: Concurrent requests | GLM FP8 (tok/s) | GLM INT4 (tok/s) | INT4 gain | |---|---|---|---| 1 | 60 | 92 | +55% | 8 | 425 | 513 | +21% | 16 | 683 | 825 | +21% | 32 | 994 | 1,267 | +27% | 64 | 1,672 | 1,933 | +16% | Prefill behaves differently. It is compute-bound, and INT4 weights have to be expanded back out before the model can multiply with them, so that extra step makes prefill slower rather than faster, GLM sustains about 10,160 tokens per second of prefill in FP8 versus 8,660 in INT4. As with the KV cache, the disaggregated design turns this into a choice rather than a compromise: we run INT4 for decode, where it wins, and FP8 for prefill, where it wins. Model accuracy stays within 0.8 points of the FP8 model across every benchmark we run, making its quality indistinguishable. Protecting a shared KV cache Both techniques above have the same effect: they let many more requests share one GPU's memory at the same time. That efficiency is the whole point, but it also means hundreds of requests are reading and writing pages of the same physical KV cache. The mechanisms that make this fast, paged attention, continuous batching, cache reuse, all rely on getting the bookkeeping exactly right, and at our request volumes, even a one-in-a-billion mistake would show up regularly. So we built KV cache integrity checking as a layer of defense. The idea is straightforward: every physical cache page gets a tag that changes whenever the page is reallocated, and the server records which pages and tags each request expects to use. Before supported decode operations read from the cache, those mappings are checked. If anything doesn't match, the affected request is aborted rather than allowed to return data from the wrong page. The question that decides whether a safety check ships is what it costs. We measured it on a mid-sized production model in a two-prefill, two-decode configuration, with 8,192-token inputs and 1,000-token outputs: Concurrency | Throughput change | p95 latency change | |---|---|---| 1 | −0.53% | +0.42% | 2 | −0.38% | +0.54% | 4 | −0.79% | +0.63% | 8 | −0.43% | +0.80% | The cost is under 1% on both throughput and tail latency, and even the upper bound of the 95% confidence interval stays near 1%. We kept it computationally cheap by running the validation as a separate batch check rather than fusing it into the attention kernel, which would have introduced a race between GPU thread groups. It's enabled per deployment, and the default path uses a no-op tracker with no measurable overhead, so deployments that don't need it pay nothing. What's next Serving frontier models efficiently is a moving target, and this is the ongoing work behind it. We're expanding FP8 KV caches across more of the fleet, validating NVFP4 weights on Blackwell (NVIDIA’s GPU architecture), and working toward making integrity checks something we can leave on everywhere at negligible cost. These optimizations will allow us to continue to support more customers at a lower cost and at the same accuracy. If squeezing the best open models onto GPUs and serving them to millions of developers sounds like your kind of problem, come work with us.

Quickstart | Configurations | MacOS | Example notebooks | FAQ AirLLM dramatically reduces inference memory usage, letting 70B large language models run on a single 4GB GPU card — without quantization, distillation, or pruning. You can even run 405B Llama 3.1 on 8GB, DeepSeek-V3 (671B) on ~12GB, and Kimi K3 (2.8T) — the largest open-source model released to date — on under 4GB, because sparse MoE models stream one expert at a time rather than a whole layer. [2026/07] Kimi K3 (2.8T) support: the largest open-source model runs on a single card in 3.72GB of VRAM, measured end to end on one RTX 6000 Ada. Per-expert streaming loads only the experts a token actually routes to. K3 brings three requirements of its own: pip install compressed-tensors flash-attn (its model code mandates flash attention regardless of what you request), a CUDA 12 build of torch, since no prebuilt flash-attn wheel exists for CUDA 13 yet, and transformers 4.56.x, as its remote code does not load on 5.x. [2026/06] v3.0: FP8 model support + the latest models. Run DeepSeek-V3 (671B) on ~12GB and Qwen3-235B on ~3GB, plus Qwen3, Llama 3.x/4, DeepSeek V2/V3, Phi-4, Gemma and more — all through a single AutoModel . [2024/08/20] v2.11.0: Support Qwen2.5 [2024/08/18] v2.10.1 Support CPU inference. Support non sharded models. Thanks @NavodPeiris for the great work! [2024/07/30] Support Llama3.1 405B (example notebook). Support 8bit/4bit quantization. [2024/04/20] AirLLM supports Llama3 natively already. Run Llama3 70B on 4GB single GPU. [2023/12/25] v2.8.2: Support MacOS running 70B large language models. [2023/12/20] v2.7: Support AirLLMMixtral. [2023/12/20] v2.6: Added AutoModel, automatically detect model type, no need to provide model class to initialize model. [2023/12/18] v2.5: added prefetching to overlap the model loading and compute. 10% speed improvement. [2023/12/03] added support of ChatGLM, QWen, Baichuan, Mistral, InternLM! [2023/12/02] added support for safetensors. Now support all top 10 models in open llm leaderboard. [2023/12/01] airllm 2.0. Support compressions: 3x run time speed up! [2023/11/20] airllm Initial version! - Quick start - Model Compression - Configurations - Run on MacOS - Example notebooks - Supported Models - Acknowledgement - FAQ First, install the airllm pip package. pip install airllm Then, initialize AirLLMLlama2, pass in the huggingface repo ID of the model being used, or the local path, and inference can be performed similar to a regular transformer model. (You can also specify the path to save the splitted layered model through layer_shards_saving_path when init AirLLMLlama2. from airllm import AutoModel MAX_LENGTH = 128 # just pass a hugging face repo id — works with almost any popular model: model = AutoModel.from_pretrained("Qwen/Qwen3-32B") # go bigger with the exact same one line: #model = AutoModel.from_pretrained("Qwen/Qwen3-235B-A22B") # 235B, runs in ~3GB #model = AutoModel.from_pretrained("deepseek-ai/DeepSeek-V3") # 671B, runs in ~12GB # or use a model's local path... #model = AutoModel.from_pretrained("/home/ubuntu/.cache/huggingface/hub/models--Qwen--Qwen3-32B/snapshots/...") input_text = [ 'What is the capital of United States?', #'I like', ] input_tokens = model.tokenizer(input_text, return_tensors="pt", return_attention_mask=False, truncation=True, max_length=MAX_LENGTH, padding=False) generation_output = model.generate( input_tokens['input_ids'].cuda(), max_new_tokens=20, use_cache=True, return_dict_in_generate=True) output = model.tokenizer.decode(generation_output.sequences[0]) print(output) Note: During inference, the original model will first be decomposed and saved layer-wise. Please ensure there is sufficient disk space in the huggingface cache directory. We just added model compression based on block-wise quantization-based model compression. Which can further speed up the inference speed for up to 3x , with almost ignorable accuracy loss! (see more performance evaluation and why we use block-wise quantization in this paper) - Step 1. make sure you have bitsandbytes installed by pip install -U bitsandbytes - Step 2. make sure airllm verion later than 2.0.0: pip install -U airllm - Step 3. when initialize the model, passing the argument compression ('4bit' or '8bit'): model = AutoModel.from_pretrained("garage-bAInd/Platypus2-70B-instruct", compression='4bit' # specify '8bit' for 8-bit block-wise quantization ) Quantization normally needs to quantize both weights and activations to really speed things up. Which makes it harder to maintain accuracy and avoid the impact of outliers in all kinds of inputs. While in our case the bottleneck is mainly at the disk loading, we only need to make the model loading size smaller. So, we get to only quantize the weights' part, which is easier to ensure the accuracy. When initialize the model, we support the following configurations: - compression: supported options: 4bit, 8bit for 4-bit or 8-bit block-wise quantization, or by default None for no compression - profiling_mode: supported options: True to output time consumptions or by default False - layer_shards_saving_path: optionally another path to save the splitted model - hf_token: huggingface token can be provided here if downloading gated models like: meta-llama/Llama-2-7b-hf - prefetching: prefetching to overlap the model loading and compute. By default, turned on. For now, only AirLLMLlama2 supports this. - delete_original: if you don't have too much disk space, you can set delete_original to true to delete the original downloaded hugging face model, only keep the transformed one to save half of the disk space. Just install airllm and run the code the same as on linux. See more in Quick Start. - make sure you installed mlx and torch - you probably need to install python native see more here - only Apple silicon is supported Example [python notebook] (https://github.com/lyogavin/airllm/blob/main/air_llm/examples/run_on_macos.ipynb) Example colabs here: Details - ChatGLM: from airllm import AutoModel MAX_LENGTH = 128 model = AutoModel.from_pretrained("THUDM/chatglm3-6b-base") input_text = ['What is the capital of China?',] input_tokens = model.tokenizer(input_text, return_tensors="pt", return_attention_mask=False, truncation=True, max_length=MAX_LENGTH, padding=True) generation_output = model.generate( input_tokens['input_ids'].cuda(), max_new_tokens=5, use_cache= True, return_dict_in_generate=True) model.tokenizer.decode(generation_output.sequences[0]) - QWen: from airllm import AutoModel MAX_LENGTH = 128 model = AutoModel.from_pretrained("Qwen/Qwen-7B") input_text = ['What is the capital of China?',] input_tokens = model.tokenizer(input_text, return_tensors="pt", return_attention_mask=False, truncation=True, max_length=MAX_LENGTH) generation_output = model.generate( input_tokens['input_ids'].cuda(), max_new_tokens=5, use_cache=True, return_dict_in_generate=True) model.tokenizer.decode(generation_output.sequences[0]) - Baichuan, InternLM, Mistral, etc: from airllm import AutoModel MAX_LENGTH = 128 model = AutoModel.from_pretrained("baichuan-inc/Baichuan2-7B-Base") #model = AutoModel.from_pretrained("internlm/internlm-20b") #model = AutoModel.from_pretrained("mistralai/Mistral-7B-Instruct-v0.1") input_text = ['What is the capital of China?',] input_tokens = model.tokenizer(input_text, return_tensors="pt", return_attention_mask=False, truncation=True, max_length=MAX_LENGTH) generation_output = model.generate( input_tokens['input_ids'].cuda(), max_new_tokens=5, use_cache=True, return_dict_in_generate=True) model.tokenizer.decode(generation_output.sequences[0]) To request other model support: here AirLLM works out of the box with virtually every popular open LLM — just pass its Hugging Face ID to AutoModel.from_pretrained(...) . That covers all the major families: Llama (2 / 3 / 3.1 / 3.3 / 4) · Qwen (1 / 2 / 2.5 / 3, including MoE and FP8) · DeepSeek (V2 / V3 / R1) · Mistral & Mixtral · Phi · Gemma · ChatGLM · Baichuan · InternLM · Yi — and most new models the day they're released. The trick: AirLLM only ever keeps one layer on the GPU at a time, so the VRAM you need depends on the model's layer size — not its total size. That's how a 671B model fits on a hobbyist card: | Model | Size | GPU VRAM | |---|---|---| | Qwen3 / Mistral / Phi (≈8B) | 8B | ~1–2 GB | | Qwen3-30B / Mixtral (MoE) | 30–47B | ~1–3 GB | | Qwen3-235B (MoE) | 235B | ~3 GB | | Llama 3.x 70B (full precision) | 70B | ~4 GB | | Llama 3.1 405B | 405B | ~8 GB | | DeepSeek-V3 | 671B | ~12 GB | Same one line of code for all of them — no special setup. A lot of the code are based on SimJeg's great work in the Kaggle exam competition. Big shoutout to SimJeg: GitHub account @SimJeg, the code on Kaggle, the associated discussion. safetensors_rust.SafetensorError: Error while deserializing header: MetadataIncompleteBuffer If you run into this error, most possible cause is you run out of disk space. The process of splitting model is very disk-consuming. See this. You may need to extend your disk space, clear huggingface .cache and rerun. Most likely you are loading QWen or ChatGLM model with Llama2 class. Try the following: For QWen model: from airllm import AutoModel #<----- instead of AirLLMLlama2 AutoModel.from_pretrained(...) For ChatGLM model: from airllm import AutoModel #<----- instead of AirLLMLlama2 AutoModel.from_pretrained(...) Some models are gated models, needs huggingface api token. You can provide hf_token: model = AutoModel.from_pretrained("meta-llama/Llama-2-7b-hf", #hf_token='HF_API_TOKEN') Some model's tokenizer doesn't have padding token, so you can set a padding token or simply turn the padding config off: input_tokens = model.tokenizer(input_text, return_tensors="pt", return_attention_mask=False, truncation=True, max_length=MAX_LENGTH, padding=False #<----------- turn off padding ) If you find AirLLM useful in your research and wish to cite it, please use the following BibTex entry: @software{airllm2023, author = {Gavin Li}, title = {AirLLM: scaling large language models on low-end commodity computers}, url = {https://github.com/lyogavin/airllm/}, version = {0.0}, year = {2023}, } Bloome is an AI-agent IM platform: build and run AI agent teams in the cloud with zero setup. Add a skill as an agent in a group chat, run it in one click from web or mobile, and share it with your team — think of it as a group chat where your AI assistants are teammates you can @mention and assign tasks to. 👉 Try Bloome Welcomed contributions, ideas and discussions! If you find it useful, please ⭐ or buy me a coffee! 🙏

3rd August 2026 One of the arguments for open source software for end-users has always been the freedom to examine and modify how that software works. The reality for most people - even expert programmers - has been that the freedom is more about being able to lean on other people to do that. Most people can't justify the time commitment needed to read and then modify the code for tools they use very often. I think LLMs have changed that equation in a way that makes the original dream much more feasible. Several times a day I'll prompt regular Claude chat to "Clone x/y from GitHub and tell me how Z works". Getting software to compile in order to start hacking on it used to be enough friction that I often wouldn't bother. Now I treat that as a zero time investment challenge: tell Codex or Claude Code to checkout and build X and then come back ten minutes later and see how it got on. I'm not habitually modifying the software I use yet, but I can see a path to that which didn't exist a year or so ago.

Russia has warned Ireland that it will face “serious consequences” if the Defence Forces attempt to board or intercept shadow fleet vessels under proposed legislation expanding the military’s operational powers in Irish waters. The Defence (Amendment) Bill 2026 would allow Irish military personnel to board vessels in the country’s maritime zones to inspect documents, equipment and activities, as well as directing vessels to alter their route. Russian Ambassador to Ireland Yuri Filatov said the legislation suggested that the Government was preparing to join the “pirate actions” of other countries, which have intercepted vessels seeking to evade anti-Russia sanctions. “This is a demonstration by Dublin that they are prepared for such actions and, in theory, could join the pirate actions of their senior partners,” Filatov said in an interview published by Russian newspaper Izvestia. “The Irish have been warned that attempts to carry out such acts of piracy will have the most serious consequences for them,” he added. The article claimed that Ireland had a shortage of patrol vessels and suggested that the interception and detention of ships would only occur with the participation of the British or French navies. The proposed amendment to the Defence Act 1954 was initiated in the Dáil in June, and would grant legal authority to the Defence Forces to board and reroute vessels in Irish waters or its Exclusive Economic Zone (EEZ) where illicit activities are suspected. The Izvestia article erroneously reported that these legislative changes had come into effect on July 24, citing Mr Filatov. It said Ireland was important to Western efforts to detain “ships allegedly acting in Moscow’s interests” because of its location close to vital Atlantic shipping routes. Defence Minister Helen McEntee said in March that the proposed legislative amendments were required in response to “emerging threats in our waters”, specifically referring to vessels linked to Russia’s “shadow fleet”. “Recent events in the Baltic Sea and the activities of Russia’s shadow fleet have highlighted the importance of ensuring that we have the legal framework necessary to respond to evolving threats,” she said. McEntee said existing legislation does not explicitly assign maritime enforcement powers to the Defence Forces. “This legislation will address that gap by clearly setting out the Defence Forces’ role in safeguarding our maritime domain while remaining fully consistent with international law,” she added.

ChatGPT Codex Voice + browser + Sites: an expert’s AI workflow | Nick Baumann (OpenAI)

Brought to you by:

• Bolt.new—Turn your idea into a real product

• Hyperagent—Deploy fleets of agents that handle real work

Nick Baumann from OpenAI joins Claire to demonstrate how he uses ChatGPT, Codex, and Voice to automate huge parts of his work and life. He walks through advanced workflows for managing tasks in parallel, monitoring email and expenses, booking travel, building and publishing websites, and even turning dozens of raw clips into finished videos.

Biggest takeaways:

1. ChatGPT Codex Voice doesn’t just answer questions. It can actually operate your computer while you keep talking. Nick Baumann demos the new interface live: he triggers it with a hotkey, the orb reads what’s on his screen through Appshots, and Codex spins up separate threads to book a flight, file an expense report, and check his calendar at the same time.

2. One of Codex’s most powerful features is also one of its least understood: thread-forking. When a task gets complicated, Codex can open a new thread, branch from an existing one, or pull context from previous conversations on its own. Users don’t have to manually organize everything first.

3. Heartbeats in ChatGPT Work offer a practical glimpse of always-on agents. Nick gets notified when a suspicious charge appears, a package is nearby, or an email needs his attention—and he set it all up without writing code.

4. ChatGPT Sites looks much closer to a real deployment platform than a lightweight demo. It includes a SQL database, file storage, environment variables, and email-based access controls. During the episode, Claire and Nick build and publish a live How I AI tips site, organized by tool and job function, in roughly the time it would normally take to set up a CMS template. For internal tools, small apps, and private resources, that starts to feel genuinely useful.

5. Nick’s video-editing workflow is one of the clearest examples of AI cutting real production work. He records 50 or 60 raw clips, uploads them, talks through the rough story he wants, and lets a custom plugin transcribe everything, scan the footage, choose the strongest takes, and assemble finished vertical videos overnight. What would normally eat up most of a content team’s day is largely handled while he sleeps.

6. Voice may be the best way to give AI context because it removes the pressure of figuring out exactly what to type. Claire calls this the “yapper’s API,” and the idea is simple: people often get more useful results when they talk freely than when they stare at an empty prompt box trying to be precise. Across two-person voice chat, mobile dictation, and Codex’s orb interface, the same pattern keeps showing up—people who talk to AI tend to give it more context, and more context usually leads to better work.

7. As AI gets more capable, latency is becoming a much bigger product decision. Nick often uses the highest-intelligence mode in Voice, but once a model has the tools it needs, speed can matter more than another small jump in reasoning quality. A lot of AI products are still designed around the assumption that waiting is unavoidable. As that changes, the tools that feel immediate will have a major advantage—especially for users who currently find AI too slow to bring into a live workflow.

If you’re enjoying these episodes, reply and let me know what you’d love to learn more about: AI workflows, hiring, growth, product strategy—anything.

Catch you next week,
Lenny

P.S. Want every new episode delivered the moment it drops? Hit “Follow” on your favorite podcast app.

Nick Baumann is on the Developer Experience team at OpenAI, where he spends his days building with, testing, and communicating the capabilities of ChatGPT Codex and ChatGPT Work. In this episode, Nick walks me through several features that have launched or evolved recently: the new voice interface with its screen-reading orb, the Heartbeats automation system in ChatGPT Work on mobile, the live ChatGPT Sites deployment feature, and his personal use case for AI-assisted UGC video editing.

What you’ll learn:

1. How two-person voice chat works

2. How Heartbeats work

3. How to build and deploy a live website with ChatGPT Sites

4. How to delegate a flight search, hotel booking, and expense report to Codex in a single voice conversation without opening a single app manually

5. Why ChatGPT Work on mobile is the most underutilized AI workflow for people already using the ChatGPT app

6. How to use a custom UGC Video plugin to feed 50 raw clips into ChatGPT, let it pull transcripts, pick the best takes, and assemble a finished vertical video overnight

Brought to you by:

Bolt.new—Turn your idea into a real product

Hyperagent—Deploy fleets of agents that handle real work

In this episode, we cover:

(00:00) Introduction to Nick Baumann

(02:56) What’s new in Codex

(05:40) ChatGPT Work and Heartbeats

(06:40) Live Codex voice demo

(13:25) Latency vs. intelligence

(14:36) Quick recap

(15:04) Voice on mobile and the ChatGPT Sites workflow

(21:24) Live UGC video demo

(32:30) How I AI website results

(34:04) Lightning round and final thoughts

Tools referenced:

• ChatGPT Codex: https://chatgpt.com/codex

• ChatGPT Sites: https://chatgpt.site

Where to find Nick Baumann:

LinkedIn: linkedin.com/in/nick--baumann

Where to find Claire Vo:

ChatPRD: https://www.chatprd.ai/

Website: https://clairevo.com/

LinkedIn: https://www.linkedin.com/in/clairevo/

X: https://x.com/clairevo

Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email jordan@penname.co.

4th August 2026 Gas Town was intended to be reusable, but I only ever wound up using it to build itself. Gas Town fell apart at the seams with Opus 4.7. Up through 4.6 it was working brilliantly. With 4.7 we saw the introduction of the "just two more things" tic, which prevented Opus from ever converging on being ready to do real work—it always wanted to fiddle with Gas Town itself. The Opus tic never went away, so Gas Town effectively burned down. It had other problems, too, but 4.7 was the final straw. — Steve Yegge, The Shape of Things to Come

3rd August 2026 - Link Blog Don't be a meat proxy (via) Niklas Gruhn coins an excellent new term - meat proxy - for people who blindly copy and paste the output of AI systems to their peers. By all means, prompt AI. But don't just relay the output. Read it, understand it, validate it, and then write a response in your own words (a decent certificate that you've done the prior steps). Making that effort is value you can add.