I build a multi-tenant platform in what is effectively a software factory: a fleet of coding agents writes most of the code, runs the checks, opens the merge requests and lands them. That changes what CI is for. A person pushes a few times a day and waits for the result. Agents push dozens of times a day, run in parallel on the same machines, never mind waiting, and believe whatever the pipeline tells them. In June that broke our CI: for 13 days, not one pipeline on main went green. A month later it had passed 74 times in a row while running five times as many jobs.
Read entry →
Follows on from Putting an RTX 3090 in an HP ProLiant ML350 Gen9. The card is a second-hand blower-style 3090 in an enterprise tower, power-capped to 275 W.
Read entry →
I run semantic search over a few thousand recorded meetings. The single thing that makes it work is not the embedding model or the vector store — it is a paragraph of context an LLM writes for every chunk before I embed it. That enrichment step roughly doubled retrieval quality. It is also the slowest, most expensive part of the pipeline, so this summer I tried to make it cheaper by swapping the model that writes the context.
Read entry →
The castle stays put. You only get a small bag of extra bricks.
Read entry →
We have been running low-bit quantization experiments on CPU only. It works, and it is slow enough that the feedback loop hurts — a context-depth sweep is most of an afternoon. The GPUs we already own are doing production work, so borrowing one was not an option. What we did have spare was an older dual-socket ML350 Gen9 tower with 512 GB of RAM, which is a good shape for MoE expert offload and had no GPU at all. So: a second-hand RTX 3090, blower-style, into an enterprise tower that was never designed for one.
Read entry →