Archives

After eighteen years of watching big data, mobile, microservices, cloud, and now AI agents arrive on the same script, here is how I separate the durable capability from the narrative.

Two ways to fail when you ship GenAI into a regulated business: never ship at all, or ship something nobody can audit. The narrow path between them, from building this in financial services and industrial operations.

Sixteen months of building production systems on the Model Context Protocol. The interoperability bet paid off. Auth, versioning, and the demo-to-production gap are still where teams bleed.

An end-to-end pipeline for fine-tuning a small model in 2026: distill the data, train adapters, hold an eval bar, ship behind a canary, and watch for the drift that quietly eats your accuracy.

Bolting a human approval step onto a badly designed agent does not make it safe. It makes a rubber stamp. The human-in-the-loop work is UX and architecture, not a checkbox.

An agent that loops and calls tools spends tokens like a single prompt never does. Here is the cost model that explains the bill, and the levers that cut it without making the agent dumber.

My day-to-day loop for LLM features in 2026: write the eval first, then the prompt, then the code, and fold every production failure back in as a case.

An agent that moves money is only acceptable if every decision it makes can be explained, replayed, and pinned to a responsible party after the fact. The audit layer is the product.

A production system that resolves stuck accounts-receivable mismatches by retrieving over invoices, contracts, remittances, and email, then proposing a fix a human approves before any money moves.

The durable enterprise AI layer is not a model or a chatbot. It is an operating system that gives agents identity, permissions, tools, memory, and an audit trail over the systems a company already runs.

Most multi-agent designs are one agent's job split across five processes that now have to argue with each other. The few cases where splitting actually pays, and the complexity to refuse.

What to instrument for LLM and agent apps before the first incident: full request and tool-call tracing, token and cost per request, latency breakdown, eval scores in production, and turning real failures into eval cases.

Catching reconciliation anomalies across institutional financial records, where the model is a rounding error and the matching and normalization is the whole job.

I left a senior leadership seat at a large company to found an AI venture and write code again. The pull, the discomfort, and what eighteen years of platform work left me wanting to do.

A closed-loop field inspection system that turns voice, a photo, and a half-filled form into a structured action, built for places where the network drops for hours.

Enterprise RAG is trustworthy because of the unglamorous parts: per-user permissions enforced at retrieval, freshness, lineage, and handling records that change. Retrieval is an access-control problem wearing a search costume.

For regulated workloads where the data legally cannot leave a building, on-prem GPU inference is back. The build-vs-rent math, the constraints nobody prices in, and the software that makes a fixed fleet feel like a platform.

On narrow, high-volume tasks a fine-tuned small model matches frontier quality at a fraction of the cost and latency. Here is the pipeline, the eval bar, and the maintenance bill nobody quotes you.

How to put real constraints around an agent that touches money or production: bounded tools, approval gates on irreversible actions, dry-run modes, spend limits, and a tool-call audit trail you can actually read.

Instead of one large agent wired to every financial system, a fabric of small MCP servers, each wrapping one system with tightly scoped tools and an approval gate on anything that writes.

The Model Context Protocol standardizes how models reach tools and data, the way a connector standard kills a drawer full of adapters. The ecosystem is thin. I'm betting on the protocol anyway.

Two years of running a self-service ML platform across twenty-odd product teams. What the paved paths got right, what we built too early, and the only success metric that turned out to matter.

A national regulator's residency and sovereignty rules redrew the topology of a payments system. Where data lives, what can leave, and where the keys sit are architecture decisions, not a config flag you toggle at the end.

A skeptical look at agent reliability in late 2024, where the impressive demos quietly fall apart in production, and the narrow places agents already pull their weight.

Two large platforms merged, and now we owned two of everything. How we collapsed the overlap into one stack without a single customer feeling it, and why the politics were harder than the code.

Function calling and JSON mode get you syntactically valid JSON. They do nothing about a model that fills the right shape with confident nonsense. The validation-and-repair layer you still have to write.

A multi-year hyperscaler commitment is an architecture decision wearing a procurement costume, and engineers who skip the room get the bill.

Moving a multi-petabyte warehouse to an open table format over object storage. The format is a weekend. The operations are the project, and nobody puts that in the deck.

Pure vector search quietly fails on the exact terms, codes, and acronyms users actually type. Combining BM25 with dense retrieval, fusing the two, and paying the latency bill it costs.

Self-hosting an open model when GPUs are scarce and finance is reading the bill. Continuous batching, KV-cache, what quantization actually costs you, and when to just call a hosted API instead.

A decision framework for RAG versus fine-tuning that is not "it depends." Three questions settle most of it, and the cases where fine-tuning actually wins are narrower than the budget requests suggest.

Shipping an LLM feature with no evals is shipping with no tests, and almost everyone is doing it. A small, hand-written harness you run on every change, plus the honest limits of grading with another model.

A year into production RAG, the retrieval problems teams keep blaming on the model are almost always chunking, metadata, and document structure. Concrete fixes, with the splitting code I actually run.

Across four countries and a few hundred engineers, the system came to look exactly like the org chart. After years of fighting that, I started using it on purpose.

Building a buy-now-pay-later product end to end and selling it to banks. The credit model is the easy part. Disbursement, reconciliation, and collections are where it lives or dies.

Real-time fraud scoring that lives inside the payment authorization path has a tiny latency budget, and a slightly worse model that fits it beats a better one that does not.

A demand-forecasting ensemble (a classical statistical model, a sequence model, and gradient boosting) that took accuracy far enough to cut inventory hard, plus the boring data problems that mattered more than the model.

The week Llama 2 dropped, half my inbox asked whether to pull inference in-house. The break-even math, the GPU scarcity, and the on-call tax nobody puts in the spreadsheet.

An internal ML platform that dozens of teams actually used, and the one test that told me whether I was paving a road or building a cage around it.

Unifying a customer record across several business units sounds like a data project. It is mostly an ownership fight, and the maintenance never ends.

Most teams adding semantic search this year should start in the Postgres they already run, not a new vector database. Where pgvector holds, where it doesn't, and how to tell which side of the line you are on.

A production retrieval pipeline over a few hundred thousand internal documents, hand-rolled in early 2023. The model is the easy part. Retrieval is where the quality lives or dies.

Collaborative filtering and plain co-occurrence carried a marketplace recommender to hundreds of millions of recs a day. The exact point where deep ranking earned its complexity, and why most teams reach for it a year early.

Owning a large cloud P&L taught me that cost is an engineering-culture problem, not a dashboard. Where the tooling earns its keep and where it is theater.

Weeks after ChatGPT launched, every exec wants it shipped into the product. Here is the production math most teams have not done yet, and the short list of who should not wait.

Scoring hundreds of thousands of small merchants with no credit-bureau file, where the ensemble was easy and the fairness, reason codes, and feedback loops were the part that took a year.

Moving a large analytics workload to ClickHouse cut query latency and the bill hard, but only after we stopped designing tables the way Postgres taught us.

A payment orchestration layer across dozens of methods is not a router. It is a failure-handling system, because every method breaks in its own way and you only find out at checkout.

A long migration of hundreds of services and multi-petabyte data across four clouds plus colo, run as boring reversible waves instead of a heroic weekend.

A feature store fixes training/serving skew and lets teams share features, but most adopt it a year early and pay the operational tax for nothing. Here is the line where it flips.

Rolling out data mesh across dozens of business units taught me that domain ownership of data is a reporting-line and incentive change first, and a technology choice a distant second.