Skip to content
Ryan de Melo
Go back

Platform Engineering: Paving Roads vs Building Cages

People still ask how that ML platform, the one I owned a few years back, ended up with a couple dozen product teams building on it. My honest answer always starts with the quarter I nearly killed it. The fastest way to kill an internal platform is to make it mandatory before it is good. I have watched other teams do it, and I came within one planning doc of doing it myself.

Feature store, model serving, the vector and embedding pieces as they came along, the deploy path that took a model from a notebook to something serving real traffic: it all got built, and it all got used. What people miss now is how close it came to being the platform everyone routed around.

The day a platform team starts measuring adoption by mandate compliance is the day it has stopped building a road and started building a cage. The two look identical on an org chart: one team owns the path, other teams are on it, the dashboard is green. What separates a road from a cage is why people are on the path at all. On a road, they chose it because it was faster. In a cage, they are there because the fence went up and their old route got a ticket queue bolted to it.

The road was already there

The teams I served were not waiting for me to invent a workflow; they already had one. They pulled features out of warehouse tables by hand, trained on a borrowed GPU, and shipped a model by copying a pickle file somewhere a server could reach it. It was ugly and it mostly worked. (The pickle file living in an S3 bucket nobody owned was a war crime, but it shipped models, which is more than most platforms can say in their first quarter.)

A paved road follows the path people already walk. That sounds obvious and almost nobody does it. The temptation is to design the workflow you wish they had, the clean one from the architecture diagram, and then stand at the trailhead explaining why their cow path is wrong. You will lose that fight, because people do not abandon a route that works for one that is correct in theory. They abandon it for one that is faster on the exact task they were already doing.

The feature store did not start as a feature store. It began as one query that fetched the features three teams kept rewriting, returned the same answer, and was faster to call than to rewrite. That was the whole pitch: not governance, not lineage, not a single source of truth, only faster than what you are doing right now, on the task you are already doing.

When the road becomes a fence

There is a specific moment when a platform team turns into a gatekeeper, and it is rarely a decision anyone announces. It arrives as a reasonable-sounding sentence in a planning doc: “All new models must deploy through the platform.” Said out loud, it sounds like maturity. What it actually means is that you have given up on being chosen and decided to be enforced.

You can hear it in the conversation that follows. Someone has a model that does not fit the golden path: an odd serving shape, a latency need the platform cannot hit, a dependency you never supported. Before the mandate, that is a feature request and a happy customer telling you exactly where the road needs to extend. After the mandate, that same person is a problem to be managed, someone who wants an exception. Now you are in the business of granting or denying exceptions, which means you are a checkpoint, and every checkpoint has a queue, and every queue grows a shadow copy of your platform built by people who got tired of waiting.

I have been that bottleneck myself, not because the platform was bad but because we had made ourselves the only door and then could not open it fast enough. The mandate did not increase real adoption. It converted enthusiastic users into resentful ones and pushed the clever ones underground, where they rebuilt the exact system we were trying to standardize, just without us watching.

The only honest test

The golden path versus mandate tension never fully goes away, and I do not think it should. Some things genuinely have to be enforced. How you handle data residency, how you log access to a model that touches money, the audit trail a regulator will ask for: those are not opinions, and pretending they are golden paths people may opt into is how you end up explaining yourself to a regulator. So enforce them, but be honest that you are building a fence. Build the smallest one that does the job, and pave a genuinely good road right up to it, so people meet the fence last instead of first.

For everything else, there is a single test, and it is the only one I trust. Is the paved road faster than asking a human, and faster than rolling your own?

That is a brutal bar, and it is supposed to be. Your self-service path has to beat a Slack message to a helpful colleague, which is the real competition, not some other platform. It also has to beat a motivated engineer’s afternoon of doing it by hand, because that engineer will absolutely spend the afternoon rather than learn your abstraction if the abstraction is slower or more confusing than the raw version underneath. If you cannot win on speed, you do not have a platform. You have a tax with a logo.

The platform that got used passed that test most of the time, and where it failed, it lost, fairly, to a human or a hand-rolled script, and that loss was the most useful signal I got all year. It told me exactly where to pave next. The version of me that wanted to write a mandate would have buried that signal under compliance and called the dashboard a success.

The honest measure of a platform is what people would do the moment you stopped requiring it. If they would stay, you built a road; if they would scatter, you built a cage, and all the mandate protected was your own headcount. I still picture the version of that platform I almost shipped, dashboard green, exceptions stacking up in a queue, and I am certain that the week I dropped the mandate, half those teams would have been back to copying a pickle file into an S3 bucket nobody owned, glad to be on their own cow path again.


Share this post:

Previous Post
Llama 2 Is Here. Should You Self-Host?
Next Post
The Real Cost of a Customer Data Platform