The Quiet Power of the Internal Tech Blog

Somewhere in your company there's an engineer who knows why the payments service has a retry policy that looks insane. They know because they were on call the night it earned that policy — the night a downstream provider went flaky for six hours and the naive retry logic turned a partial outage into a self-inflicted DDoS. That knowledge is in one person's head. When they leave, it leaves with them.

Unless someone wrote it down.

Documentation nobody reads vs. writing people remember

Let's be honest about internal documentation: most of it is a write-only medium. Wikis go to die. The architecture decision record that was supposed to capture why we chose the message queue gets written six months after the decision, by someone reconstructing it from Slack threads. The runbook covers the happy path and three of the sad ones; the fourth sad one is the one that pages you.

But there's a different kind of internal writing, and it works. It's the incident retrospective that reads like a story instead of a template. It's the "here's what I learned migrating the auth service" post that an engineer writes because they need to think it through, and then it turns out half the org needed to read it. It's the decision record written before the decision, while the tradeoffs are still alive and contested, instead of after, when everyone's just justifying what already happened.

The difference isn't format. It's honesty. The writing that gets read is the writing that tells the truth about what happened — including the parts where someone made a bad call at 3 AM and the system survived anyway.

Incident lore is an asset

Every company has incident lore: the outage everyone references by name, the deploy that became a verb. "Don't pull a Jenkins on the database migration." That lore is doing real work — it's compressed experience, transmitted orally, and it shapes behavior more than any policy document.

The problem with oral tradition is that it doesn't scale and it doesn't survive turnover. The new hire in the satellite office doesn't know what "pulling a Jenkins" means. The lore becomes an in-joke instead of a lesson.

Writing it down fixes this, but only if the writing captures the actual story. A good incident write-up has a narrative arc: here's what we believed, here's what happened, here's what we believe now. It names the surprise. The surprise is the valuable part — it's the thing nobody's mental model predicted, which means it's the thing everyone's mental model needs to absorb.

The best internal incident writing I've seen reads less like a report and more like a field note from someone who was there. Because it is.

Decision records, written when they matter

Architecture decision records have a timing problem. Written after the fact, they're archaeology — interesting, maybe, but nobody's mind is changed by them. The doc exists to make the decision better. The fact that it also becomes a record is a bonus. Teams that get this backwards — writing the ADR to satisfy a process, after the decision is already made — get the worst of both worlds: the overhead of documentation with none of the clarity.

I've started treating decision docs as discussion documents with a shelf life, not as monuments. The doc exists to make the decision better. The fact that it also becomes a record is a bonus. Teams that get this backwards — writing the ADR to satisfy a process, after the decision is already made — get the worst of both worlds: the overhead of documentation with none of the clarity.

Write in public (internally)

Here's the part that makes people nervous: the best internal writing is visible. Not in a private doc shared with three people, but somewhere the whole engineering org can stumble across it. The junior engineer who reads a senior's migration post-mortem learns more in twenty minutes than in a month of ticket work. The team in another timezone avoids a mistake because someone wrote down the mistake they made.

Yes, this requires a culture where writing honestly about failures doesn't get you punished. That's a leadership problem, not a tooling problem. No wiki platform ever fixed a blame culture.

But if you have even a pocket of that culture — one team that writes real postmortems, one engineer who documents the weird retry policy and why — protect it. Feed it. That person is doing the most undervalued work in your organization: turning experience into something that outlives the people who had it.

The payments service retry policy looks insane until you know the story. Write down the story.

CISA's Secure by Design, One Year In

CISA's Secure by Design pledge asked software vendors to do something radical: take responsibility for the security of the products they ship. Not document the risks and hand them to the customer. Not sell the secure version as the premium tier. Build it in from the start.

A lot of vendors signed. The pledge asked for concrete commitments — things like eliminating default passwords, publishing vulnerability disclosure policies, and working toward reducing entire classes of vulnerabilities rather than whack-a-moling individual bugs. On paper, it was the most meaningful shift in the vendor-customer security relationship in years.

So where are we?

What the pledge got right

The core idea is sound, and it's worth saying plainly: the customer should not be the QA department for a vendor's security. For decades, the industry's unspoken deal was that buyers would configure, patch, monitor, and generally babysit products into something resembling secure — and vendors would sell them the tools to do the babysitting. Secure by Design flips that deal. The radical part isn't the technology. It's the accountability.

The pledge also did something subtle and important: it gave security-conscious buyers language. Before the pledge, asking a vendor "what are you doing about entire classes of vulnerabilities?" got you a blank stare and a brochure. Now there's a shared vocabulary. Procurement teams can point at the pledge and ask where a vendor stands. That's real progress, even if it's boring progress.

Where it gets wobbly

Here's the uncomfortable part. A pledge is not a regulation. There's no enforcement mechanism, no audit, no penalty for signing the thing and then quietly doing nothing. The vendors who were already serious about security signed it and kept doing what they were doing. The vendors who weren't... also signed it, in many cases. A signature is cheap. Secure defaults are expensive.

And there's a deeper structural problem the pledge can't fix on its own: the market doesn't reliably reward secure products. Buyers say they care about security, and then they buy on features and price, like they always have. Until procurement genuinely penalizes insecure products — until "we can't buy this because your secure-by-design posture is weak" is a sentence someone actually says in a buying decision — the pledge runs on goodwill. Goodwill is a fine fuel for a pilot program. It's a terrible fuel for an industry transformation.

The other wobble: "secure by design" is easy to say and genuinely hard to define at the margins. Eliminating default passwords? Clear. Publishing a disclosure policy? Clear. But "reducing classes of vulnerabilities" is a spectrum, and every vendor gets to grade its own homework. Without independent verification, we're back to trusting the brochure — just a nicer brochure with better vocabulary.

What it means for buyers

If you're evaluating vendors, the pledge is useful but insufficient. Here's how I'd use it:

First, treat a signature as a starting point, not a credential. Ask what specifically changed in their development process since they signed. If the answer is vague, the signature is decorative.

Second, ask about the classes of vulnerabilities, not the count of CVEs patched. A vendor that eliminated SQL injection across their product line through parameterized queries and framework guardrails has done something real. A vendor that patched forty SQL injection CVEs last year has done something too — something that suggests the first thing never happened.

Third, look at the defaults. This is the fastest test in the book. Install the product the way a tired admin at 11 PM would install it — next, next, finish — and see what you get. If it's secure, they mean it. If it takes a forty-page hardening guide to make it safe, the pledge is a poster on the wall.

The pledge moved the conversation. That's not nothing. But conversations don't patch vulnerabilities, and signatures don't change defaults. The idea is right. The follow-through is still an open question — and it's the buyers, not CISA, who get to decide whether it was real.

The vendors are waiting to see if we meant it. We should probably decide.