Nobody Ships Blind Anymore
Ask any software engineer whether they'd push a new service to production without logging, metrics, or alerting, and you'll probably get a funny look. Observability is table stakes in that world. If a service slows down, throws errors, or quietly stops responding, someone knows about it within minutes, usually before a customer does.
Now ask a detection engineer how they'd know if one of their rules stopped working last Tuesday.
When I worked for an MSSP we saw this more frequently than I’d like to admit. Usually we found out a rule broke when a customer was doing an investigation and found something that we should have. Those were never fun conversations to have and back then there wasn’t a solid solution for this problem, it was something that was just accepted as part of business. On top of that it usually took at least an hour to discover the root cause of the rule not catching what it was intended to catch.
Detection rules are code. They run on a schedule, they depend on upstream data, and they break in all the same ways software does. Yet most detection programs still treat them like something you write once, deploy, and trust. We've talked about this idea on the blog before under the name continuous validation. Going forward, we're calling it what it really is: Security Observability.
Why a New Name
Continuous validation is accurate, but it sounds like a task. Something you schedule, run, and check off. Observability describes a property of the system itself. An observable detection program is one where you can answer basic questions at any moment without going on a scavenger hunt:
- Are my rules running, and are they running without errors?
- Is the data they depend on still arriving?
- Does my coverage today match what I think it is?
- Are the alerts I'm generating worth an analyst's time?
The idea hasn't changed. The framing just makes it clearer that this is an always-on state, not a quarterly exercise.
The Problem: Build and Monitor Live in Different Worlds
In most programs I've seen, the build side and the monitoring side of detection engineering are completely separate. Rules get written in a repo or directly in the SIEM. They get tested, maybe peer reviewed, and deployed. Then they disappear into production, and whatever monitoring exists lives somewhere else entirely, if it exists at all, but it usually didn’t.
That split creates four blind spots.
Rule health. Searches time out, get skipped, hit resource limits, or fail after a platform upgrade renames a field. Sometimes SIEMs log these failures, but if they do the logs sit in an internal index nobody checks until something goes wrong.
Log source health. This is the dangerous one. When a log source goes quiet, every rule that depends on it goes quiet too. A rule that returns zero results because nothing bad happened looks exactly like a rule that returns zero results because it has no data to look at. One is good news. The other is a gap you don't know you have.
Coverage drift. Rules get disabled during an incident and never turned back on. Data sources get retired. Tuning exclusions pile up until a rule no longer catches the technique it was written for. The coverage map you presented last quarter slowly stops being true.
Rule efficacy. Some rules fire constantly and get ignored. Others haven't fired in a year, which could mean the environment is clean or could mean the rule is broken. Without tracking outcomes, there's no way to tell the difference.
Now, when I worked for the aforementioned MSSP they were at least monitoring log source health. This helped cut down on some of the misses that most organizations would fall victim to, but the other three blind spots were still present, and back then nobody there was thinking about coverage drift or a way to solve it.
Why the Obvious Fixes Fall Short
The usual answer is to build some dashboards in the SIEM or schedule periodic detection audits. Both help, and both miss the point.
Dashboards built on the side are disconnected from the rules themselves. They can tell you a log source dropped, but not which detections just lost their data or which ATT&CK techniques you can no longer see. Someone still has to connect those dots by hand, usually under pressure.
Audits have the opposite problem. They produce accurate, detailed answers that start going stale the day the report is finished. A rule that broke the week after the audit stays broken until the next one.
The underlying issue is context. The team that built a detection knows what it depends on, what it's supposed to catch, and what "healthy" looks like. When that knowledge lives in one tool and the monitoring lives in another, the context gets lost in between.
The Solution: One Platform, Both Sides
Security Observability works best when building and monitoring happen in the same place. When the rule you write is the same object you watch in production, the context comes along for free.
That means when a log source goes silent, you immediately see which rules are affected and which techniques just dropped out of your coverage. When a rule starts erroring after a platform change, you see it next to the rule's history and the change that likely caused it. When a detection hasn't fired in six months, you can check whether its data is still flowing before assuming the environment is clean. And when a noisy rule gets tuned, you can see whether the tuning quietly removed the behavior it was built to catch.
None of this replaces good detection engineering. It makes the work you've already done visible, so you can trust it.
Piecing It All Together
Software teams didn't adopt observability because it was trendy. They adopted it because running production systems without it meant finding out about problems from customers. Detection programs are in the same position today, except the customer who finds the gap is usually an attacker.
Building detections and monitoring them shouldn't be two separate jobs in two separate tools. At Rilevera, we built the platform so detection teams can do both in one place, because a detection you can't observe is a detection you're taking on faith.
