In years past at Black Hat here in Las Vegas, there was row after row of hardware, then, in later years, row after row of software for your workstation. Now there’s row after row of middleware designed to interpret all that data in real time and try to make sense of it all, to find the security needle in the haystack that suggests attack patterns we should act upon amidst the daunting amount of bits, bytes, hard drives, server rooms and the resulting deluge of unstructured data chaos.
"For years we’ve been piling up data in the server room somewhere – backups of backups of backups – in giant piles of hard drives."
If you don’t know where your data is, it’s a lot like not having any. For years we’ve been piling up data in the server room somewhere – backups of backups of backups – in giant piles of hard drives. Now we’re rethinking it all – increasingly trying to share resources among small, purpose-built compute instances that make every effort and NOT creating huge droves of duplicate data (whether multiple full operating systems or actual unique data sets).
You don’t even need a whole operating system anymore. We started by slicing operating systems into smaller virtual machines, and now that has continued to smaller sub-chunks of isolated containers like Docker that all co-exist on a parent server with loads of memory, processor and hard drive banks. We’re shrinking our data to what’s essential so as to manage the deluge.
Rather than living with a couple cabinets full of hard drives, we’re optimizing everything and finding new virtual buckets (that we often rent from others) for the data, and hoping some middleware system just magically takes care of everything.
Except when it doesn’t. Adding new layers of abstraction tends to reduce the fidelity of the data you’re receiving to what the developers of the middleware were told by marketing that average customers would buy. So now you’re betting your security on another layer of marketing-driven doo-dads.
It’s daunting now to understand the new ways to secure data which could be physically located pretty much anywhere. So we have to think about protecting data by using a granular container-based approach. If we keep small bits of data contextualized, containerized and protected, we can worry less about where it actually might reside.
Unless it all stops working. Then it’s precariously difficult to understand where your data is, or if it still exists at all. It’s dangerously difficult nowadays to determine if some big data fault might happen and suddenly leave you with absolutely nothing, or a half-baked recovery toolset that you just have to hope works.
It used to be that you could walk into the server room and grab a hard drive and try to restore it. Now that’s just not the case. At that time you could use standard tools to get your data back if you had to. And while that may have been a daunting process, middleware guarantees you don’t get your information back at all unless the vendor’s toolset happens to work really, really well in all situations in which you might find yourself.
And what If you don’t? The likelihood that a provider will spin up a new tool feature in time for you to do meaningful damage control is virtually non-existent.
The best you can hope to do is focus on middleware that’s built by people who really get security, and not by a marketing department. You then have a much higher chance of getting your data back, or having meaningful functionality that allows the best up times anyway. In short, while you need middleware to manage the whole deluge, choose wisely. Otherwise, you might be fine for a while, but when “bad things” happen, you’ll wish you had.
