Can our Managed SIEM providers please get their heads out of the 90's?

rant mode on   

(I had tried standard <> tags, and the CMS tried to process them!  LOL)
 
I've been a customer of SIEM (Security Incident and Event Monitoring) for about 30 years (cough), and have never had a "good" customer experience.  

SIEM are complex (and expensive) systems that closely integrate with every server/appliance/network device on the floor, and try to make sense of the data flowing through to identify security concerns.   This data is typically formatted  proprietary to the vendor of the source product.

When a vendor wants to implement SIEM in your infrastructure, for each server they enroll, the vendor asks about "use cases", or the set of rules that define what types of security events you should care about. 

"Mr Customer, how many failed login attempts to you want to capture before we alert you?"

As a customer, how the hell should I know?  What's the industry norm?  You're the SIEM expert, tell me what your other customers are doing!  This approach has stiffled progress/uptake in the industry.  SIEM is typically implemented grudgingly as an audit checkbox.  

"Ok, yes, we have SIEM, and things are reporting to it... CHECK... Next..."

 This is a very expensive and time consuming effort to acquire a check box, but this is also how the vendors are selling it.  Compliance sells products.

There is so much more that SIEM can and should do, like correlating firewall sessions with EndPoint Protection alerts.  Identifying patterns (anomalies) in VPN users activities, alerting on movement of data between rogue cloud applications (shadow IT)...  but those tasks take planning and scripting skills. Time and budget that an average Information Security team does not have.  So the tools get put in to fill the checkbox, and all of the capabilities they have sits idle.   (I hear you out there... prove me wrong, tell me YOUR good news story!)

Another typical issue with implementing SIEM is scaling/sizing of the SIEM infrastructure itself.  The vendors usually define the size of your SIEM based on incoming "events per second".  There are many calculators out there to help you determine size, but they don't tell you that a) this is a best case scenario, or b) EPS depends entirely on what you CHOSE TO LOG!

It's a rare event that you buy too much SIEM for your requirements.  SIEM is expensive, and most of us will err on the side of budget.... and then find out we spec'ed  3-4 times smaller than required.

rant mode off

So let me tell you a little story now of my most recent experience that turned all this on it's head:

There's a little "Managed Detection and Response" company, eSentire out of Cambridge Ontario, that I had seen at several trade shows. Fatigued by vendors proclaiming that their product/service was better than the next coming of Christ, I had watched them warily.  But heard good news from all sources.

I had an opportunity at one of my clients to replace an very non-functional implementation of Arcsight that one of Toronto's finest  managed security providers had failed to deliver appropriately. (I'll just leave that there)

We looked at possible opportunities for bringing SIEM back in-house, as well as talked to about a dozen Managed Security Service Providers, and the daunting conversation of use cases kept coming up time after time.  Vendor would ask us what we wanted to monitor, how many, how long, what's the alerting criteria, blah blah blah...  (insert Charlie Brown adults talking here)

In the mix, we had eSentire come in and present.  I had prepared my VP of IT and director of Security Operations as to the types of questions we would encounter, and typical responses regarding EPS per logging device, and use cases based upon product.

eSentire took the conversation in a completely new direction:

Us:  "Ok, tell us what we need to provide for use cases, and possibly some guidance on what makes sense...."

eSentire:   "We looked at your company, it's size, and market space.  We have dozens of similar customers as you.  Do you think your use cases might differ much from theirs?"

Us: "Ummm... no... probably not."

eSentire:  "Good, we can start there as a baseline, and monitor,  next?"

Us:  "Ok, what about Events Per Second, and storage?" 

eSentire:  "Based on existing customers, and the list of systems you want to integrate, we'll put a log collector in your infrastructure and monitor and manage it's capacity. "

 

That was nine months ago.  We signed up almost immediately, and the full implementation was a few weeks (not the typical 12-18 months I'm used to with those other systems).  We were getting reports daily, weekly, monthly, that made sense, and had executive presentations that I could actually take to my management.

We've also signed up for and are very happy with their Network Interceptor (Managed Intrusion Prevention) and Continuous Vulnerability Assessment services.




Resources:

Gartner: SIEM Use Cases

http://www.esecurityplanet.com/products/top-siem-products.html

Alienvault: What kind of logs do you need for an effective SIEM?

Gartner: Managed Detection and Response Companies
eSentire: Managed Detection and Response

SANS: Benchmarking SIEM
Why and How to calulate Events per Second
Solarwinds: Estimating Log Generation
Qradar: Sizing, Determining Events Per Second
A good EPS sizing chart and writeup from Buzzcircuit
https://www.emc.com/collateral/guide/11020-rsa-siem.pdf

Arcsight: Enterprise Security Manager