I probably should have posted something sooner, but I’ll speaking with Rod Trent and Cameron Fuller about SCOM and Sentinel Integration. If you’re interested in seeing a demo of the product in action and getting a chance to ask some questions, drop in. It’s a digital conference, so it’s a matter of clearing some time on your calendar. If you’re interested in registering, you can do so here.
This link has general information on the event itself.
I look forward to meeting you.
For the Sentinel people, I’m guessing that most of what I’m writting in here is not really news, but for those of us who have more experience in SCOM, I’m going to post some useful KQL to help validate that data is flowing properly.
From a time perspective, the data flow is not instantaneous. That said, if you have generated an alert, you should see it in Sentinel within a few minutes. The same goes for events. The wait should not be long.
There are a number of tables in Sentinel that you can use KQL to enumerate. Our data, however, is only going to two tables (the Alert table and Events table). Alerts, hopefully, are somewhat sporadic, but I’ll note that the namespace in the alert name should be unique enough to ensure that these alerts are there. A simple KQL query such as this should enumerate those items:
|
Alert |
Please note that in both alerts and events, pay close attention to the Time Range in Sentinel. The default is 24 hours, but you can configure it, and Sentinel will also remember your custom configurations.
Events are a bit harder to track. Those will come in as event IDs no different than any other event. They also will fill up a bit faster, so you should see them. But that said, you can filter by what is coming through SCOM. The Source System with our events will always say OpsManager. That’s our first filter. The MP out of the box also sends 4624, 4625, and 4688 by default. The other events need the minimal configuration MP that is included by default in order to start seeing them. Once you have everything configured, testing event flow should be straight forward, as 4624 events are pretty common. This is a KQL query I’d run against Sentinel to ensure you’re getting data flow:
|
Event |
Syslog capabilities within SCOM is something that’s been around for a while, but not much has been written on the topic. Syslog is primarily used for Linux/Unix systems, but other applications/systems have this capability as well. When SCOM was built, it was given syslog capabilities, but Microsoft did not provide much in terms of documentation on how to do this. There were, however, templates in the console.
Given the idea behind this solution was to simply on-prem deployment, one of the big gaps that needed addressing is syslog, given that there really is not a good on-prem solution for syslog and Sentinel. As such, enabling this capability was a high priority for us as we developed this solution.
As it turns out, this wasn’t too hard to do. The biggest thing that I need to address is targeting. Interestingly enough, the trick to making this work is simply to create an empty group and target it for Syslog monitoring. For the record, you can find that group in the On Prem Security Monitoring for Sentinel MP. The name of the group is as follows:
On Prem Security Monitoring: Syslog Computer
If you want to create additional syslog alerts on top of what I’m documenting here, you can either target that group or create an empty group of your own. Also of note, SCOM does not care if it has discovered your system or not. Syslog alerts will alert regardless if you have discovered the system.
The other big thing to note is that presently there are no libraries to send Syslog events into the cloud. There were some at one time, but they’ve apparently been removed. As such, the only syslog data we can send into the cloud is alert data.
From the client side, all you need to do is tell systems using syslog to send their syslog events to your SCOM management server. That’s it. There’s also a cool syslog PowerShell capability that you can download to test.
By default, we are sending the following facilities into Azure: 4, 10, 13, and 14.
Any other facility is not being sent. If you want to send more, you’ll need to create custom alert rules and you’ll need to following the instructions in this post to send them into the cloud.
This is a bit of a longer post, but if you want to send custom data into the cloud, the solution described here will let you do it. To answer the first and obvious question, why? Simply put, this solution is only sending a minimal amount of security events into Log Analytics. You can find the full list here, and you will quickly notice that we aren’t sending everything there. Given that cost is a primary reason why people may want this, I felt it wise to allow the user of the product to have more control over what does and does not go into the cloud. If you want to send more data in the cloud, then this post is for you. There are also reasons outside of security that you may want to consider doing this. Azure Monitor, for instance can tap into the same Log Analytics workspace… so if you want Azure Monitor to alert off of a non security event, you can use this same concept to selectively forward those alerts and/or events into the cloud.
Prior to this week, there really wasn’t much in terms of options to pick and choose what goes into the cloud. Unfortunately, for customers that want some control over the data they send into Log Analytics, one will need to have a basic understanding of XML, as the options to do this are not exposed via the SCOM console, and the alert mechanism wasn’t even a public write action that could be referenced by other authors.
That said, with the On Prem Security Monitoring for Sentinel solution, we now have an option. First, with the help of Tyson Paul, we were able to identify the correct libraries to use. We then took the internal only XML and created a public module in this solution that can be referenced by any author. It will create a dependency on the On Prem Security Monitoring for Sentinel solution, but that XML is also reusable, so you can grab it yourself if you’d rather not install this solution. If that’s the direction you want to go, do the following
First, export a copy of On Prem Security Monitoring for Sentinel to XML. I would note that hopefully, the full XML will be published on systemcenter.wiki, but for now, that’s what you’ll need to do.
Second, copy the XML from the following modules and insert into your management pack. I would recommend doing a find and replace to change the IDs if you do this so as not to run into namespace issues.
Note in that scenario, there are dependencies, so you’ll need to have your aliases correct in order to get that to work… at that point, you can use the write alert to cloud as a write action for any of your custom rules/monitors that generate alerts.
That’s the hard way in my opinion 
If you’re comfortable using the On Prem Security Monitoring for Sentinel, you can simply paste the following write actions and change the highlighted aliases to match your references.
For event collection:
| <WriteAction ID=”CloudEvent” TypeID=”MIT!Microsoft.SystemCenter.CollectCloudGenericEvent”> |
Note that MIT is the alias I’m using for the Microsoft.IntelligencePacks.Types management pack. It’s downloaded when you connect SCOM to Log Analytics. You wont’ find this publicly available for download anywhere.
For alerts:
|
<WriteAction ID=”CloudAlert” TypeID=”Security!SecurityMonitoringForSentinel.WriteAlertToCloud”> |
There is a bit more more to unpack for alerts. Security is the alias for the On Prem Security Monitoring for Sentinel MP. You may need to change that accordingly. Your custom alert will have it’s own custom name and if you want the same alert message to go into Sentinel, you’ll need to use that. Obviously things like custom fields, alert parameters, priority, severity, and suppression are all configurable. You can use the same fields in the SCOM alert if you’d like. This will work. What I did was to copy the write action for the alert that I had written for SCOM into a second write action. I then replaced the write action ID tag with the one in the above box. It really depends on how you want to customize it. I’m pretty lazy, so that was my solution.
I have included a sample management pack in the SCOM portion of the repository should you want to do this.
Cameron Fuller at Catapult Systems reached out to me during SCOMathon 2021 with an idea of how to integrate Security Monitoring and Sentinel. This has been on my bucket list for a while, but learning the KQL piece and creating the necessary workbooks and/or playbooks would be a pretty massive undertaking at this point, not to mention a learning curve. Fortunately he had a much better way. For the last couple months I’ve been working with him and Rod Trent to make this a reality and I can finally say that we have a finished product. You will need to do a migration to get that to work as the cloud version of this solution has a dependency on a management pack obtained from connecting SCOM to Log Analytics.
A couple things that are very important to note. I chose to release this as two separate products. If you’re currently using Security Monitoring, you’ll need to remove it and install On Prem Security Monitoring for Sentinel. This won’t be a major lift for your SCOM team. I intentionally kept the namespaces the same so that all you have to do is change the management pack reference in your customizations management pack(s).
The original version and Sentinel versions are nearly identical in what they do. There are no features in the on-prem version that are missing from the version that integrates with Azure. The alerts and events are the same, which is good. The only real difference is that the Sentinel version will send the alert and corresponding events to the cloud as well as to your SCOM console. There are a few additional features to the Sentinel version which will be documented here. By default logon events and 4688 events are being forwarded to the cloud only, not to the db/dw. Additional rules are also disabled by default to forward other events required by Sentinel, and I’ve included an unsealed MP enabling them if this is something you are interested in. Also, I’ve enabled SCOM’s syslog capabilities with this solution. I’ll keep a running list of this management pack’s features here.
It is also worth noting that I intentionally designed this solution so that it cannot run side by side. There’s a reason for this, namely that security monitoring is already combing security logs which is something SCOM doesn’t do well. We saw what happened with the 2019 UR 3 patch, and while I could have changed the namespace to allow it, it would also have meant plenty of rework, and duplicate collection/alerting rules. I didn’t like that, so you’re going to use one or the other. If you’re already connected to Log Analytics, you have the ability to put this data in Sentinel using the OnPremSecurityMonitoringForSentinel management pack.
Before going into the details of the how, I want to emphasize the value proposition for this management pack. Simply put, Sentinel charges by data ingested. This is a blocker to Sentinel adoption because ingesting security log data across hundreds if not thousands of servers CAN GET VERY EXPENSIVE. Much of what I’m looking for in Security Monitoring is already in Sentinel, and of course Sentinel can do far more than SCOM can and looks for far more indicators that what I’ve written in this solution. Because of this, I think the value proposition for this management pack is enormous. What we are effectively doing with this solution is applying an inbound filter to data being put into Sentinel. Instead of sending everything and letting Sentinel do its thing, we’re sending a lot less data into the cloud, essentially allowing you the customer to do a much more inexpensive try before you buy so to speak while still giving Sentinel a lot of very relevant data. I would personally still argue that there’s value in sending all data into Sentinel, or at the very least your domain controllers, because with question it’s looking for far more than what’s in this management pack, not to mention that it can correlate and alert off of the data in ways that SCOM cannot do. Then again, I’m both a Microsoft employee and a share holder, so take my little conflict of interest with a grain of salt.
In summary, it does allow you to put a lot of relevant data into Sentinel. It certainly doesn’t include everything, but it does give Sentinel a window into your on-prem environment that you may not currently have at a fraction of the cost.
You can find the download here.
You can find the install instructions here.
You can find the documentation here.
You can download the solution here.
You can find installation instructions here.
You can find instructions for sending additional data into the cloud here.
The current management pack contains all of the features of Security Monitoring. This information can be found at this link. Additional write actions are included to send alert data and event data that the MP already collects into the cloud. I am following the work done here.
The following additional features have been added to this management pack.
I do highly recommend taking a look at the components to this that Rod has written. They are available for download from the same location. You can find the write up on that here.
That’s it for now.
Step 1, if you haven’t already done this, connect your SCOM environment to a Log Analytics Workspace. Ideally this should be the same workspace that Sentinel is using. However, if not, Sentinel can be extended to connect to other workspaces. There are also a couple of things to watch out for here:
Once connected, ensure that the advisor MPs download. If you are having problems at this stage, you should troubleshoot. This is a good place to start.
Step 2, From Log Analytics, add the Alert Management solution to your workspace. You can find it doing a search for Operations Manager in the solutions section of Log Analytics.
Step 3, back up your security monitoring overrides management pack(s)
Step 4, uninstall Security Monitoring
Step 5, Install On Prem Security Monitoring for Sentinel. Optionally, add the unsealed management pack to enable forwarding of security events specifically required by Sentinel. This will potentially add some costs to the deployment, but it should remain considerably lower than if you configured the Microsoft Monitoring Agent to talk directly to Log Analytics as is typically done today.
Step 6, Create an override of any of the rules/monitors in this pack and store in a custom MP. It doesn’t matter which one, you’re going to delete this. You simply need the management pack information in the references section of this customization.
Step 7, export this custom MP to disk and then delete it from SCOM.
Step 8, From this custom MP, you’re going to want to copy the reference information. The namespace between these MPs is the same, which means all of the rules and overrides for your original customization can be re-imported into SCOM, but you’ll need to change the refence info. This is a screenshot of what you’ll need to grab from the custom MP created in step 6. You only need the ID, Version, and Public key token info. Copy those 3 lines highlighted (they will be different from this screenshot, so do not type them).
Step 8, edit the custom management pack exported in Step 2. DO NOT REPLACE THE ALIAS (highlighted below), replace only the ID, Version, and Public key token info taken from the above pack. These are the same 3 items that you copied.
Step 9, re-import your old overrides management pack back into SCOM.
Step 10, You need to add your SCOM management server(s) as managed computers in order to ensure that they send data into the cloud as doing this ahead of downloading MPs has been known to cause issues with downloading MPs. I would note that I have been able to successfully get data to sync to the cloud without doing this… It has also, however, been the fix that I need in order to get data to move to the cloud. I don’t have a good explanation for that at the moment, but this timing is VERY important. You must have the advisor MPs before you do this step. You are welcome to skip this piece if you’re getting data into the Alert and Event tables from SCOM, but if you aren’t getting data from SCOM in the cloud, I’d revisit it.
Step 11, import Rod’s components on the Sentinel side. He has written some really good analytics rules and had a lot more coming.
Optional. If you want to forward additional events/alerts into the cloud, review this post.
That’s it. I kept the namespaces the same between the management packs to allow for ease of migration should you choose to move in that direction. You can have this solution up and running in a few minutes time.
You can find the download at this location.
There isn’t much to this change. Just some minor fixes and a couple of requests. Due to the nature of the SCOM 2019 UR 3 issues with this management pack, I have turned off all collection rules by default. This means that reporting on things like legacy protocols won’t work until they are turned on.
I was informed by the SCOM product team that they have released a hotfix for 2019 UR3 affecting security Monitoring. It is for the time being a hotfix that reverts 2019 back to it’s UR2 behavior. As I understand it, there are plans to revisit this issue, but they were unable to fix everything and have to go back to the drawing board so to speak on the issue.
You can find the hotfix here.
I’ll be releasing the next version of Security Monitoring shortly, though I’m trying to time it with something else altogether. Here’s your teaser…
Nathan Gau's SCOM blog 