Putting Sysmon v9.0 AND/OR Grouping Logic to the Test

Sysmon v9.0 is now available, and now permits AND or OR statements across rules. This is interesting and I believe might help some of the issues I have experienced when writing very specific signature-like rules.

In this short post, I will share a little bit of my initial exploration of the new AND/OR capabilities provided by the latest version of Sysmon, and provide a basic example of a potential rule that can be written with the new features.

Before installing or updating the schema version of current Sysmon configs, I believe it is important to go over some of the current Sysmon filtering capabilities and limitations to understand the potential value of this update.

Weren’t AND and OR Statements Possible Already?

Yes, and No! According to the Sysmon documentation, within a rule, filter conditions have an OR behavior by default.

If I wanted to do the following:

Collect ProcessCreate events including only processes that their names end with cmd.exe or powershell.exe

I would simply write the following rule:

You can run Sysmon in DebugMode (-t) and point it (-i) to your custom Sysmon config to see how the rule matches events on specific conditions.

C:\WINDOWS\system32>sysmon.exe -t -i c:\Users\cbrown\Downloads\test.xml

Now, if you decide to add a new condition with a different field name to the rule, the OR behavior is still respected. The rule will still match on any command line argument provided by powershell or cmd besides whoami.

In other words, if I wanted to do the following:

Collect ProcessCreate events including processes that their names end with cmd.exe or powershell.exe and events where command line arguments contain whoami

I could easily write the following rule:

Make sure you run Sysmon in DebugMode again and run a few tests

However, if I wanted to:

Collect ProcessCreate events including processes that their names end with cmd.exe or powershell.exe and their command line arguments contain whoami ONLY

This would not be possible with Sysmon v8.04 and older versions.

The only AND statement that one was able to create until Sysmon V8.04 was by using Include and Exclude rules for the same ID (ProcessCreate, NetworkConnect, ImageLoad, etc).

For example, if I wanted to:

Collect ProcessCreate events including processes that their names end with cmd.exe or powershell.exe, and exclude events where the command line contains whoami

I would write the following rule:

You can verify that only events with the string whoami as part of the command line arguments are being excluded.

Then, what do I get with Sysmon 9.0?

According to the Sysmon documentation:

It is also possible to override the way that rules are combined by using a rule group which allows the rule combine type for one or more events to be set explicitly to AND or OR.

Let’s install Sysmon v9.0 and start running some tests.

Install latest Sysmon 9.0

• Download Sysmon Zip
• Install it via sysmon.exe -accepteula -i

Explore Sysmon event manifest and config schema

Every time there is a new Sysmon release, I highly recommend to document and understand potential changes to the overall event manifest or config schema.

Event Manifest/Schema

Here is where you can find what schema version you need to use for your new Sysmon configs and the event schema for each Sysmon event. This does not provide detailed information of the new logic or features that can be defined in a Sysmon configuration. You can infer some of the new features by new event IDs or fields added to the event schema, but that is not enough. For this update, we will be updating the schema version to 4.2 .

You can get the event manifest information by running:

sysmon -s

You can also access the new Sysmon 9.0 event log manifest xml here

Configuration Schema

Sysmon uses document type definition (DTD) to validate the XML configs that you create. As my teammate Matt Graeber said in this awesome post, Sysmon should ship with an XSD 😉.

You can pull the DTD schema with strings.exe, but Matt already did that for you and has it available here. If you take a look at it, you will see that:

RuleGroup can be applied to every Sysmon event

RuleGroup has a name and groupRelation options

• name values most likely will be showing under the current RuleName field since there is not a RuleGroup RuleName field in the event schema.
• groupRelation can be “and” or “or” and it is required when using RuleGroup.

As you can see, the DTD schema also gives you field names and the valid logic that can be applied to them. This is very useful when you want to explore the new capabilities provided by each Sysmon release.

Now that we understand a few of the changes to the event and config schema, let’s start testing this new version.

Basic “and” groupRelation

Let’s see if we can do something similar to what was not possible with 8.04:

Collect ProcessCreate events including processes that their names end with cmd.exe or powershell.exe and their command line arguments contain whoami ONLY

We have to create a RuleGroup and use the following schema for it:

<Sysmon schemaversion="4.2">
  <!-- Capture all hashes -->
  <HashAlgorithms>*</HashAlgorithms>
  <EventFiltering>
    <!-- Event ID 1 == Process Creation -->
    <RuleGroup name="group 1" groupRelation="and">
      <ProcessCreate onmatch="include">
        <Image condition="end with">cmd.exe</Image>
        <Image condition="end with">powershell.exe</Image>
        <CommandLine condition="contains">whoami</CommandLine>
      </ProcessCreate>
    </RuleGroup>
  </EventFiltering>
</Sysmon>

Start Sysmon in DebugMode and test a few commands:

That is working the same way as before so far!

Now, what if I do not use whoami but ping as part of the command line arguments? . Unlike the previous Sysmon versions, this time I did not get any matches because of the new “and” groupRelation logic.

Nice! That works as expected now! . We can have the following logic working:

• Match events with Image ending in cmd.exe AND whoami in CommandLine arguments ONLY
• Match events with Image ending in powershell.exe AND whoami in CommandLine arguments ONLY

What about multiple “and” groupRelations?

What if I want to create separate rule groups for each application (cmd and powershell). For example, I want to define two “and” groupRelations via two Inclusion filters:

<Sysmon schemaversion="4.2">
  <!-- Capture all hashes -->
  <HashAlgorithms>*</HashAlgorithms>
  <EventFiltering>
    <!-- Event ID 1 == Process Creation -->
    <RuleGroup name="group 1" groupRelation="and">
      <ProcessCreate onmatch="include">
        <Image condition="end with">cmd.exe</Image>
        <CommandLine condition="contains">whoami</CommandLine>
      </ProcessCreate>
    </RuleGroup>
    <RuleGroup name="group 2" groupRelation="and">
      <ProcessCreate onmatch="include">
        <Image condition="end with">powershell.exe</Image>
        <CommandLine condition="contains">whoami</CommandLine>
      </ProcessCreate>
    </RuleGroup>
  </EventFiltering>
</Sysmon>

When I try to debug the config, I get the following message:

Could not report error in RuleEngine: Multiple rule filters of the same type - Last error: The data is invalid.

Apparently, I cannot have two RuleGroup of the same filter type (i.e. Include-Include). What if I change the second filter type to an Exclusion and keep both “and” groupRelations applied to each RuleGroup?

<Sysmon schemaversion="4.2">
  <!-- Capture all hashes -->
  <HashAlgorithms>*</HashAlgorithms>
  <EventFiltering>
    <!-- Event ID 1 == Process Creation -->
    <RuleGroup name="group 1" groupRelation="and">
      <ProcessCreate onmatch="include">
        <Image condition="end with">cmd.exe</Image>
        <CommandLine condition="contains">whoami</CommandLine>
      </ProcessCreate>
    </RuleGroup>
    <RuleGroup name="group 2" groupRelation="and">
      <ProcessCreate onmatch="exclude">
        <Image condition="end with">powershell.exe</Image>
        <CommandLine condition="contains">whoami</CommandLine>
      </ProcessCreate>
    </RuleGroup>
  </EventFiltering>
</Sysmon>

That seem to work just fine!

Once again, we can have two groupRelations on the same ID (ProcessCreate, NetworkConnect, ImageLoad, etc), but not multiple filters (Inclusion-Exclusion) of the same type.

Can I mix “and” and “or” groupRelations?

Let’s try that:

<Sysmon schemaversion="4.2">
  <!-- Capture all hashes -->
  <HashAlgorithms>*</HashAlgorithms>
  <EventFiltering>
    <!-- Event ID 1 == Process Creation -->
    <RuleGroup name="group 1" groupRelation="and">
      <ProcessCreate onmatch="include">
        <Image condition="end with">cmd.exe</Image>
        <CommandLine condition="contains">whoami</CommandLine>
      </ProcessCreate>
    </RuleGroup>
    <RuleGroup name="group 2" groupRelation="or">
      <ProcessCreate onmatch="exclude">
        <Image condition="end with">powershell.exe</Image>
        <CommandLine condition="contains">whoami</CommandLine>
      </ProcessCreate>
    </RuleGroup>
  </EventFiltering>
</Sysmon>

So far, Yes! The config is valid, so you can do that too.

However, our two rules did not match on our initial basic conditions 🤔

That is because we are defining an AND-Inclusion with whoami as part of the command line arguments, and an OR-Exclusion with whoami as part of the command line arguments as well. Therefore, the OR-Exclusion will override the initialCommandLine condition. Be careful!

How is this useful?

So far, I find these new features useful for a more targeted signature-like collection strategy approach.

For example, if I wanted to do the following in previous versions:

Collect ProcessCreate events including processes that their names end with powershell.exe and their ParentProcess name ends with WmiPrvSe.exe

I had to write a rule like this:

<Sysmon schemaversion="4.1">
  <!-- Capture all hashes -->
  <HashAlgorithms>*</HashAlgorithms>
  <EventFiltering>
    <!-- Event ID 1 == Process Creation -->
    <ProcessCreate onmatch="include">
      <Image condition="end with">powershell.exe</Image>
      <ParentImage condition="contains">WmiPrvSe.exe</ParentImage>
    </ProcessCreate>
  </EventFiltering>
</Sysmon>

However, as we already know, that rule will basically collect every event where the Image value of a process ends with powershell.exe and not necessarily when the ParentImage ends with WmiPrvSe.exe. Therefore, you will collect more events than what you originally want.

In the image below, we can see how there is a match, but by the Image condition only, and not by the combination of Image and ParentImage.

With Sysmon v9.0, you can re-write the rule the following way:

<Sysmon schemaversion="4.2">
  <!-- Capture all hashes -->
  <HashAlgorithms>*</HashAlgorithms>
  <EventFiltering>
    <!-- Event ID 1 == Process Creation -->
    <RuleGroup name="Potential Lateral Movement via WMI and PS" groupRelation="and">
      <ProcessCreate onmatch="include">
        <Image condition="end with">powershell.exe</Image>
        <ParentImage condition="end with">WmiPrvSe.exe</ParentImage>
      </ProcessCreate>
    </RuleGroup>
  </EventFiltering>
</Sysmon>

and get the following results:

You can test if you get other ProcessCreate events where the Image value of a process ends with powershell.exe only.

As you can see in the image above, all the other events with Image ending with powershell.exe and without ParentImage ending with WmiPrvSE.exe are excluded.

What does the new event look like?

The event I captured above with the “Potential Lateral Movement via Wmi and PS” rule was the following:

- <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
    - <System>
        <Provider Name="Microsoft-Windows-Sysmon" Guid="{5770385F-C22A-43E0-BF4C-06F5698FFBD9}" /> 
        <EventID>1</EventID> 
        <Version>5</Version> 
        <Level>4</Level> 
        <Task>1</Task> 
        <Opcode>0</Opcode> 
        <Keywords>0x8000000000000000</Keywords> 
        <TimeCreated SystemTime="2019-02-20T05:48:21.384221800Z" /> 
        <EventRecordID>5051065</EventRecordID> 
        <Correlation /> 
        <Execution ProcessID="2400" ThreadID="3272" /> 
        <Channel>Microsoft-Windows-Sysmon/Operational</Channel> 
        <Computer>DESKTOP-LFD11QP.RIVENDELL.local</Computer> 
        <Security UserID="S-1-5-18" /> 
    </System>
    - <EventData>
        <Data Name="RuleName">Potential Lateral Movement via WMI and PS</Data> 
        <Data Name="UtcTime">2019-02-20 05:48:21.373</Data> 
        <Data Name="ProcessGuid">{1C9FDC81-EA25-5C6C-0000-0010C2232900}</Data> 
        <Data Name="ProcessId">572</Data> 
        <Data Name="Image">C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe</Data> 
        <Data Name="FileVersion">10.0.17134.1 (WinBuild.160101.0800)</Data> 
        <Data Name="Description">Windows PowerShell</Data> 
        <Data Name="Product">Microsoft® Windows® Operating System</Data> 
        <Data Name="Company">Microsoft Corporation</Data> 
        <Data Name="CommandLine">powershell -c whoami</Data> 
        <Data Name="CurrentDirectory">C:\WINDOWS\system32\</Data> 
        <Data Name="User">RIVENDELL\cbrown</Data> 
        <Data Name="LogonGuid">{1C9FDC81-D3A8-5C6C-0000-0020B9570400}</Data> 
        <Data Name="LogonId">0x457b9</Data> 
        <Data Name="TerminalSessionId">1</Data> 
        <Data Name="IntegrityLevel">Medium</Data> 
        <Data Name="Hashes">SHA1=1B3B40FBC889FD4C645CC12C85D0805AC36BA254,MD5=95000560239032BC68B4C2FDFCDEF913,
        SHA256=D3F8FADE829D2B7BD596C4504A6DAE5C034E789B6A3DEFBE013BDA7D14466677,IMPHASH=741776AACCFC5B71FF59832DCDCACE0F</Data> 
        <Data Name="ParentProcessGuid">{1C9FDC81-D3A0-5C6C-0000-001033BD0200}</Data> 
        <Data Name="ParentProcessId">2580</Data> 
        <Data Name="ParentImage">C:\Windows\System32\wbem\WmiPrvSE.exe</Data> 
        <Data Name="ParentCommandLine">C:\WINDOWS\system32\wbem\wmiprvse.exe -secured -Embedding</Data> 
    </EventData>
  </Event>

Something to remember is that the new schema does not provide a new field for the RuleGroup feature. The value defined for the RuleGroup name variable is passed to the already existing RuleName field as shown in the image below:

Can I add multiple tags like before?

Yes, you can do the same thing as in the previous versions and add multiple tags. You can still add ATT&CK tags 😉 to your RuleGroups.

<Sysmon schemaversion="4.2">
  <!-- Capture all hashes -->
  <HashAlgorithms>*</HashAlgorithms>
  <EventFiltering>
    <!-- Event ID 1 == Process Creation -->
    <RuleGroup name="technique_id=T1047,tactic=Execution,platform=Windows" groupRelation="and">
      <ProcessCreate onmatch="include">
        <Image condition="end with">powershell.exe</Image>
  <ParentImage condition="end with">WmiPrvSE.exe</ParentImage>
      </ProcessCreate>
    </RuleGroup>
  </EventFiltering>
</Sysmon>Your event will look like before:

You will get the same RuleName syntax as shown below:

Can I still add tags at the field-condition level?

As you might have noticed, when you use the RuleGroup name for tagging, you are not setting a tag for each field-condition. You are tagging the whole group of field-conditions. However, you can still provide tags for each field-condition. Just remember that if you set both RuleGroup name and Field name tags, the tag at the field-condition level takes precedence.

How do I parse multiple RuleName values?

If you are not parsing the RuleName field already, and you are using Logstash to transform your data, you could use the Logstash KV filter plugin to parse multiple tags (foo=bar syntax). I showed this here before.

Logstash KV Filter Config

According to Logstash documentation, the KV filter plugin helps to parse messages (or specific event fields) which are of the foo=bar variety.

• source => The field to perform key=value searching on
• field_split => A string of characters to use as single-character field delimiters for parsing out key-value pairs.
• value_split => A non-empty string of characters to use as single-character value delimiters for parsing out key-value pairs.
• prefix => A string to prepend to all of the extracted keys.
• transform_key => Transform keys to lower case, upper case or capitals.

filter {
  if [log_name] == "Microsoft-Windows-Sysmon/Operational"{
    if [RuleName] {
      kv {
        source => "[RuleName]"
        field_split => ","
        value_split => "="
        prefix => "rule_"
        transform_key => "lowercase"
      }
    }  
  }
}

What do I have to change in my Sysmon configs if I decide to upgrade to Sysmon v9.0?

If you decide to upgrade to version 9.0 and use the RuleGroup features, you will have to apply the following changes:

• Schema version needs to changed to 4.2
• Any rule (Include or Exclude) will need to be under a RuleGroup and have a groupName (and|or) defined. This is because if you leave your current rules the way they are, the default OR behavior that you had within your field-condition filters will turn into an AND behavior just like the initial basic “and” example I showed you earlier.
• If you are not changing the logic itself of your config, you will just need to add “or” group relations to each of your rules. For example, If you already have ProcessCreate rules, just add a RuleGroup with a groupRelation “or” as shown below:

<RuleGroup name="name of rule" groupRelation="or">
  <ProcessCreate onmatch="include">
   ..
  </ProcessCreate>
</RuleGroup>

That will maintain the logic of your current rules. Make sure you do that to every ID (ProcessCreate, NetworkConnect, ImageLoad, etc.) and filter type (Include|Exclude)

If you decide to upgrade to version 9.0, but NOT use the new RuleGroup features, you can keep the schema version to 4.10 or below, and your config will still work like before (default OR behavior within conditions respected).

That’s it! I hope this post was useful for those wondering about the new features provided by the latest Sysmon release (v9.0). This was just my first attempt to understand the new capabilities, and I already see a few use cases for organizations to reduce the amount of data produced by very permissive configurations. Don’t get me wrong, I love to collect as much as I can after prioritizing what would provide enough context to validate the detection of adversarial techniques and sub-techniques, but it is also useful to be able to be specific in some areas and reduce the noise.

References

https://posts.specterops.io/categorizing-and-enriching-security-events-in-an-elk-with-the-help-of-sysmon-and-att-ck-6c8e30234d34

https://posts.specterops.io/working-with-sysmon-configurations-like-a-pro-through-better-tooling-be7ad7f99a47

https://docs.microsoft.com/en-us/sysinternals/downloads/sysmon#event-filtering-entries

[UPDATE 02–22–19] Added extra information about keeping schema version lower than 4.2 to NOT use new features per feedback from @S0xbad1dea