Kloud Managed Services team responds to the global (WannaCry) ransomware attack

It’s been a busy week for our Managed Services team, who have collaborated together across our globally distributed team to provide a customer first security response to this most recent ransomware attack.

From the outset our teams acted swiftly to ensure the several hundred servers, platforms & desktop infrastructure under our management were secure.

Based on our deep knowledge of our customers, their environments and landscapes Kloud Managed Services were able to swiftly develop individual security approaches to remediate discovered security weak points.

In cases where customers have opted for Kloud’s robust approach to patch management the process has ensured customer environments were proactively protected against the threat. For these customers no action was required other than to confirm the relevant patches were in place for all infrastructure Kloud manage.

Communication is key in responding to these types of threats and the Managed Services team’s ability to collaborate and share knowledge quickly internally and then externally with our customers has ensured we remain on top of this threat and are ever vigilant for the next one.

Watching the watcher – Monitoring the EC2Config Service

EC2Config service is a nifty Windows service provided by Amazon that performs many important chores on instances based on AWS Windows Server 2003-2012 R2 AMIs. These tasks include (but are not limited to):

  • Initial start-up tasks when the instance is first started (e.g. executing the user data, setting random Administrator account password etc)
  • Display wallpaper information to the desktop background.
  • Run Sysprep and shut down the instance

More details about this service can be found at Amazon’s webpage

Another important aspect of EC2Config service is that it can be configured to send performance metrics to CloudWatch. Example of these metrics are Available Memory, Free Disk Space, Page File Usage to name a few. The problem we faced is sometimes this service will either stop or fail to start due to a misconfigured configuration file. Having this service running all the time was critical for monitoring and compliance reasons.

To make sure that this service was running and publishing metrics to CloudWatch, we came up with a simple solution. We used a Python script written as a Lambda function to query Windows performance metrics for the last 10 minutes (function scheduled to run every 30-minute interval configurable through Lambda Trigger) and if the metric was missing, send an alert.

Following is the code written for this purpose. The salient features of the code are:

  1. The function lambda_handler is invoked by Lambda
  2. Variable are initialised, currently these are coded in to the function but they can also be parametrized using Environment Variables feature of a Lambda function
  3. Ec2 and CloudWatch objects are initialised
  4. Running Instances are retrieved based on “running” filter
  5. If an Instance is running for less than the period requested than ignore this instance (this avoids false alarms for instances started in the last few minutes)
  6. Cloudwatch metric ‘Available Memory’ for the instance is retrieved for last 10 min. This can be substituted with any other metric name. Please also take note of the Dimension of the metric
  7. Datapoint result is inspected, if no Datapoint is found this instance is added to a list (later used for alert)
  8. If the list has some values, an alert is sent via SNS topic

#
#
# AWS Lambda Python script to query for Cloudwatch metrics for all running 
# EC2 instance and if unavailable send a message through an SNS topic
# to check for EC2Config service
#
# Required IAM permissions:
#   ec2:DescribeInstances
#   sns:Publish
#   cloudwatch:GetMetricStatistics
#
# Setup:
# Check these in the code (Search *1 and *2): 
#   *1: Confirm details of the parameters
#   *2: Confirm details of the dimensions
#

from __future__ import print_function
import boto3,sys,os
from calendar import timegm
from datetime import datetime, timedelta

def check_tag_present(instance, tag_name, tag_value):
    for tag in instance.tags:
        if tag['Key'] == tag_name:
            if tag['Value'] == tag_value:
                return True

    return False

def send_alert(list_instances, topic_arn):
    if topic_arn == "":
        return

    instances = ""

    for s in list_instances:
        instances += s
        instances += "\n"

    subject = "Warning: Missing CloudWatch metric data"
    message = "Warning: Missing CloudWatch metric data for the following instance id(s): \n\n" + instances + "\n\nCheck the EC2Config service is running and the config file in C:\\Program Files\\Amazon\\Ec2ConfigService\\Settings is correct."
    client = boto3.client('sns')
    response = client.publish(TargetArn=topic_arn,Message=message,Subject=subject)
    print ("*** Sending alert ***")

def lambda_handler(event, context):
    
    # *1-Provide the following information
    _instancetagname = 'Environment' # Main filter Tag key
    _instancetagvalue = 'Prod'       # Main filter Tag value
    _period = int(10)                # Period in minutes
    _namespace = 'WindowsPlatform'   # Namespace of metric
    _metricname = 'Available Memory' # Metric name
    _unit = 'Megabytes'              # Unit
    _topicarn =  ''                  # SNS Topic ARN to write message to
    _region = "ap-southeast-2"       # Region

    ec2 = boto3.resource('ec2',_region)
    cw = boto3.client('cloudwatch',_region)

    filters = [{'Name':'instance-state-name','Values':['running']}]

    instances = ec2.instances.filter(Filters=filters)

    now = datetime.now()

    print('Reading Cloud watch metric for last %s min\n' %(_period))

    start_time = datetime.utcnow() - timedelta(minutes=_period)
    end_time = datetime.utcnow()

    print ("List of running instances:")

    list_instances=[]

    for instance in instances:
        
        if check_tag_present(instance, _instancetagname, _instancetagvalue)==False:            
            continue #Tag/Value missing, ignoring instance

        print ("Checking ", instance.id)

        i=1
        
        date_s=instance.launch_time
        date_s=date_s.replace(tzinfo=None)
        new_dt = datetime.utcnow() - date_s

        instance_name = [tag['Value'] for tag in instance.tags if tag['Key'] == 'Name'][0]
        minutessince = int(new_dt.total_seconds() / 60)
        
        if minutessince < _period:
            print ("Not looking for data on this instance as uptime is less than requested period.\n")
            continue

        metrics = cw.get_metric_statistics(
            Namespace=_namespace,
            MetricName=_metricname,            
            Dimensions=[{'Name': 'InstanceId','Value': instance.id}], 
            StartTime=start_time,
            EndTime=end_time,
            Period=300,
            Statistics=['Maximum'],
            Unit=_unit
        )
        
        datapoints = metrics['Datapoints']

        for datapoint in datapoints:
            if datapoint['Maximum']:
                print (i,")\nDatapoint Data:",datapoint['Maximum'],"\nTimeStamp: ",datapoint['Timestamp'],"\n")
                i+=1
            else:
                print ("Cloudwatch has no Maimum metrics for",_metricname,"instance id: ", instance.id)

        if i == 1: #No data point found
            print ("Cloudwatch has no metrics for",_metricname," for instance id: ", instance.id)
            list_instances.append(instance_name + " (" + instance.id+ ")" + ", CW Server Name: " + cw_server_name)
            
        print ("=================================================\n")

    if len(list_instances) > 0:
        send_alert(list_instances, _topicarn)

Please note: The function needs some permissions to execute, so the following policy should be attached to lambda function’s role:

{
    "Version": "2012-10-17",
    "Statement": [{
        "Sid": "Stmt1493179460000",
        "Effect": "Allow",
        "Action": ["ec2:DescribeInstances"],
        "Resource": ["*"]
    },
    {
        "Sid": "Stmt1493179541000",
        "Effect": "Allow",
        "Action": ["sns:Publish"],
        "Resource": ["*"]
    },
    {
        "Sid": "Stmt1493179652000",
        "Effect": "Allow",
        "Action": ["cloudwatch:GetMetricStatistics"],
        "Resource": ["*"]
    }]
}

VPC ( Virtual Private Cloud) Configuration

Introduction

This blog is Part 01 of a 02 part series related to custom VPC configurations

Part 01 discusses the following scenario

  • Creating a VPC with 02 subnets ( Public and Private )
  • Creating a bastion host server in the public subnet
  • Allowing the Bastion host to connect to the servers in the Private Subnet using RDP.

Part 02 will discuss the following

  • Configuring NAT Instances
  • Configuring VPC Peering
  • Configuring VPC flow Logs.

What is a VPC

VPC can be described as a logical Datacenter where AWS resources can be deployed.

The logical datacenter can be connected to your physical datacenter through VPN or direct connect. Details (https://blog.kloud.com.au/2014/04/10/quickly-connect-to-your-aws-vpc-via-vpn/)

This section deals with the following tasks.

  • Creating the VPC
  • Creating Subnets
  • Configuring Subnets for internet access

1 Creating the VPC

The following steps should be followed for configuring VPC. we can use the wizard to create a VPC but this document will focus on the detailed method where every configuration parameter is defined by the user.

Step 01.01 : Logon to the AWS console

Step 01.02 : Click on VPC

Step 01.03 : Select Your VPCs

01

Step 01.04 : Select Create VPC

02

Step 01.05 Enter the following details in the Create VPC option

  • Enter the details of the Name Tag
  • Enter the CIDR Block. keep in mind that the block size cannot be greater that /16.

Step 01.06: Click on Yes,Create

04

We have now created a VPC. The following resources are also created automatically

  • Routing table for the VPC
  • Default VPC Security Group
  • Network ACL for the VPC

Default Routing Table ( Route Table Id = rtb-ab1cc9d3)

Check the Routing table below for the VPC. If you check the routes of the route table, you see the following

  • Destination :10.0.0.0/16
  • target : Local
  • Status: Active
  • Propagated: No

12

This route ensures that all the subnets in the VPC are able to connect with each other. All the subnets created in the VPC are assigned to the default route table therefore its best practice not to change the default route table. For any route modification, a new route table can be created and assigned to subnets specifically.

Default Network Access Control List ( NACL id= acl-ded45ca7)

Mentioned below is the snapshot of the default NACL created when the VPC was created.

06.GIF

Default security group for the VPC ( Group id = sg-5c088122)

Mentioned below is the snapshot of the default Security Group created when the VPC was created.

07.GIF

Now we need to create Subnets. Keeping in mind that the considered scenario needs 02 subnets ( 01 Private and 01 Public ).1.

2 Creating Subnets

Step 02.01 : Go to the VPC Dashboard and select Subnets

08

Step 02.02 : Click on Create Subnet

09

Step 02.03: Enter the following details in the Create Subnet window

  • Name Tag: Subnet IPv4 CIDR Block ) – “Availability Zone” =  10.0.1.0/24 – us-east-1a
  • VPC : Select the newly created VPC = vpc-cd54beb4 | MyVPC
  • Availability Zone: us-east-1a
  • IPv4 CIDR Block :10.0.1.0/24

Step 02.04: Click on Yes,Create

10

Now we have created subnet 10.0.1.0/24

We will use the same steps to create another subnet. 10.0.2.0/24 in availability zone us-east-1b

  • Name Tag: Subnet IPv4 CIDR Block ) – “Availability Zone” =  10.0.2.0/24 – us-east-1b
  • VPC : Select the newly created VPC = vpc-cd54beb4 | MyVPC
  • Availability Zone: us-east-1b
  • IPv4 CIDR Block :10.0.2.0/24

11

3 Configuring subnets

Now that we have 02 subnets and we need to configure the 10.0.1.0/24 as the public subnet and 10.0.2.0/24 as the private subnets. The following tasks need to be performed for the activity

  • Internet Gateway creation and configuration
  • Route table Creation and configuration
  • Auto Assign Public IP Configuration.

3.1 Internet gateway Creation and Configuration ( IGW Config )

Internet gateways as the name suggest provide access to the internet. They are assigned to VPC and routing table is configured to direct all internet based traffic to the internet gateway.

Mentioned below are the steps for creating and configuring the internet gateway.

Step 03.01.01 : Select Internet Gateways  from the VPC dashboard and click on Create Internet Gateway

13.GIF

Step 03.01.02 : Enter the name tag and click on Yes,Create

14.GIF

The internet gateways is created but not attached to any VPC.( internet gateway Id = igw-90c467f6)

Step 03.01.03: Select the Internet Gateway and click on Attach to VPC

15

Step 03.01.04 : Select your VPC and click on Yes,Attach

16.GIF

We have now attached the Internet Gateway to the VPC. Now we need to configure the route tables for internet access.

3.2 Route Table creation and Configuration ( RTBL Config)

A default route table ( with id rtb-ab1cc9d3) was created when the VPC was created. Its best practice to create a separate route table to internet access.

Step 03.02.01 : Click on the Route Table section in the VPC Dashboard and click Create Route table

17

Step 03.02.02: Enter the following details in the Create Route Table window and click on Yes,Create

  • Name tag: Relevant Name = InternetAccessRoutetbl
  • VPC : Your VPC = vpc-cd54b3b4 | MyVPC

19

Step 03.02.03 : Select a newly created Route table( Route Table Id = rtb-3b78ad43 | InternetAccessRouteTbl) and Click Routes and then Edit

20

Step 03.02.04: Click on Add Another Route

21

Step 03.02.05 : Enter the following values in the route and click on Save

  • Destination: 0.0.0.0/0
  • Target : Your Internet Gateway ID  = igw-90c467f6 ( in my case )

22.GIF

Route table needs subnet associations. The subnets which we want to make Public should be associated with the route table. In our case, we would associate Subnet 10.0.1.0/24 to the route table.

Step 03.02.06: Click on Subnet Associations

23.GIF

You should be able to see the message “You do not have any subnet associations”

Step 03.02.07: Click on Edit

24.GIFStep 03.02.08: Select the subnet you want to configure as a Public Subnet. In our case 10.0.1.0/24 and Click on Save

25.GIF

03.03 Auto Assign Public IP Configuration

Both the subnets created ( 10.0.1.0/24 and 10.0.2.0/24) will not assign public IP addresses to the instances deployed in them as per their default configuration.

We need to configure the public subnet ( 10.0.1.0/24 ) to provide Public IPs automatically.

Step 03.03.01: Go to the Subnets section in the VPC dashboard.

Step 03.03.02: Select the Public Subnet

Step 03.03.03: Click on Subnet Actions

Step 03.03.04: Select Modify auto-assign IP Settings

26.GIF

Step 03.03.05: Check the Enable Auto-assign Public IPv4 Addresses  in the  Modify Auto-Assign IP Settings Window and click on Save

27.GIF

After this configuration, any EC2 instance deployed in the 10.0.1.0/24 subnet will be assigned a public IP.

4 Security

security group acts as a virtual firewall that controls the traffic for one or more instances. When you launch an instance, you associate one or more security groups with the instance. You add rules to each security group that allow traffic to or from its associated instances. You can modify the rules for a security group at any time; the new rules are automatically applied to all instances that are associated with the security group. When we decide whether to allow traffic to reach an instance, we evaluate all the rules from all the security groups that are associated with the instance.

We will create 02 security groups,

  • Public-Private ( Will contains access rules from Public Subnet to Private Subnet )
  • Internet-Public ( will contains the ports allowed from the internet to the Public Subnet )

Step 4.1  : Click on Security Groups in the Network and Security section

Step 4.2 : Click on Create Security Group

28

Step 4.3 : Enter the following details on the Create Security group Window and click on Create

  • Security Group Name : Public-Private
  • Description : Rules between Private subnet and Public subnets
  • VPC : Select the VPC we created in the exercise.
  • Click on Add Rules to Add the following rules
    • Type = RDP , Protocol = TCP , POrt Range = 3389 , Source = Custom : 10.0.1.0/24
    • Type = All ICMP – IPV4, Protocol = ICMP , Port Range = 0 – 65535 , Source = Custom, 10.0.1.0/24

29

Step 4.4 : Enter the following details on the Create Security group Window and click on Create

  • Security Group Name : Public-Internet
  • Description : Rules between Public and the internet
  • VPC : Select the VPC we created in the exercise.
  • Click on Add Rules to Add the following rules
    • Type = RDP , Protocol = TCP , POrt Range = 3389 , Source =Anywhere
    • Type = All ICMP – IPV4, Protocol = ICMP , Port Range = 0 – 65535 , Source = Anywhere

30

4 EC2 installation

Now we will deploy 02 EC2 instances . One EC2 Instances Named PrivateInstance in the 10.0.2.0/24 subnet and one instance named PublicInstance in the 10.0.1.0/24 subnet.

Public Instance Configuration :

  • Instance Name : Public Instance
  • Network : MyVPC
  • Subnet : 10.0.1.0/24
  • Auto-Assign Public ip : Use subnet setting ( enabled )
  • Security Group : Public-Internet security group
  • IAM Role : As per requirement

Private Instance Configuration :

  • Instance Name : Private Instance
  • Network : MyVPC
  • Subnet : 10.0.2.0/24
  • Auto-Assign Public ip : Use subnet setting ( disabled)
  • Security Group : Public-Private security group
  • IAM Role : As per requirement

Once the deployment of the EC2 instance is complete, you can connect to the PublicInstance through RDP and from there connect further to the Private instances.

Check Patch Status of ‘WannaCrypt’ / ‘WannaCry’ using PowerShell

A short but sweet blog today, mindful that today most Australians will be coming back to work after the ‘WannaCrypt’ attack that was reported in the media on Friday.

I would like to just point out the work of Kieran Walsh – he’s done the ‘hard yards’ of extracting all of the Knowledge Base (KB) article numbers that you need to be searching for, to determine your patching status of Microsoft Security Bulletin MS17-010  (https://technet.microsoft.com/en-us/library/security/ms17-010.aspx).  Microsoft’s detailed blog about the ‘WannaCrypt ransomware’ can be found here: https://blogs.technet.microsoft.com/mmpc/2017/05/12/wannacrypt-ransomware-worm-targets-out-of-date-systems/

If you don’t have an Enterprise patch deployment tool such as SCCM or WSUS (there are many many others), Kieran’s script executes a simple ‘Get-Hotfix’ PowerShell command remotely against a Windows Server or workstation, and uses all the computer objects in Active Directory as a reference.  I personally haven’t run this yet, so please test this first against a test AD if you have one.  The ‘Get-Hotfix’ command is relatively ‘benign’ so the risk is low.

Conversely, if you’re looking to run this on your local workstation, I’ve modified his script and made a simple ‘local’ check.  Copy and paste this into a PowerShell window with ‘administrator’ permissions:

#— Script start

# List of all HotFixes containing the patch
$hotfixes = @(‘KB4012598’, ‘KB4012212’, ‘KB4012215’, ‘KB4015549’, ‘KB4019264’, ‘KB4012213’, ‘KB4012216’, ‘KB4015550’, ‘KB4019215’, ‘KB4012214’, ‘KB4012217’, ‘KB4015551’, ‘KB4019216’, ‘KB4012606’, ‘KB4015221’, ‘KB4016637’, ‘KB4019474’, ‘KB4013198’, ‘KB4015219’, ‘KB4016636’, ‘KB4019473’, ‘KB4013429’, ‘KB4015217’, ‘KB4015438’, ‘KB4016635’, ‘KB4019472’, ‘KB4018466’)
# Search for the HotFixes
$hotfix = Get-HotFix | Where-Object {$hotfixes -contains $_.HotfixID} | Select-Object -property “HotFixID”
# See if the HotFix was found
if (Get-HotFix | Where-Object {$hotfixes -contains $_.HotfixID}) {write-host “Found hotfix” $_.HotfixID

} else {

write-host “Didn’t find hotfix”

}
#— Script end
Please follow all official Microsoft advice in applying the correct patch as per the security bulletin link above.  Conversely look to disable ‘SMBv1’ services on your workstations until you can get them patched.  Good luck.
** Update @ 4:30pm (15/05/2017).  In my testing, I’ve found the Windows 10 patches listed in the Security Bulletin have been superseded by newer KB numbers.  I’ve added three KB’s for the 64-bit version of Windows 10, version 1511.  I’d suggest looking at the ‘Package Details’ tab of the Microsoft Catalog site (e.g http://www.catalog.update.microsoft.com/Search.aspx?q=KB4013198) for the latest KB numbers.  I’ll try to add all KBs for Windows 10 by tomorrow AEST (the 16th).  Alternative, keep an eye on updates to Kieran’s script as he gets update from the community.
a1.PNG

** Update @ 5pm – The MS blog about the ransomware attack itself specifically states Windows 10 machines are not impacted even though there are patches for the security bulletin that apply to Windows 10.  Ignore Windows 10 devices in your report unless there’s updated information from Microsoft.

** Update @ 8pm: Kieran has updated his script to exclude Windows 10 computer objects from the AD query.

** Update @ 9:30 am 16/05:  Updated list of KBs from Kieran’s script (who has been sourcing the latest KB list from the community)

** Updated @ 2pm 17/05:  Updated list of KBs (including Windows 10 updates) from the comments area from Kieran’s script (user: d83194).  For future updates, I’d suggest reviewing Kieran’s comments for the latest KB articles.  I’ll let you make the decision about whether to keep the Windows 10 filter (-notlike ‘Windows 10‘) in Kieran’s script.  Maybe produce two reports (with Windows 10/without Windows 10).

Understanding Password Sync and Write-back

For anyone who has worked with Office 365/Azure AD and AADConnect, you will of course be aware that we can now sync passwords two ways from Azure AD to our on-premises AD. This is obviously a very handy thing to do for myriad reasons, and an obvious suggestion for a business intending to utilise Office 365. The conversation with the security bod however, might be a different kettle of fish. In this post, I aim to explain how the password sync and write-back features work, and hopefully arm you with enough information to have that chat with the security guys.

Password Sync to AAD

If you’ve been half-listening to any talks around password sync, the term ‘it’s not the password, it’s a hash of a hash’ is probably the line you walked away with, so let’s break down what that actually means. First up, a quick explanation of what it actually means to hash a value. Hashing a value is applying a cryptographic algorithm to a string to irreversibly convert that string into another sting of a fixed length. This differs from encryption in that with encryption we intend to be able to retrieve the data we have stored, so must hold a decryption key – with hashing, we are unable to derive the original string from the hashed string, and nor do we want to. If it makes no sense to you that we wouldn’t want to reverse a hashed value, it will by the end of this post.

So, in Active Directory when a user sets their password, the value stored is not actually the password itself, it’s an MD4 hash of the password once it’s been converted to Unicode Little Endian format. Using this process, my password “nicedog” ends up in Active Directory as 2993E081C67D79B9D8D3D620A7CD058E. So, this is the first part of the “hash of a hash”.

The second part of the “hash of a hash” is going to go buck wild with the hashing. Once the password has been securely transmitted to AADConnect (I won’t detail that process, because we end up with our original MD4 hash anyway), the MD4 hash is then:

  1. Converted to a 32-byte hexadecimal string
  2. Converted into binary with UTF-16 encoding
  3. Salted with 10 additional bytes
  4. Resultant string hashed and re-hashed 1000 times with HMAC-SHA256 via PBKDF2

So, as you can see, we’re actually hashing the hash enough to make Snoop Dogg blush. Now, we finally have the have the hash which is securely sent to Azure AD via an SSL channel. At this point, the hashed value which we send is so far removed from the original hash that even the most pony-tailed, black tee-shirt and sandal wearing security guy would have to agree it’s probably pretty secure. Most importantly, if Azure AD was ever compromised and an attacker did obtain your users password hashes, they are unable to be used to attack any on-premises infrastructure with a “pass the hash” attack, which would be possible if AD’s ntds.dit was compromised and password hashes extracted.

So now Azure AD has these hashes, but doesn’t know your passwords – how are user’s passwords validated? Well it’s just a matter of doing the whole thing again with a user provided password when that user logs on. The password is converted, salted, hashed and rehashed 1000 times in the exact same manner, meaning the final hash is exactly the same as the one stored in Azure AD – the two hashes are compared, and if we have a match, we’re in. Easy. Now let’s go the other way.

Password Write-back to AD

So how does it work going back the other way? Differently. Going back the other way, we can’t write a hash ourselves to Active Directory, so we need to get the actual password back from AAD to AD and essentially perform a password reset on-prem allowing AD to then hash and store the password. There are a couple of things which happen when we install AADConnect and enable Password Write-back to allow us to do this

  1. Symmetric keys are created by AAD and shared with AADConnect
  2. A tenant specific Service Bus is created, and a password for it shared with AADConnect

The scenario in which we are writing a password back from AAD to AD is obviously a password reset event. This is the only applicable scenario where we would need to, because a password is only set in AAD if that user is “cloud only”, so they would of course not have a directory to write back to. Only synced users need password write-back, and only upon password reset. So AAD gets the password back on-premises by doing the following:

  1. User’s submitted password is encrypted with the 2048-bit RSA Key generated when you setup write-back
  2. Some metadata is added to the package, and it is re-encrypted with AES-GCM
  3. Message sent to the Service Bus via and SSL/TLS channel
  4. On-premises AADConnect agent wakes up and authenticates to Service Bus
  5. On-premises agent decrypts package
  6. AADConnect traces cloudanchor attribute back to the connected AD account via the AADConnect sync engine
  7. Password reset is performed for that user against a Primary Domain Controller

So, that’s pretty much it. That’s how we’re able to securely write passwords back and forth between our on-premises environment and AAD without Microsoft ever needing to store a user’s password. It’s worth noting for that extra paranoid security guy though that if he’s worried about any of the encryption keys or shared passwords being compromised, you can re-generate the lot simply by disabling and re-enabling password writeback.

Hopefully this post has gone some way towards helping you have this particular security conversation, no matter what year the security guys Metallica tour tee-shirt is from.

Monitor SharePoint Online Activity with OMS and Power BI

As more organisations move their data into the cloud there is now a big focus on getting more insight and visibility in what data is being moved up into the cloud, where it is being stored, how it is being used and by whom.

This post looks at how we can provide greater insight into SharePoint Online, but can also be applicable across other document management systems.

While Office 365 provides an audit capability out of the box to view user activity, this is not highly accessible to those who do not administer Office 365 directly. This blog post looks at what is possible using off-the-shelf Microsoft cloud apps that can provide this visibility to those in the business that require it.

Two main use cases are:

  1. Change and Adoption – tracking uptake of SharePoint and getting visibility into who in the business is using it the most/least
  2. Document Governance – track access, changes and external sharing of documents to find patterns and for forensic analysis

What is useful is to have a high-level dashboard view of trends with items of interest flagged, and the ability to then ‘drill down’ as required to get those next levels of detail. We also need to be able to provide this capability outside of the Office 365

What is involved?

Firstly, there is your Office 365 tenant, where we’re going to draw the data from. As you may know, once auditing is enabled SharePoint makes audit logs available so that an administrator can see who has done what and when.

More details on enabling auditing can be found here: https://support.office.com/en-us/article/Configure-audit-settings-for-a-site-collection-a9920c97-38c0-44f2-8bcb-4cf1e2ae22d2

Now, these audit logs can be pulled into Microsoft Operations Management Suite (OMS) which is the Microsoft cloud-first Management as a Service platform. From there we can start to get some useful views of the data, search and aggregate log data, and also set alerts as desired.

Getting the log info into OMS is as simple as enabling the Microsoft Operations Management Suite solution for Office 365 from the OMS marketplace, and once this is enabled and you have configured OMS to connect to your O365 tenant, OMS will pull the audit log data from the Office 365 Management Activity API.

More information on the O365 Management API can be found here: https://msdn.microsoft.com/en-nz/office-365/office-365-managment-apis-overview

With all this useful information now in OMS we start building out custom solutions to show us a good high-level dashboard view of what’s happening in SharePoint Online, providing a visual representation and helping to identify patterns and anomalies.

These views are based on log queries using the Office 365 Management API Schema

Type=OfficeActivity OfficeWorkload=sharepoint

You can get a copy of the SPO Activity.omsview solution that I created (containing the views above) and import it directly into your OMS workspace from here:

https://github.com/ColdHarbour/OMS_SOLS

Reporting for the Business

Now, these OMS views and log searches are great for those in your business who are adept at using a SIEM (Security Information and Event Management) tool such as OMS, but for consumption of reports by a wider audience you may want to consider a different medium. In this case, Microsoft has you covered with Power BI.

OMS has native integration with Power BI, you just need to enable the feature and from then on, any log searches can be pushed across to Power BI.

Once the dataset is in Power BI, these can be used to create reports and dashboards that can then be consumed by your stakeholders in the business so they can have a view into usage of SharePoint as well

How does it work?

The technical integration details of how all this works under the covers is pretty straightforward.

So that’s a high level view of what’s possible using OMS and Power BI to pull log data from Office 365 and turn it into useful reports and dashboards.

Please let me know in the comments if you’d like to see more on OMS and how we’re using it at Kloud (e.g. more O365, Azure, Windows Infrastructure, Containers, ServiceNow integration)

Using the Lithnet PowerShell Modules to generate full object metadata FIM/MIM HTML Reports

How many times have you wanted a consolidated report out of FIM/MIM for an object? What connectors does it have, what are the values of the attributes, which Management Agent contributed the value(s) and when? Individually of course you can get that info using the Metaverse Search and looking at the object in MIM Portal. But what if you wanted it all with a single query? This blog post provides an approach to doing just that. The graphic above shows a screenshot of a sample output. Click this Sample Report for full resolution version of the screenshot above. Note: The updated version of the script below outputs DisplayName for the ExpectedRulesList attribute so it actually provides valuable information. 

Overview

The approach is quite simple. It is;

  • Query the FIM/MIM Metaverse for an object
  • Take the response from the Metaverse to build the Connectors and Metaverse Hologram reports
  • Use the connector information to query the MIM Service MA (this example assumes it is on the same server. If not add the following line into the script with the appropriate values) and get the objects MIM Service Connector Space info
    Set-ResourceManagementClient -BaseAddress http://fimsvc:5727;
  • Take information retrieved above to then query the MIM Service and return the information for the object.
  • Format all the output for HTML, apply a simple style sheet, output to file and display in the default browser

NOTE: If you combine this with the Get-MVObject query building script detailed here it can be a relatively simple solution. That script even uses the same variables $queries and $query as outputs from the search and input into the HTML Report.

NOTE: You could possibly run it remotely from the MIM Sync Server too, if you leverage Remote Powershell to your FIM/MIM Sync server as detailed here.

The Script

Here it is. Lines 23 and 24 contain a hard-coded query. Update for your search criteria, or as detailed above combine this with the Get-MVObject query building script detailed here .  The Output directory specified in Line 7 is where the stylesheet and the resultant HTML file will be placed. Update for your needs.

For the Expected Rules List (unlike the screenshot as I’ve modified the script afterwards), the script gets the DisplayName for them and puts that in the report. DisplayName is more valuable than an ERE ObjectID.

Scripting queries for Lithnet Get-MVObject searches into the Microsoft Identity Manager Metaverse

It probably seems obvious by now, but I seem to live in PowerShell and Microsoft Identity Manager. I’m forever looking into the Microsoft Identity Manager Metaverse for objects.

However, sometimes I get tripped up by the differences in Object Classes between the FIM/MIM Service and the Metaverse, the names of the Object Classes (obviously not Person, Group and Contact) and in situations where they are case-sensitive.  If you’re using the Sync Service Manager Metaverse Search function though you get a pick list. But getting the data out to do something else with isn’t an option.

Solution

I’ve looked to quickly provide a similar function to the pick lists in the Metaverse Search GUI via Powershell which then gets executed by the Get-MVObject PowerShell Module.

UPDATE: 17 May 2017 The Lithnet MIIS Automation PowerShell Module has been updated for Get-MVObject to support the ObjectType Scope. I’ve updated the script to include the scope parameter based on the ObjectClass selected at the beginning of the script. 

I’ve defaulted the ObjectClass to Person so you can just press enter. But if you have custom ObjectClasses in your Metaverse you may need to change the index number in Line 48 from 5 to whichever index Person appears in your environment. Same goes for the default attribute of AccountName in the Attribute list. It appears at index 5 (Line 77) in my attribute list.

Process

Basically just run the PowerShell script and choose your options. The script needs interaction with the FIM/MIM Sync server, so you run it from the FIM/MIM Sync server. If you want to run it remotely (of course you do), then Remote PowerShell is your friend. Checkout how to do that to the FIM/MIM Server in this post here.

The Script itself will query the FIM/MIM MV Schema and return a list of Object Classes. As detailed above, in Line 48 of the script I have ‘index 5’ as the default which in my environment is Person and as such you can just hit enter if that is the Object Class you want to choose attributes from in the next step. Otherwise type the name of the ObjectClass you want. You don’t have to worry about case sensitivity as the script handles that. You can only choose a single ObjectClass obviously, but the menu ui I’ve used allows for multiple selections. Just press enter when prompted for another option for ObjectClass.

You’ll then be presented with a list of attributes from the chosen Object Class above. Again as detailed above I have it defaulting to ‘accountName’ which is index 5 in my list. Change (Line 77) for the default you want. This means you can just hit enter if accountName is what you’re querying on (which is common). Or choose another option. This then also allows you to also choose multiple attributes (which will be added to an array). This means you can use this for complex queries such as;

accountName startsWith 'dar'
sn startsWith 'rob'
mail contains '@kloud'

If you want to choose multiple attributes for your query and one of them is the default option, make sure you specify one of the attributes that is not the default first so that you get the option to specify more. When you’ve chosen all the attributes you are going to use in your query hit enter and the script will take an empty response as the end of your choices.

Now for each attribute chosen you will be prompted for an Operator. Pretty simple. Just choose from the available options. Note: all operators are shown but not all operators can be used for all attribute types. e.g. Don’t select ‘EndsWith’ for a Boolean attribute type and expect it to work. If you choose an operator other than the default (equals in my example) hit enter when prompted for the second time and the script will take an empty response as the end of your choices.

Finally provide what you the value is for the search term for the attribute. If the value has spaces, don’t worry about putting the value in quotes. The script takes care of that.

The last two steps will iterate through, for queries where you have chosen multiple attributes.

And you’re done. $query is the variable that contains the results. In line 115 I’m using Show-Object from the PowershellCookBook PSM. That then gives you a GUI representation of the result as shown below. If the query returns multiple results this will only show the last.

Line 114 outputs the value of the attributes ($query.attributes) to the console as well. If you have multiple objects returned $query will show them as shown below.

Finally if you want to run the query again, or just make a subtle change, you shouldn’t have to go through that again. Get the value of $querytxt and you’ll get the query and the command to execute it. $querytxt is also output to the console as shown below. Copy and paste it into Powershell ISE, update and execute.

The Script

Here is the raw script. Hardly any error handling etc, but enough to get you started and tailor it for your requirements. Enjoy.

Patching EC2 through SSM

 

Why Patch Manager?

AWS SSM Patch Manager is an automated tool that helps you simplify your operating system patching process, including selecting the patches you want to deploy, the timing for patch roll-outs, controlling instance reboots, and many other tasks. You can define auto-approval rules for patches with an added ability to black-list or white-list specific patches, control how the patches are deployed on the target instances (e.g. stop services before applying the patch), and schedule the automatic roll out through maintenance windows.

These capabilities help you automate your patch maintenance process to save you time and reduce the risk of non-compliance. All capabilities of EC2 Systems Manager, including Patch Manager, are available free of charge so you only pay for the resources you manage.

The article can be used to configure patching for instances hosted in AWS Platform.

You will need to have the necessary pre-requisite knowledge regarding, EC2, and IAM section of the AWS. If so then please read on.

The configuration has three major sections

  • EC2 instance configuration for patching
  • Default Patching Baseline Configuration
  • Maintenance Window configuration.

1  Instance Configuration

We will start with the First section which is configuring the Instances to be patched. This requires the following tasks.

  1. Create Amazon EC2 Role for patching with two policies attached
    • AmazonEC2RoleForSSM
    • AmazonSSMFullAccess
  2. Assign Roles to the EC2 Instances
  3. Configure Tags to ensure patching in groups.

Important: The Machines to be patched should be able to contact Windows Update Services.  Mentioned below article contains the URLS which should be accessible for proper patch management.

https://technet.microsoft.com/en-us/library/cc708605(v=ws.10).aspx

Mentioned below are the detailed steps for the creation of an IAM role for Instances to be Patched using Patch Manager.

Step 1: Select IAM —–> Roles and Click on Create New Role

1

Step 2: Select Role Type —-> Amazon EC2

2.PNG

Step 3: Under Attach Policy Select the following and Click Next

  • AmazonEC2RoleForSSM
  • AmazonSSMFullAccess

3.PNG

Step 4: Enter the Role Name and Select Create Role (At the bottom of the page)

4.PNG

Now you have gone through the first step in your patch management journey.

Instances should be configured to use the above created role to ensure proper patch management. (or any roles which has AmazonEC2RoleforSSM and AmazonSSMFullAccess policies attached to it.)

5.PNG

We need to group our AWS hosted servers in groups cause no one with the right frame of mind wants to patch all the servers in one go.

To accomplish that we need to use Patch Groups (explained later).

For example:

We can configure Patch manager to Patch EC2 instances with Patch Group Tag = Group01 on Wednesday and EC2 instances with Patch Group Tag = PatchGroup02 on Friday.

To utilize patch groups, all EC2 instances should be tagged to support cumulative patch management based on Patch Groups.

Congratulations, you have completed the first section of the configuration. Keep following just two to go.

Default Patch Baseline configuration.

Patches are categorized using the following attributes :

  • Product Type: like Windows version etc.
  • Classification: CriticalUpdates, SecurityUpdates, SevicePacks, UpdateRollUps
  • Severity: Critical,Important,Low etc.

Patches are prioritized based on the above factors.

A Patch baseline can be used to configure the following using rules

  • Products to be included in Patching
  • Classification of Patches
  • Severity of Patches
  • Auto Approval Delay: Time to wait (Days) before automatic approval)

Patch Baseline is configured as follows.

Step 01: Select EC2 —> Select Patch Baselines (under the Systems Manager Services Section)

Step 02: Click on Create Patch Baseline

6.PNG

Step 03: Fill in the details of the baseline and click on Create

7.PNG

Go to Patch Baseline and make the newly created baseline as your default.

8.PNG

At this point, the instances to be patched are configured and we have also configured the patch policies. The next section we provide AWS the when (Date and Time) and what (task) of the patching cycle.

Maintenance Windows Configuration

As the name specifies, Maintenance Windows give us the option to Run Tasks on EC2 Instances on a specified schedule.

What we wish to accomplish with Maintenance Windows is to Run a Command (Apply-AWSPatchBaseline), but on a given schedule and on a subset of our servers. This is where all the above configurations gel together to make patching work.

Configuring Maintenance windows consist of the following tasks.

  • IAM role for Maintenance Windows
  • Creating the Maintenance Window itself
  • Registering Targets (Selecting servers for the activity)
  • Registering Tasks (Selecting tasks to be executed)

Mentioned below are the detailed steps for configuring all the above.

Step 01: Create a Role with the following policy attached

  • AmazonSSMMaintenanceWindowRole

9.PNG

Step 02: Enter the Role Name and Role Description

10.PNG

Step 03: Click on Role and copy the Role ARN

Step 04: Click on Edit Trust Relationships

11.PNG

Step 05: Add the following values under the Principal section of the JSON file as shown below

“Service”: “ssm.amazonaws.com”

Step 06: Click on Update Trust Relationships (on the bottom of the page)

12.PNG

At this point the IAM role for the maintenance window has been configured. The next section details the configuration of the maintenance window.

Step 01: Click on EC2 and select Maintenance Windows (under the Systems Manager Shared Resources section)

13.PNG

Step 02: Enter the details of the maintenance Windows and click on Create Maintenance Windows

14.PNG

At this point the Maintenance Window has been created. The next task is to Register Targets and Register Tasks for this maintenance window.

Step 01: Select the Maintenance Window created and click on Actions

Step 02: Select Register Targets

15.PNG

Step 03: Enter Owner Information and select the Tag Name and Tag Value

Step 04: Select Register Targets

16.PNG

At this point the targets for the maintenance window have been configured. This leaves us with the last activity in the configuration which is to register the tasks to be executed in the maintenance window.

Step 01: Select the Maintenance Window and Click on Actions

Step 02: Select Register Task

17.PNG

Step 03: Select AWS-ApplyPatchBaseline from the Document section

18.PNG

Step 04: Click on Registered targets and select the instances based on the Patch Group Tag

Step 05: Select Operation SCAN or Install based on the desired function (Keep in mind that an Install will result in a server restart).

Step 06: Select the MaintenanceWindowsRole

Step 07: Click on Register Tasks

19.PNG

After completing the configuration, the Registered Task will run on the Registered Targets based on the schedule specified in the Maintenance Window

The status of the Maintenance Window can be seen in the History section (as Shown below)

20.PNG

Hope this guide does get you through the initial patching configuration for your EC2 instances in Amazon.

Also in AWS the configuration can be done using CLI as well. Lets leave that for another blog for now.

Thanks for Reading.

MfaSettings.xml updates not taking effect

First published at https://nivleshc.wordpress.com

Last week, I was at a client site, extending their Microsoft Identity Manager (MIM) 2016 Self Service Password Reset Solution so that it could use Azure MultiFactor Authentication (MFA). This is an elegant solution since instead of using Questions and Answers to authenticate yourself when trying to reset your password, you can use One Time Passwords (OTP), sent as a security code via a text message to your registered mobile device.

I followed the steps as outlined in https://github.com/Microsoft/MIMDocs/blob/master/MIMDocs/DeployUse/working-with-self-service-password-reset.md to enable Azure MFA, and everything went smoothly.

I then proceeded to testing the solution.

Using the Password Registration Portal, I registered my mobile number against my test user account.

I then opened the Password Reset Portal, entered my test user username and proceeded to wait for the text message from Microsoft Azure with the security code, so that I could enter it in the next screen.

MIM_Verify_MobilePhoneVerification

I waited and waited (for at least 5 min), unfortunately the text message didn’t arrive 😦

Ok, troubleshooting time.

On my Microsoft Identity Manager 2016 Service Server, I opened the Windows EventLogs viewer and then expanded the section for Forefront Identity Manager event logs. Aha, I was on the right track as I saw a lot of errors reported.

MIMServiceServer_EventLogs

I went through the event log entries and found one which looked abit odd. The error essentially said that the certificate path contained illegal characters.

MIMServiceServer_Error_CertificatePath

I couldn’t make much sense of this error, so I opened the MfaSettings.xml file to check, and I quickly realised my mistake. I had included the certificate file path within quotes!

I quickly removed the unnecessary ” “ , saved the MfaSettings.xml file and restarted my testing process.

I went through the password reset process again, and yet again, I didn’t receive any text message from Microsoft Azure with the security code 😦

I re-checked the eventlogs and noticed the same Exception: Illegal characters in path for the Certificate File Path error. Thinking that I might have forgotten to save the previous modification to the MfaSettings.xml file, I opened the file to confirm. The quotes were no where to be seen! Alas, the plot thickens my dear Watson!

I couldn’t find any explanation for this behaviour. Then, thinking that maybe the MIM server was having issues accessing the long filepath for the Certificate file, I moved the certificate file to a folder that was closer to the root of the C:\ drive, updated the MfaSettings.xml file appropriately and repeated my testing.

Again, no text message 😦

Checking the event logs, I noticed the same dreaded Exception: Illegal characters in path for the Certificate File Path error again.

However, looking closer at the error, I realised that the file path was reported as C:\Program Files\Microsoft Forefront Identity Manager\2010\Service\MfaCerts\cert_key.p12, which wasn’t correct since I had moved the certificate file to another folder and updated the MfaSettings.xml file accordingly!

Suddenly I had that light bulb moment 😉 Updates to the MfaSettings.xml file were not being read by the MIM Server! This could only mean that it wasn’t monitoring this file for any changes, quite the opposite to what I had initially assumed!

To force MIM to re-read the MfaSettings.xml file, I restarted the Forefront Identity Manager Service service and went through my password reset testing process again.

Eureka! This time around, I received the text message from Microsoft Azure with the security code! Checking the Event logs, I couldn’t find any new occurrences of the Exception: Illegal characters in path for the Certificate File Path error. Hurray!

I completed the password reset process and confirmed that the password for my test account had indeed been changed.

I hope this post helps others.

BTW, below is a sample of the MFASettings.xml file (for security reasons, the keys have been scrambled, however as seen below, none of the values need quotes)

<?xml version=”1.0″ encoding=”utf-8″ ?>
<SubscriberKeys>
<LICENSE_KEY>1A3FED2C1BZA</LICENSE_KEY>
<GROUP_KEY>a1b234c567890e123456gh1234567eij</GROUP_KEY>
<CERT_PASSWORD>1ABDCDEF1GHDAVWA</CERT_PASSWORD>
<CertFilePath>C:\Program Files\Microsoft Forefront Identity Manager\2010\Service\MfaCert\cert_key.p12</CertFilePath>
<Username>john.doe</Username>
<DefaultCountryCode>61</DefaultCountryCode>
</SubscriberKeys>