Where’s the source!

SauceIn this post I will talk about data (aka the source)! In IAM there’s really one simple concept that is often misunderstood or ignored. The data going out of any IAM solution is only as good as the data going in. This may seem simple enough but if not enough attention is paid to the data source and data quality then the results are going to be unfavourable at best and catastrophic at worst.
With most IAM solutions data is going to come from multiple sources. Most IAM professionals will agree the best place to source the majority of your user data is going to be the HR system. Why? Well simply put it’s where all important information about the individual is stored and for the most part kept up to date, for example if you were to change positions within the same company the HR systems are going to be updated to reflect the change to your job title, as well as any potential direct report changes which may come as a result of this sort of change.
I also said that data can come and will normally always come from multiple sources. At typical example of this generally speaking, temporary and contract staff will not be managed within the central HR system, the HR team simply put don’t care about contractors. So where do they come from, how are they managed? For smaller organisations this is usually something that’s manually done in AD with no real governance in place. For the larger organisations this is less ideal and can be a security nightmare for the IT team to manage and can create quite a large security risk to the business, so a primary data source for contractors becomes necessary what this is is entirely up to the business and what works for them, I have seen a standard SQL web application being used to populate a database, I’ve seen ITSM tools being used, and less common is using the IAM system they build to manage contractor accounts (within MIM 2016 this is through the MIM Portal).
There are many other examples of how different corporate applications can be used to augment the identity information of your user data such as email, phone systems and to a lessor extent physical security systems building access, and datacentre access, but we will try and keep it simple for the purpose of this post. The following diagram helps illustrate the dataflow for the different user types.

IAM Diagram

What you will notice from the diagram above, is even though an organisation will have data coming from multiple systems, they all come together and are stored in a central repository or an “Identity Vault”. This is able to keep an accurate record of the information coming from multiple sources to compile what is the users complete identity profile. From this we can then start to manage what information is flowed to downstream systems when provisioning accounts, and we can also ensure that if any information was to change, it can be updated to the users profiles in any attached system that is managed through the enterprise IAM Services.
In my next post I will go into the finer details of the central repository or the “Identity Vault”

So in summary, the source of data is very important in defining an IAM solution, it ensures you have the right data being distributed to any managed downstream systems regardless of what type of user base you have. My next post we will dig into the central repository or the Identity Vault, this will go into details around how we can set precedence to data from specific systems to ensure that if there is a difference in the data coming from the difference sources that only the highest precedence will be applied we will also discuss how we augment the data sets to ensure that we are also only collecting the necessary information related to the management of that user and the applications that use within your business.

As per usual, if you have any comments or questions on this post of any of my previous posts then please feel free to comment or reach out to me directly.

Resolving Microsoft Identity Manager “sync-rule-validation-parsing-error” error

A couple of weeks back I inherited a Microsoft Identity Manager development environment that wasn’t quite complete. When I performed a sync on a user object I got the following error;  sync-rule-validation-parsing-error

Looking into the error for further details, Details and Stack Trace were both greyed out as shown below.

I looked at the object being exported on the MA and the awaiting export details and found slightly different information. The error was CS to MV to CS synchronization failed 0x8023055a 

Still not a lot to go on. So I looked in the Application Event Log and nothing. Anything in the System Event Log? No, nothing.

So my attention turned to the Export Synchronization Rule. Here is a partial screenshot of the Export Sync Rule. The object (user) in question had been flagged as inactive and the intent appeared to be a clearing of a number of attributes. Sending “” (crude empty/null) to an attribute isn’t very elegant.

I changed each to use the null function. So for export, null() will flow to each of the attributes. I tried the export again and the same error and problem resulted.

Running short on ideas I created a brand new Export Synchronization Rule and replicated the configuration except for the attributes being exported. Then I added one attribute into the rule at a time, tested the export and repeated until I could replicate the error.

I was able to replicate the error once I hit the terminalServer attribute.
*Note: the screenshot below is prior to changing over to flow null() instead of “”.

Sending null() to the terminalServer Active Directory attribute was causing the error. It was at this point I actually just removed that flow rule and continued with other tasks.

Coming back to this later, and thinking it through I understand the error. When dealing with Terminal Services you actually normally manage four attributes that are part of the userParameters attribute. The four attributes that define a users Terminal Services Profile are;

  • allowLogon
  • terminalServicesHomeDirectory
  • terminalServicesProfilePath
  • terminalServicesHomeDrive

For a user that has a fully configured set of Terminal Services attributes, sending null() to the terminalServer attribute isn’t going to work.

So, posting this as I couldn’t find any reference to sync-rule-validation-parsing-error or CS to MV to CS synchronization failed 0x8023055a elsewhere and chances are I’ll come across it again, and it’ll probably help someone else too.

Integration of Microsoft Identity Manager with Azure Platform-as-a-Service Services

Overview

This isn’t an out of the box solution. This is a bespoke solution that takes a number of elements and puts them together in a unique way. I’m not expecting anyone to implement this specific solution (but you’re more than welcome to) but to take inspiration from it to implement solutions relevant to your environment(s). This post supports a presentation I did to The MIM Team User Group on 14 June 2017.

This post describes a solution that;

  • Leverages an Azure WebApp (NodeJS) to present a simple website. That site can be integrated easily in the FIM/MIM Portal
  • The NodeJS website leverages an Azure Function App to get a list of users from the FIM/MIM Synchronization Server and allows the user to use typeahead functionality to find the user they want to generate a FIM/MIM object report on
  • On selection of a user, a request will be sent to another Azure Function App to generate and return the report to the user in a new browser window

This is shown graphically below.

 

Report Request UI

The NodeJS WebApp is integrated into the FIM/MIM portal. Bootstrap Typeahead is used to find the user to generate a report on. The Typeahead userlist if fulfilled by an Azure Function into the MIM Sync Metaverse. The Generate Report button fires off a call to FIM/MIM via another Azure Function into the MIM Sync and MIM Service to generate the report.

The returned report opens in a new tab in the users browser. The report contains details of the FIM/MIM connectors the user is represented on.

The values of all attributes for the users hologram from the Metaverse are displayed along with the MA the value came from and the last modified date.

Finally the metadata report from the MIM Service MA Connector Space and the MIM Service.

Prerequisites

These are numerous, but I’ve previously posted about them. You will need;

I encourage you to digest those posts to understand how to configure the prerequisites for this solution.

Additional Solution Requirements

To bring all the individual components together, there are a few additional tasks to enable this solution.

  • Enable CORS on your Azure Function App Configuration (see details further below)
  • If you want to display User Object Photos as part of the report, you will likely need to synchronize them into FIM/MIM from an authoritative source (e.g. Office365/Exchange Online)   Checkout this post  and additional details further below
  • In order to embed the NodeJS WebApp into the FIM/MIM Portal, this post provides the details. Change the target URL from PowerBI URL to your NodeJS site
  • Object Report Request WebApp (see below for sample site)

Azure Functions Cross Origin Resource Sharing (CORS)

You will need to configure CORS to allow the NodeJS WebApp to access the Azure Functions (from both local and Azure). Reflect your port number if it is different from 3000, and use the DNS name for your Azure WebApp.

Sample UI NodeJS HTML

Here is a sample HTML file for your NodeJS WebApp with the UI to provide Input for LoginID fulfilled by the NodeJS Javascript file further below.

Sample UI NodeJS JavaScript

The following NodeJS JavaScript supports the HTML UI above. It populates the LoginID typeahead box and takes the Submit Report button to fulfill the report for the desired object(s). Yes if you use the UI to select (individually) multiple different objects all will be returned in their separate output windows.

As the HTML file above indicates you will need to obtain and make available as part of your NodeJS project the typeahead.bundle.js library.

Azure PowerShell Trigger Function App for AccountNames Lookup

The following Azure Function takes the call from the load of the NodeJS WebApp to populate the typeahead userlist.

Azure PowerShell Trigger Function App for User Object Report

Similar in structure to the Username List Lookup Azure Function above, but in the ScriptBlock you embed the Report Generation Script that is detailed here. Modify for what you want to report on.

Photos in the Report

If you want to display images in your report, you will need to determine if the user has an image during the MV metadata report generation part of the script. Add the following lines (updating for the name of your Image attribute; mine is named EXOPhoto) after the Try {} Catch {} in this section $obj = @() ; foreach ($attr in $attributes.Keys)

 # Display the Objects Photo rather than Base64 string 
if ($attr.equals("EXOPhoto")){ 
   $objectphoto = "<img src=$([char]0x22)data:image/jpeg;base64,$($attributes.$attr.Values.Valuestring)$([char]0x22)>" 
   $val = "System.Byte[]" 
}

Then in the output of the HTML report at the end of the report generation insert the $objectphoto variable into the HTML stream.

# Output MIM Service Object Data 
$MIMServiceObjOut = $MIMServiceObjectMetaData | Sort-Object -Property Attribute | ConvertTo-Html -Fragment 
$htmlreport = ConvertTo-HTML -Body "$htmlcss<h1>Microsoft Identity Manager User Object Report</h1><h2>Query</h2>$sourcequery</br><b><center>$objectphoto</br>NOTE: Only attributes with values are displayed.</center></b><h2>Connector(s) Summary</h2>$connectorsummary<h2>MetaVerse Data</h2>$objectmetadata <h2>MIM Service CS Object Data</h2>$MIMServiceCSobjectmetadata <h2>MIM Service Object Data</h2>$MIMServiceObjOut" -Title "MIM Object Report" 

 

As you can see above I’ve also injected the CSS ($htmlcss) into the output stream at the beginning of the Body section.  Somewhere in your script block you will need to define your CSS values. e.g.

 # StyleSheet for nice pretty output 
$htmlcss = "<style> 
   h1, h2, th { text-align: center; } 
   table { margin: auto; font-family: Segoe UI; box-shadow: 10px 10px 5px #888; border: thin ridge grey; } 
   th { background: #0046c3; color: #fff; max-width: 400px; padding: 5px 10px; } 
   td { font-size: 11px; padding: 5px 20px; color: #000; } 
   tr { background: #b8d1f3; } 
   tr:nth-child(even) { background: #dae5f4; } 
   tr:nth-child(odd) { background: #b8d1f3; } 
</style>"

Summary

An interesting solution integrating Azure PaaS Services with Microsoft Identity Manager via PowerShell and the extremely versatile Lithnet FIM/MIM PowerShell Modules.

Please share your implementations enhancing your FIM/MIM Solution.

MIM/FIM Full Sync of select objects only

As I detailed in my previous blog here, sometimes there is a need to perform a full synchronization of just a select set of objects in the MIM/FIM Synchronization Service. In my case, it was to all the Synchronization Rules which helped resolve my issue which required a selected Full Synchronization performed. For this customer’s FIM environment, I manually performed the Preview/Full Synchronization on 51 objects as I just needed it done. My colleague Darren Robinson suggested I look at scripting it using the ‘Lithnet PowerShell Module for FIM/MIM Synchronization Service’ located here. Up until this point, I hadn’t used this module in anger and I dearly wished I did as it would have saved plenty of time!

Ryan Newington has done a tremendous job with his modules released to the community to make our lives easier. He even enhanced the tool recently to make this particular task easier (note: make sure you grab the latest version, at least version v1.0.6351, as the script below will not work if you have an older version). The script I now regularly use on my customer’s large FIM environment after making changes to Synchronization Rules, where I mentioned in my previous blog we struggle for the opportunity to perform a Full Synchronization of the entire sync engine rule base, serves my purpose for Synchronization Rules, but you can easily modify the search criteria to perform a Full Synchronization of the objects you require. Here is the script and leave a comment if it has helped you.

Synchronizing Exchange Online/Office 365 User Profile Photos with FIM/MIM

Introduction

This is Part Two in the two-part blog post on managing users profile photos with Microsoft FIM/MIM. Part one here detailed managing users Azure AD/Active Directory profile photo. This post delves deeper into photos, specifically around Office 365 and the reason why you may want to manage these via FIM/MIM.

Background

User profile photos should be simple to manage. But in a rapidly moving hybrid cloud world it can be a lot more complex than it needs to be. The best summary I’ve found of this evolving moving target is from Paul Ryan here.

Using Paul’s sound advice we too are advising our customers to let users manage their profile photo (within corporate guidelines) via Exchange Online. However as described in this article photos managed in OnPremise Active Directory are synchronized to Azure AD and on to other Office365 services only once. And of course we want them to be consistent across AD DS, Azure AD, Exchange Online and all other Office365 Services.

This post details synchronizing user profile photos from Exchange Online to MIM for further synchronization to other systems. The approach uses a combination of Azure GraphAPI and Exchange Remote PowerShell to manage Exchange Online User Profile Photos.

The following graphic depicts the what the end goal is;

Current State

  • Users historically had a photo in Active Directory. DirSync/ADSync/AzureADConnect then synchronized that to Azure AD (and once only into Office 365).
  • Users update their photo in Office365 (via Exchange Online and Outlook Web Access)
    • the photo is synchronized across Office365 Services

Desired State

  • An extension of the Current State is the requirement to be able to take the image uploaded by users in Exchange Online, and synchronize it back to the OnPremise AD, and any other relevant services that leverage a profile photo
  • Have AzureADConnect keep AzureAD consistent with the new photo obtained from Office365 that is synchronized to the OnPrem Active Directory
  • Sync the current photo to the MIM Portal

Synchronizing Office365 Profile Photos

Whilst Part-one dealt with the AzureAD side of profile photos as an extension to an existing AzureAD PowerShell Management Agent for FIM/MIM, I’ve separated out the Office365 side to streamline it and make it as efficient as possible. More on that later. As such I’ve created a new PowerShell Management Agent specifically for Office365 User Profile Photos.

I’m storing the Exchange Online photo in the MIM Metaverse as a binary object just as I did for the AzureAD photo (but in a different attribute ). I’m also storing a checksum of the photos (as I did for the AzureAD Photo, but also in a different attribute) to make it easier for comparing what is in Azure AD and Exchange Online, to then be used to determine if changes have been made (eg. user updated their profile photo).

Photo Checksum

For generating the hash of the profile photos I’m using Get-Hash from the Powershell Community Extensions.  Whilst PowerShell has Get-FileHash I don’t want to write the profile photos out to disk and read them back in just to get the checksum. That slows the process up by 25%. You can get the checksum using a number of different methods and algorithms. Just be consistent and use the same method across both profile photos and you’ll be comparing apples with apples and the comparison logic will work.

Some notes on Photos and Exchange Online (and MFA)

This is where things went off on a number of tangents. Initially I tried accessing the photos using Exchange Online Remote PowerShell.

CAVEAT 1: If your Office365 Tenant is enabled for Multi-Factor Authentication (which it should be) you will need to get the Exchange Online Remote PowerShell Module as detailed here. Chances are you won’t have full Office365 Admin access though, so as long as the account you will be using is in the Recipient Management Role you should be able to go to the Exchange Control Panel using a URL like https://outlook.office365.com/ecp/?realm=&lt;tenantname>&wa=wsignin1.0 where tenantname is something like customer.com.au From the Hybrid menu on in the right handside pane you will then be able to download the Microsoft.Online.CSE.PSModule.Client.application I had to use Internet Explorer to download the file and get it installed successfully. Once installed I used a few lines from this script here to load the Function and start my RPS session from within PowerShell ISE during solution development.

CAVEAT 2: The EXO RPS MFA PS Function doesn’t allow you to pass it your account password. You can pass it the identity you want to use, but not the password. That makes scheduled process automation with it impossible.

CAVEAT 3: The RPS session exposes the Get-UserPhoto cmdlet which is great. But the RPS session leverages the GraphAPI. The RPS PS Module doesn’t refresh it’s tokens, so if the import takes longer than 60 minutes then using this method you’re a bit stuffed.

CAVEAT 4: Using the Get-UserPhoto cmdlet detailed above, the syncing of photos is slow. As in I was only getting ~4 profile photos per minute slow. This also goes back to the token refresh issue as for pretty much any environment of the size I deal with, this is too slow and will timeout.

CAVEAT 5: You can whitelist the IP Address (or subnet) of your host so MFA is not required using Contextual IP Addressing Whitelisting. At that point there isn’t really a need to use the MFA Enabled PREVIEW EXO RPS function anyway. That said I still needed to whitelist my MIM Sync Server(s) from MFA to allow integration into the Graph API. I configured just the single host. The whitelist takes CIDR format so that looks like /32 (eg. 11.2.33.4/32)

Performance Considerations

As I mentioned above,

  • using the Get-UserPhoto cmdlet was slow. ~4 per minute slow
  • using the GraphAPI into Exchange Online and looking at each user and determining if they had a photo then downloading it, was also slow. Slow because at this customer only ~50% of their users have a photo on their mailbox. As such I was only able to retrieve ~145 photos in 25 minutes. *Note: all timings listed above were during development and actually outputting the images to disk to verify functionality. 

Implemented Solution

After all my trial and error on this, here is my final approach and working solution;

  1. Use the Exchange Online Remote PowerShell (non-MFA version) to query and return a collection of all mailboxes with an image *Note, add an exception for your MIM Sync host to the white-listed hosts for MFA (if your Office365 Tenant is enabled for MFA) so the process can be automated
  2. Use the Graph API to obtain those photos
    • with this I was able to retrieve ~1100 profile photos in ~17* minutes (after ~2 minutes to query and get the list of mailboxes with a profile photo)

Pre-requisites

There’s a lot of info above, so let me summarize the pre-requisties;

  • The Granfeldt PowerShell MA
  • Whitelist your FIM/MIM Sync Server from MFA (if your Office 365 environment is enabled for MFA)
  • Add the account you will run the MA as, that will in turn connect to EXO via RPS to the Recipient Management Role
  • Create a WebApp for the PS MA to use to access users Profile Photos via the Graph API (fastest method)
  • Powershell Community Extensions to generate the image checksum

Creating the WebApp to access Office365 User Profile Photos

Go to your Azure Portal and select the Azure Active Directory Blade from the Resource Menu bar on the left. Then select App Registrations and from the Manage Section of the Azure Active Directory menu, and finally from the top of the main pane select “New Application Registration“.

Give it a name and select Web app/API as the type of app. Make the sign-in URL https://localhost and then select Create.

Record the ApplicationID that you see in the Registered App Essentials window. You’ll need this soon.

Now select All Settings => Required Permissions. Select Read all users basic profiles in addition to Sign in and read user profile. Select Save.

Under Required Permissions select Add and then select 1 Select an API, and select Office 365 Exchange Online then click Select.

Choose 2 Select Permissions and then select Read user profiles and Read all users’ basic profiles. Click Select.

Select Grant Permissions

From Settings select Keys, give your key a Description, choose a key lifetime and select Save. RECORD the key value. You’ll need this along with the WebApp ApplicationID/ClientID for the Import.ps1 script.

Using the information from your newly registered WebApp, we need to perform the first authentication (and authorization of the WebApp) to the Graph API. Taking your ApplicationID, Key (Client Secret) and the account you will use on on the Management Agent (and that you have assigned the Recipient Management Role in Exchange Online) and run the script detailed in this post here. It will authenticate you to your new WebApp via the GraphAPI after asking you to provide the account you will use on the MA and Authorizing the permissions you selected when registering the app. It will also create a refresh.token file which we will give to the MA to automate our connection. The Authorization dialog looks like this.

Creating the Management Agent

Now we can create our Management Agent using the Granfeldt PowerShell Management Agent. If you haven’t created one before checkout a post like this one, that further down the post shows the creation of a Granfeldt PSMA. Don’t forget to provide blank export.ps1 and password.ps1 files on the directory where you place the PSMA scripts.

PowerShell Management Agent Schema.ps1

PowerShell Management Agent Import.ps1

As detailed above the PSMA will leverage the WebApp to read users Exchange Profile Photos via the Graph API. The Import script also leverages Remote Powershell into Exchange Online (for reasons also detailed above). The account you run the Management Agent as will need to be added to the Recipient Management Role Group in order to use Remote PowerShell into Exchange Online and get the information required.

Take the Import.ps1 script below and update;

  • Update lines 11, 24 and 42 for the path to where you have put your PSMA. Mine is under the Extensions directory in a directory named EXOPhotos.
  • copy the refresh.token generated when authenticating and authorizing the WebApp earlier into the directory you specified in line 42 above.
  • Create a Debug directory under the directory you specified in lines 11,24 and 42 above so you can see what the MA is doing as you implement and debug it the first few times.
  • I’ve written the Import to use Paged Imports, so make sure you tick the Paged Imports checkbox on the configuration of the MA
  •  Update Lines 79 and 80 with your ApplicationID and Client Secret that you recorded when creating your WebApp

Running the Exchange User Profile Photos MA

Now that you have created the MA, you should have select the EXOUser ObjectClass and the attributes defined in the schema. You should also create the EXOPhoto (as Binary) and EXOPhotoChecksum (as String) attributes in the Metaverse on the person ObjectType (assuming you are using the built-in person ObjectType).

Configure your flow rules to flow the EXOPhoto and EXOPhotoChecksum on the MA to their respective attributes in the MV.

Create a Stage Only run profile and run it. If you have done everything correctly you will see photos come into the Connector Space.

Looking at the Connector Space, I can see EXOPhoto and EXOPhotoChecksum have been imported.

After performing a Synchronization to get the data from the Connector Space into the Metaverse it is time to test the image that lands in the Metaverse. That is quick and easy via PowerShell and the Lithnet MIIS Automation PowerShell Module.

$me = Get-MVObject -ObjectType person -Attribute accountName -Value "drobinson"
$me.Attributes.EXOPhoto.Values.ValueBinary
[System.Io.File]::WriteAllBytes("c:\temp\myOutlookphoto.jpg" ,$me.Attributes.EXOPhoto.Values.ValueBinary )

The file is output to the directory with the filename specified.

Opening the file reveals correctly my Profile Photo.

Summary

In Part one we got the AzureAD/Active Directory photo. In this post we got the Office365 photo.

Now that we have the images from Office365 we need to synchronize any update to photos to Active Directory (and in-turn via AADConnect to Azure AD). Keep in mind the image size limits for Active Directory and that we retrieved the largest photo available from Office365 when synchronizing the photo on. There are a number of PowerShell modules for photo manipulation that will allow you to resize accordingly.

How to Synchronize users Active Directory/Azure Active Directory Photo using Microsoft Identity Manager

Introduction

Whilst Microsoft FIM/MIM can be used to do pretty much anything your requirements dictate, dealing with object types other than text and references can be a little tricky when manipulating them the first time. User Profile Photos fall into that category as they are stored in the directory as binary objects. Throw in Azure AD and obtaining and synchronizing photos can seem like adding a double back-flip to the scenario.

This post is Part 1 of a two-part post. Part two is here. This is essentially the introduction to the how-to piece before extending the solution past a users Active Directory Profile Photo to their Office 365 Profile Photo. Underneath the synchronization and method for dealing with the binary image data is the same, but the API’s and methods used are different when you are looking to implement the solution for any scale.

As for why you would want to do this, refer to Part two here. It details why you may want to do this.

Overview

As always I’m using my favourite PowerShell Management Agent (the Granfeldt PSMA). I’ve updated an existing Management Agent I had for Azure AD that is described here. I highly recommend you use that as the basis for the extra photo functionality that I describe in this post. Keep in mind the AzureADPreview, now AzureAD Powershell Module has change the ADAL Helper Libraries. I detail the changes here so you can get AuthN to work with the new libraries.

Therefore the changes to my previous Azure AD PowerShell MA are to add two additional attributes to the Schema script, and include the logic to import users profile photo (if they have one) in the Import script.

Schema.ps1

Take the schema.ps1 from my Azure AD PSMA here and add the following two lines to the bottom (before the $obj in the last line where I’ve left an empty line (29)).

$obj | Add-Member -Type NoteProperty -Name "AADPhoto|Binary" -Value 0x20 
$obj | Add-Member -Type NoteProperty -Name "AADPhotoChecksum|String" -Value "23973abc382373"

The AADPhoto attribute of type Binary is where we will store the photo. The AADPhotoChecksum attribute of type String is where we will store a checksum of the photo for use in logic if we need to determine if images have changed easily during imports.

Import.ps1

Take the import.ps1 from my Azure AD PSMA here and make the following additions;

  • On your MIM Sync Server download/install the Pscx PowerShell Module.
    • The Pscx Powershell Module is required for Get-Hash (to calculate Image checksum) based on variables vs a file on the local disk
    • You can get the module from the Gallery using Install-Module Pscx -Force
    • Add these two lines up the top of the import.ps1 script. Around line 26 is a good spot
# Powershell Module required for Get-Hash (to calculate Image checksum)
Import-Module Pscx
  • Add the following lines into the Import.ps1 in the section where we are creating the object to pass to the MA. After the $obj.Add(“AADCity”,$user.city) line is a good spot. 
  • What the script below does is create a WebClient rather than use Invoke-RestMethod or Invoke-WebRequest to get the users Azure AD Profile image only if the ‘thumbnailPhoto@odata.mediaContentType’ attribute exists which indicates the user has a profile photo. I’m using the WebClient over the PowerShell Invoke-RestMethod or Invoke-WebRequest functions so that the returned object is in binary format (rather than being returned as a string), which saves having to convert it to binary or output to a file and read it back in. The WebClient is also faster for transferring images/data.
  • Once the image has been returned (line 8 below) the image is added to the object as the attribute AADPhoto to be passed to the MA (line 11)
  • Line 14 gets the checksum for the image and adds that to the AADPhotoChecksum attribute in line 16.

Other changes

Now that you’ve updated the Schema and Import scripts, you will need to;

  • Refresh your schema on your Azure AD PSMA to get the new attributes (AADPhoto and AADPhotoChecksum) added
  • Select the two new attributes in the Attributes section of your Azure AD PSMA
  • Create in your MetaVerse via the MetaVerse Designer two new attributes on the person (or whatever ObjectType you are using for users), for AADPhoto and AADPhotoChecksum. Make sure that AADPhoto is of type Binary and AADPhotoChecksum is of type string.

  • Configure your Attribute Flow on your Azure AD PSMA to import the AADPhoto and AADPhotoChecksum attributes into the Metaverse. Once done and you’ve performed an Import and Sync you will have Azure AD Photos in your MV.

  • How do you know they are correct ? Let’s extract one from the MV, write it to a file and have a look at it. This small script using the Lithnet MIIS Automation PowerShell Module makes it easy. First I get my user object from the MV. I then have a look at the text string version of the image (to make sure it is there), then output the binary version to a file in the C:\Temp directory.
$me = Get-MVObject -ObjectType person -Attribute accountName -Value "drobinson"
[string]$myphoto = $me.Attributes.AADPhoto.Values.ValueString
[System.Io.File]::WriteAllBytes("c:\temp\UserPhoto.jpg" ,$me.Attributes.AADPhoto.Values.ValueBinary )
  • Sure enough. The image is valid.

Conclusion

Photos are still just bits of data. Once you know how to get them and manipulate them you can do what ever you need to with them. See Part two that takes this concept and extends it to Office 365.

Using the Lithnet PowerShell Modules to generate full object metadata FIM/MIM HTML Reports

How many times have you wanted a consolidated report out of FIM/MIM for an object? What connectors does it have, what are the values of the attributes, which Management Agent contributed the value(s) and when? Individually of course you can get that info using the Metaverse Search and looking at the object in MIM Portal. But what if you wanted it all with a single query? This blog post provides an approach to doing just that. The graphic above shows a screenshot of a sample output. Click this Sample Report for full resolution version of the screenshot above. Note: The updated version of the script below outputs DisplayName for the ExpectedRulesList attribute so it actually provides valuable information. 

Overview

The approach is quite simple. It is;

  • Query the FIM/MIM Metaverse for an object
  • Take the response from the Metaverse to build the Connectors and Metaverse Hologram reports
  • Use the connector information to query the MIM Service MA (this example assumes it is on the same server. If not add the following line into the script with the appropriate values) and get the objects MIM Service Connector Space info
    Set-ResourceManagementClient -BaseAddress http://fimsvc:5727;
  • Take information retrieved above to then query the MIM Service and return the information for the object.
  • Format all the output for HTML, apply a simple style sheet, output to file and display in the default browser

NOTE: If you combine this with the Get-MVObject query building script detailed here it can be a relatively simple solution. That script even uses the same variables $queries and $query as outputs from the search and input into the HTML Report.

NOTE: You could possibly run it remotely from the MIM Sync Server too, if you leverage Remote Powershell to your FIM/MIM Sync server as detailed here.

The Script

Here it is. Lines 23 and 24 contain a hard-coded query. Update for your search criteria, or as detailed above combine this with the Get-MVObject query building script detailed here .  The Output directory specified in Line 7 is where the stylesheet and the resultant HTML file will be placed. Update for your needs.

For the Expected Rules List (unlike the screenshot as I’ve modified the script afterwards), the script gets the DisplayName for them and puts that in the report. DisplayName is more valuable than an ERE ObjectID.

Scripting queries for Lithnet Get-MVObject searches into the Microsoft Identity Manager Metaverse

It probably seems obvious by now, but I seem to live in PowerShell and Microsoft Identity Manager. I’m forever looking into the Microsoft Identity Manager Metaverse for objects.

However, sometimes I get tripped up by the differences in Object Classes between the FIM/MIM Service and the Metaverse, the names of the Object Classes (obviously not Person, Group and Contact) and in situations where they are case-sensitive.  If you’re using the Sync Service Manager Metaverse Search function though you get a pick list. But getting the data out to do something else with isn’t an option.

Solution

I’ve looked to quickly provide a similar function to the pick lists in the Metaverse Search GUI via Powershell which then gets executed by the Get-MVObject PowerShell Module.

UPDATE: 17 May 2017 The Lithnet MIIS Automation PowerShell Module has been updated for Get-MVObject to support the ObjectType Scope. I’ve updated the script to include the scope parameter based on the ObjectClass selected at the beginning of the script. 

I’ve defaulted the ObjectClass to Person so you can just press enter. But if you have custom ObjectClasses in your Metaverse you may need to change the index number in Line 48 from 5 to whichever index Person appears in your environment. Same goes for the default attribute of AccountName in the Attribute list. It appears at index 5 (Line 77) in my attribute list.

Process

Basically just run the PowerShell script and choose your options. The script needs interaction with the FIM/MIM Sync server, so you run it from the FIM/MIM Sync server. If you want to run it remotely (of course you do), then Remote PowerShell is your friend. Checkout how to do that to the FIM/MIM Server in this post here.

The Script itself will query the FIM/MIM MV Schema and return a list of Object Classes. As detailed above, in Line 48 of the script I have ‘index 5’ as the default which in my environment is Person and as such you can just hit enter if that is the Object Class you want to choose attributes from in the next step. Otherwise type the name of the ObjectClass you want. You don’t have to worry about case sensitivity as the script handles that. You can only choose a single ObjectClass obviously, but the menu ui I’ve used allows for multiple selections. Just press enter when prompted for another option for ObjectClass.

You’ll then be presented with a list of attributes from the chosen Object Class above. Again as detailed above I have it defaulting to ‘accountName’ which is index 5 in my list. Change (Line 77) for the default you want. This means you can just hit enter if accountName is what you’re querying on (which is common). Or choose another option. This then also allows you to also choose multiple attributes (which will be added to an array). This means you can use this for complex queries such as;

accountName startsWith 'dar'
sn startsWith 'rob'
mail contains '@kloud'

If you want to choose multiple attributes for your query and one of them is the default option, make sure you specify one of the attributes that is not the default first so that you get the option to specify more. When you’ve chosen all the attributes you are going to use in your query hit enter and the script will take an empty response as the end of your choices.

Now for each attribute chosen you will be prompted for an Operator. Pretty simple. Just choose from the available options. Note: all operators are shown but not all operators can be used for all attribute types. e.g. Don’t select ‘EndsWith’ for a Boolean attribute type and expect it to work. If you choose an operator other than the default (equals in my example) hit enter when prompted for the second time and the script will take an empty response as the end of your choices.

Finally provide what you the value is for the search term for the attribute. If the value has spaces, don’t worry about putting the value in quotes. The script takes care of that.

The last two steps will iterate through, for queries where you have chosen multiple attributes.

And you’re done. $query is the variable that contains the results. In line 115 I’m using Show-Object from the PowershellCookBook PSM. That then gives you a GUI representation of the result as shown below. If the query returns multiple results this will only show the last.

Line 114 outputs the value of the attributes ($query.attributes) to the console as well. If you have multiple objects returned $query will show them as shown below.

Finally if you want to run the query again, or just make a subtle change, you shouldn’t have to go through that again. Get the value of $querytxt and you’ll get the query and the command to execute it. $querytxt is also output to the console as shown below. Copy and paste it into Powershell ISE, update and execute.

The Script

Here is the raw script. Hardly any error handling etc, but enough to get you started and tailor it for your requirements. Enjoy.

Scripting the generation & creation of Microsoft Identity Manager Sets/Workflows/Sync & Management Policy Rules with the Lithnet Resource Management PowerShell Module

Introduction

Yes, that title is quite a mouthful. And this post is going to be quite long. But worth the read if you are having to create a number of rules in Microsoft/Forefront Identity Manager, or even more so the same rule in multiple environments (eg. Dev, Staging, Production).

My colleague David Minnelli introduced using the Lithnet RMA PowerShell Module and the Import-RMConfig cmdlet recently for bulk creation of MIM Sets and MPR’s. David has a lot of the background on Import-RMConfig and getting started with it. Give that a read for a more detailed background.

In this post I detail using Import-RMConfig to create a Set, Workflow, Synchronization Rule and Management Policy Rule to populate a Development AD Domain with Users from a Production AD Domain. This process is designed to run on a combined MIM Service/Sync Server. If your roles a separated (as they likely will be in a Production environment) you will need to run these scripts on the MIM Sync Server (so it can query the Management Agents, and you will need to add in a line to connect to the MIM Service (eg. Set-ResourceManagementClient ) at the beginning of the script.

In my environment I have two Active Directory Management Agents, each connected to an AD Forest as shown below.

On each of the AD MA’s I have a Constant Flow Attribute (named Source) configured to flow in a value representing the source AD Forest. I’m doing this in my environment as I have more than one production forest (hence the need for automation). You could simply use the Domain attribute for the Set criteria. That attribute is used in the Set later on. Mentioning it up front so it make sense.

Overview

The Import-RMConfig cmdlet uses XML and XOML files that contain the configuration required to create the Set, Workflow, Sync Rule and MPR in the FIM/MIM Service. The order that I approach the creation is, Sync Rule, Workflow, Set and finally the MPR.

Each of these objects as indicated above leverage an XML and/or XOML input file. I’ve simplified base templates and included them in the scripts.

The Sync Rule Script includes a prompt to choose a folder (you can create one through the GUI presented) to store the XML/XOML files to allow the Import-RMConfig to use them. Once generated you can simply reference the files with Import-RMConfig to replicate the creation in another environment.

Creating the Synchronization Rule

For creation of the Sync Rule we need to define which Management Agent will be the target for our Sync Rule. In my script I’ve automated that too (as I have a number to do). I’m querying the MIM Sync Server for all its Active Directory MA’s and then providing a dialog to allow you to choose the target MA for the Sync Rule. That dialog simply looks like the one below.

Creating the Sync Rule will finally ask you to give the Rule a name. This name will then be used as the base Display Name for the Set, MPR and Workflow (and a truncated version as the Rule ID’s).

The script below in the $SyncRuleXML section defines the rules of the Sync Rule. Mine is an Outbound Sync Rule, with a base set of attributes and transforming the users UPN and DN (for the differing Development AD namespace). Update lines 42 and 45 for the users UPN and DN your namespace.

Creating the Workflow

The Workflow script is pretty self-explanatory. A simple Action based workflow and is below.

Creating the Set

The Set is the group of objects that will be synchronized to the target management agent. As my Sync Rule is only for Users my Set is also only contains users. As stated in the Overview I have an attribute that defines the authoritative source for the objects. I’m also using that in my Set criteria.

Creating the Management Policy Rule

The MPR ties everything together. Here’s that part of the script.

Tying them all together

Here is the end to end automation, and the raw script that you could use as the basis for automating similar rule generation. The Sync Rule could easily be updated for Contacts or Groups. Remember the attributes and object classes are case sensitive’.

  • Through the Browse for Folder dialog I created a new folder named ProvisionDevAD

  • I provided a Display Name for the rules

  • I chose the target Management Agent

  • The SyncRule, Workflow, Set and MPR are created. The whole thing takes seconds.

  • Script Complete

Let’s take a look at the completed objects through the MIM Portal.

Sync Rule

The Sync Rule is present as we named it. Including the !__ prefix so it appears at the top of the list.

Outbound Sync Rule based on a MPR, Set and Workflow

The Resources will be created and if deleted de-provisioned.

And our base attribute flows.

Set

Our Set was created.

Our naming aligns with what we input

And a Criteria based Set. As per the Overview I have an attribute populated by a Constant flow rule that I based my set on. You’ll want to update for you environment.

Workflow

The Action Workflow was created

All looks great

And it applies our Sync Rule

MPR

And finally our MPR. Created as a Transition In MPR with Action Workflow

Set Transition and naming all aligned

The Transition Set configured for the Set that was created

And the Workflow configured with the Workflow that was just created

Summary

When you have a lot of Sync Rules to create, or you know you will need to re-create them numerous times, potentially in different environments automation is key. This just scratches the surface on what can be achieved, and made so much easier using the Lithnet PowerShell Modules.

Here’s is the full script. Note: You’ll need to make a couple of minor changes as indicated earlier, but you should be able to create a Provisioning Rule end to end pretty quick to validate the process. Then customize accordingly for your environment and requirements. Enjoy.

FIM: An object with DN “CN=BLAH” already exists in management agent “BLAH MA”

If you treat the FIM Synchronization Service well and your configuration is good, it will reward you and the ‘magic’ will happen. At my customer site, the ‘magic’ stopped working and I was faced with an increasing number of synchronization errors being ‘An object with DN “CN=<blah>” already exists in management agent “<blah> MA” ‘. For users that were already provisioned correctly, the FIM Synchronization Service was attempting to re-provision a duplicate object in the destination directory but obviously the account already existed. Why was this occurring and how to resolve it? To be honest it stumped me for quite a while until I remembered I was faced with the same scenario a few years ago with yet another FIM Synchronization Service with many 100s of thousands of objects.

Before I start, I’ll preface that this blog assumes a very good working knowledge of the Microsoft Forefront Identity Manager product, now known as Microsoft Identity Manager. Let’s delve deeper.

We should all know that best practise when making changes to any part of the FIM Synchronization Service configuration, or new or updates to FIM Service Synchronization Rules, you should do a Full Synchronization cycle to ensure the changes and rules are all processed against the objects in the solution. When you are working on a large solution with 100s of thousands of objects, it is not always possible to do this Full Sync cycle at will and it needs to be typically scheduled as a change over a weekend, maybe even a long weekend! We had attempted a full synchronization cycle on this customer solution in the past and it took just on 2 days to complete. So when you don’t have the opportunity perform this cycle, you use your prior knowledge, expertise and judge if the solution can continue to function if a change has been made. In my case, a new attribute flow in a Synchronization Rule which was not critical to the existing objects in the solution. So I thought I could avoid a Full Sync cycle and everything appeared to be working as expected.

Over a few weeks, these sync errors started to occur as illustrated below and they were increasing until I got to around 40 errors and further troubleshooting action needed to be taken.

The customer had rules in the solution to detect when an existing user account was disabled and perform some functions in the FIM Service. If the user account was re-enabled, the ‘provisioning’ MPR was triggered as the user was back as a member in the ‘provision accounts’ Transition Set. Workflows assigned a handful of Synchronization Rules against the user as Expected Rule Entries (EREs), which had already applied to the user when it first came to existence and provisioned. These duplicate EREs when flowed from the FIM Service to the FIM Synchronization Service were throwing the errors.

Back in the test environment, the ‘magic’ was still working with no errors thrown. What normally happens in this situation is if you happen to assign a provisioning Sync Rule to a user account, but that user already had one previously successfully ‘applied’ (and was already provisioned), this duplicate ERE would be processed by the Sync Service and also marked as ‘Applied’. It was smart enough to know that the provisioning event was not required because an existing ERE had already been assigned. When the Sync Service flows the status of the duplicate ERE back to the FIM Service, the duplicate is deleted and world order is restored with this ‘magic’. Sorry I find no way of easily describing it without detailing my scenario, so I hope you are still following me. So why was it not working in production?

I’m not sure of the internal innards of the Sync Service, but what I have found is when a delta import change of a Synchronization Rule flows into the Sync Service and a ‘Delta Sync’ is processed on this change, it seems to have an adverse effect on the other Sync Rules. Sync Rules that have previously been working fine start behaving badly and to put it simply, the ‘magic’ breaks.

The fix is to obviously perform a Full Synchronization on the whole solution because a configuration item has changed. However, without having the easy opportunity to do this, I needed to perform a Preview/Full Synchronization on each Synchronization Rule in the solution as illustrated below. Although no obvious change is shown in the preview window, it seems like each Sync Rule is re-processed/reset into good working order and the ‘magic’ is restored.

After performing this change against all Sync Rule objects, all synchronization errors have been cleared and the ‘magic’ now works as expected. I didn’t find much in the forums or public domain about this scenario, so I hope this helps somebody in the future. I’m now working on getting the full synchronization cycle actioned! Let me know in the comments below if this has saved you.