commands

How to configure a Graphical PowerShell Dev/Admin/Support User Interface for Azure/Office365/Microsoft Identity Manager

During the development of an identity management solution I find myself with multiple PowerShell/RDP sessions connected to multiple environments using different credentials often to obtain trivial data/information. It is easy to trip yourself up as well with remote powershell sessions to differing environments. If only there was a simple UI that could front-end a set of PowerShell modules and make those simple queries quick and painless. Likewise to allow support staff to execute a canned set of queries without providing them elevated permissions.

I figured someone would have already solved this problem and after some searching with the right keywords I found the powershell-command-executor-ui from bitsofinfo . Looking into it he had solved a lot of the issues with building a UI front-end for PowerShell with the powershell-command-executor and the stateful-process-command-proxy. That solution provided the framework for what I was thinking. The ability to provide a UI for PowerShell using powershell modules including remote powershell was exactly what I was after. And it was built on NodeJS and AngularJS so simple enough for some customization.

Introduction

In this blog post I’ll detail how I’ve leveraged the projects listed above for integration with;

Initially I had a vision of serving up the UI from an Azure WebApp. NodeJS on Azure WebApp’s is supported, however with all the solution dependencies I just couldn’t get it working.

My fallback was to then look to serve up the UI from a Windows Server 2016 Nano Server. I learnt from my efforts that a number of the PowerShell modules I was looking to provide a UI for, have .NET Framework dependencies. Nano Server does not have full .NET Framework support. Microsoft state to do so would mean the server would no longer be Nano.

For now I’ve deployed an Azure Windows Server 2016 Server secured by an Azure NSG to only allow my machine to access it. More on security later.

Overview

Simply, put the details in Github for the powershell-command-executor provide the architecture and integration. What I will detail is the modifications I’ve made to utilize the more recent AzureADPreview PowerShell Module over the MSOL PowerShell Module. I also updated the dependencies of the solution for the latest versions and hooked it into Microsoft Identity Manager. I also made a few changes to allow different credentials to be used for Azure and Microsoft Identity Manager.

Getting Started

I highly recommend you start with your implementation on a local development workstation/development virtual machine. When you have a working version you’re happy with you can then look at other ways of presenting and securing it.

NodeJS

NodeJS is the webserver for this solution. Download NodeJS for your Windows host here. I’m using the 64-bit version, but have also implemented the solution on 32-bit. Install NodeJS on your local development workstation/development virtual machine.

You can accept all the defaults.

Following the installation of NodeJS download the powershell-command-executor-ui from GitHub. Select Clone 0r Download, Download ZIP and save it to your machine.

Right click the download when it has finished and select Extract All. Select Browse and create a folder at the root of C:\ named nodejs. Extract powershell-command-executor-ui.

Locate the c:\nodejs\powershell-command-executor-ui-master\package.json file.

Using an editor such as Notepad++ update the package.json file ……

…… so that it looks like the following. This will utilise the latest versions of the dependencies for the solution.

From an elevated (Administrator) command prompt in the c:\nodejs\powershell-command-executor-ui-master directory run “c:\program files\nodejs npm” installThis will read the package.json file you edited and download the dependencies for the solution.

You can see in the screenshot below NodeJS has downloaded all the items in package.json including the powershell-command-executor and stateful-process-command-proxy.

When you now list the directories under C:\nodejs\powershell-command-executor-ui-master\node_modules you will see those packages and all their dependencies.

We can now test that we have a working PowerShell UI NodeJS website. From an elevated command prompt whilst still in the c:\nodejs\powershell-command-executor-ui-master directory run “c:\Program Files\nodejs\node.exe” bin\www

Open a browser on the same host and go to http://localhost:3000”. You should see the default UI.

Configuration and Customization

Now it is time to configure and customize the PowerShell UI for our needs.

The files we are going to edit are:

  • C:\nodejs\powershell-command-executor-ui-master\routes\index.js
    • Update Paths to the encrypted credentials files used to connect to Azure, MIM. We’ll create the encrypted credentials files soon.
  • C:\nodejs\powershell-command-executor-ui-master\public\console.html
    • Update for your customizations for CSS etc.
  • C:\nodejs\powershell-command-executor-ui-master\node_modules\powershell-command-executor\O365Utils.js
    • Update for PowerShell Modules to Import
    • Update for Commands to make available in the UI

We also need to get a couple of PowerShell Modules installed on the host so they are available to the site. The two I’m using I’ve mentioned earlier. With WMF5 intalled using Powershell we can simply install them as per the commands below.

Install-Module AzureADPreview
Install-Module LithnetRMA

In order to connect to our Microsoft Identity Manager Synchronization Server we are going to need to enable Remote Powershell on our Microsoft Identity Manager Synchronization Server. This post I wrote here details all the setup tasks to make that work. Test that you can connect via RPS to your MIM Sync Server before updating the scripts below.

Likewise for the Microsoft Identity Manager Service Server. Make sure after installing the LithnetRMA Powershell Module you can connect to the MIM Service using something similar to:

# Import LithnetRMA PS Module
import-module lithnetrma

# MIM AD User Admin
$username = "mimadmin@mim.mydomain.com"
# Password 
$password = "Secr3tSq1rr3l!" | convertto-securestring -AsPlainText -Force
# PS Creds
$credentials = New-Object System.Management.Automation.PSCredential $Username,$password

# Connect to the FIM service instance
# Will require an inbound rule for TCP 5725 (or your MIM Service Server Port) in you Resource Group Network Security Group Config
Set-ResourceManagementClient -BaseAddress http://mymimportalserver.:5725 -Credentials $credentials

 

\routes\index.js

This file details the encrypted credentials the site uses. You will need to generate the encrypted credentials for your environment. You can do this using the powershell-credentials-encryption-tools. Download that script to your workstation and unzip it. Open the credentialEncryptor.ps1 script using an Administrator PowerShell ISE session.

I’ve changed the index.js to accept two sets of credentials. This is because your Azure Admin Credentials are going to be different from your MIM Administrator Credentials (both in name and password). The username for my Azure account looks something like myname@mycompany.com whereas for MIM it is Domainname\Username.

Provide an account name for your Azure environment and the associated password.

The tool will create the encrypted credential files.

Rename the encrypted.credentials file to whatever makes sense for your environment. I’ve renamed it creds1.encrypted.credentials.

Now we re-run the script to create another set of encrypted credentials. This time for Microsoft Identity Manager. Once created, rename the encrypted.credentials file to something that makes sense in your environment. I’ve renamed the second set to creds2.encrypted.credentials.

We now need to copy the following files to your UI Website C:\nodejs\powershell-command-executor-ui-master directory:

  • creds1.encrypted.credentials
  • creds2.encrypted.credentials
  • decryptUtil.ps1
  • secret.key

Navigate back to Routes.js and open the file in an editor such as Notepad++

Update the index.js file for the path to your credentials files. We also need to add in the additional credentials file.

The changes to the file are, the paths to the files we just copied above along with the addition var PATH_TO_ENCRYPTED_RPSCREDENTIALS_FILE for the second set of credentials used for Microsoft Identity Manager.

var PATH_TO_DECRYPT_UTILS_SCRIPT = "C:\\nodejs\\powershell-command-executor-ui-master\\decryptUtil.ps1";
var PATH_TO_ENCRYPTED_CREDENTIALS_FILE = "C:\\nodejs\\powershell-command-executor-ui-master\\creds1.encrypted.credentials";
var PATH_TO_ENCRYPTED_RPSCREDENTIALS_FILE = "C:\\nodejs\\powershell-command-executor-ui-master\\creds2-encrypted.credentials";
var PATH_TO_SECRET_KEY = "C:\\nodejs\\powershell-command-executor-ui-master\\secret.key";


Also to initCommands to pass through the additional credentials file


initCommands: o365Utils.getO365PSInitCommands(
 PATH_TO_DECRYPT_UTILS_SCRIPT,
 PATH_TO_ENCRYPTED_CREDENTIALS_FILE,
 PATH_TO_ENCRYPTED_RPSCREDENTIALS_FILE,
 PATH_TO_SECRET_KEY,
 10000,30000,3600000),

Here is the full index.js file for reference.

 

public/console.html

The public/console.html file is for formatting and associated UI components. The key things I’ve updated are the Bootstrap and AngularJS versions. Those are contained in the top of the html document. A summary is below.

https://ajax.googleapis.com/ajax/libs/angularjs/1.6.1/angular.min.js
https://cdnjs.cloudflare.com/ajax/libs/angular.js/1.6.1/angular-resource.min.js
http://javascripts/ui-bootstrap-tpls-2.4.0.min.js
http://javascripts/console.js
<link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css">
<link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap-theme.min.css">

You will also need to download the updated Bootstrap UI (ui-bootstrap-tpls-2.4.0.min.js). I’m using v2.4.0 which you can download from here. Copy it to the javascripts directory.

I’ve also updated the table types, buttons, colours, header, logo etc in the appropriate locations (CSS, Tables, Div’s etc). Here is my full file for reference. You’ll need to update for your colours, branding etc.

powershell-command-executor\O365Utils.js

Finally the O365Utils.js file. This contains the commands that will be displayed along with their options, as well as the connection information for your Microsoft Identity Manager environment.

You will need to change:

  • Line 52 for the address of your MIM Sync Server
  • Line 55 for the addresses of your MIM Service Server
  • Line 141 on-wards for what commands and parameters for those commands you want to make available in the UI

Here is an example with a couple of AzureAD commands, a MIM Sync and a MIM Service command.

Show me my PowerShell UI Website

Now that we have everything configured let’s start the site and browse to it. If you haven’t stopped the NodeJS site from earlier go to the command window and press Cntrl+C a couple of times. Run “c:\Program Files\nodejs\node.exe” bin\www again from the C:\nodejs\powershell-command-executor-ui-master directory unless you have restarted the host and now have NodeJS in your environment path.

In a browser on the same host go to http://localhost:3000 again and you should see the site as it is below.

Branding and styling from the console.html, menu options from the o365Utils.js and when you select a command and execute it data from the associated service …….

… you can see results. From the screenshot below a Get-AzureADUser command for the associated search string executed in milliseconds.

 

Summary

The powershell-command-executor-ui from bitsofinfo is a very extensible and powerful NodeJS website as a front-end to PowerShell.

With a few tweaks and updates the look and feel can be easily changed along with the addition of any powershell commands that you wish to have a UI for.

As it sits though keep in mind you have a UI with hard-coded credentials that can do whatever commands you expose.

Personally I am running one for my use only and I have it hosted in Azure in its own Resource Group with an NSG allowing outgoing traffic to Azure and my MIM environment. Incoming traffic is only allowed from my personal management workstations IP address. I also needed to allow port 3000 into the server on the NSG as well as the firewall on the host. I did that quickly using the command below.

# Enable the WebPort NodeJS is using on the firewall 
netsh advfirewall firewall add rule name="NodeJS WebPort 3000" dir=in action=allow protocol=TCP localport=3000

Follow Darren on Twitter @darrenjrobinson

mim-db-not-populated

Resolving “The Microsoft Identity Manager server database could not be successfully populated” installation error

Here is yet another of those Microsoft Identity Manager installation errors that doesn’t give you much information and when looking for a resolution you can’t find an exact match through Dr Google.

Nearing the end of the Microsoft Identity Manager Service and Portal installation you receive the “The Microsoft Identity Manager server database could not be successfully populated” error.

Looking into the installation log (which I’m in the good practice of always initiating when doing an installation of the MIM Service/Portal these days eg. msiexec /i “e:\Service and Portal\Service and Portal.msi” /l*v c:\temp\install.log )  didn’t give up much information at all. Fatal Error. Dialog created.

Looking at the server that the installation was being done on I could see that it was being spanked. This server is for a customers development environment, hosted in Azure but also done rather frugally (my Virtual Machine was running, SQL, MIM Sync, all the dependencies for the MIM Portal and then the MIM Service/Portal itself). FWIW the initially provisioned VM was an Azure DS1v2 server. Seems a character may have got lost in the VM request where an Azure DS11v2 server would have been more appropriate.

I re-sized the Azure VM and actually chose to go with an Azure DS3v2 VM size. I kicked off the Microsoft Identity Manager Service & Portal installation again and ….

….. SUCCESS.

Hope this helps someone else who may find themselves in a similar position.

Follow Darren on Twitter @darrenjrobinson

2337-error-only

Microsoft Identity Manager installation error “Internal Error 2337. 0, Microsoft.MetadirectoryServices.host.dll”

Today I was doing a fresh installation of Microsoft Identity Manger 2016 with Service Pack 1 into a new development environment. The exact binary is “en_microsoft_identity_manager_2016_with_service_pack_1_x64_dvd_9270854”

Not too far into the installation of the Microsoft Identity Manager Synchronization Server I got the “Internal Error 2337. 0, Microsoft.MetadirectoryServices.host.dll” error as shown below.

Doing a few searches didn’t throw me any bones. I could see that the installation had added the MIM Sync Server Service Account to the Logins on the SQL Server.

I then recalled that there was an updated version that was released just before Xmas (14 Dec 2016). The exact binary is “en_microsoft_identity_manager_2016_with_service_pack_1_x64_dvd_9656597” and you can get the updated Microsoft Identity Manager 2016 with SP1 media here.

Re-running the installation and ….

….. SUCCESS.

Now I’m not going to spend any time trying to figure out was is bad with the base MIM 2016 SP1 media, I’m just going to pretend it never existed and use the latest. Onward and upwards.

Follow Darren on Twitter @darrenjrobinson

 

 

 

 

 

overview

Using an Azure Function to search the FIM/MIM Metaverse, create a Set and update the Set membership in the the FIM/MIM Service

Introduction

This is the third and last post in this series of integrating Microsoft Identity Manager with Azure Functions.

The first detailed how to use an Azure Function to retrieve data from the MIM Service Server. The second detailed how to use an Azure Function to retrieve data from the MIM Sync (Metaverse) Server.

This third post combines the two and then performs an action in the MIM Service. The practical purpose of this could be functions like “find all users in location y” and “enable them for entitlement x” or “add an attribute value on each of their objects”.

Overview

The reasoning for the two stage approach is that in my experience it is a lot easier to search the Metaverse than the MIM Service to find an object(s), but also the Metaverse has all the information about objects whereas the MIM Service is a ShadowVerse of the Metaverse containing a subset of the managed objects metadata.

Moving forward then the architecture is a hybrid of the first two posts that introduced the concepts associated with integrating MIM with Azure Functions. As per the other two posts this is a base working example and concept.

Prerequisites

The prerequisites are the same as for the 1st and 2nd posts in this series. You’ll need to work through those examples to setup the dependencies and prerequisites. From there you can create one more Azure Function that brings everything together. That’s what I’m covering in this post.

Therefore the prerequisites are;

  • Azure Tenant and a Function Plan
  • Microsoft Identity Manager implementation
  • Remote Powershell configured for your MIM Sync Server
  • Lithnet FIM MIIS Automation Powershell Module installed on your MIM Sync Server
  • The necessary Firewall Rules on your MIM Sync Server and your Azure Network Security Group (assuming your MIM Infrastructure is in Azure) to allow Azure Functions to communicate with MIM Sync and Service Servers

 

This Example performs the following

In this example the HTTP Trigger Azure Function;

  • Takes input for ObjectType, Attribute, AttributeValue, SetName
  • Searches the MIM Sync Metaverse for the input ObjectType, with the AttributeValue in the Attribute
  • Connects to the MIM Service
  • Creates the Set based of the input SetName if it doesn’t exist
  • Adds the objects from the search to the Set
  • Returns the objects added to the Set

In a real world implementation you’d do the above with a criteria based set. This post is a concept of search and find, performing a create and updating. That has many practical applications.

Create your new Azure Function

Just like the other two posts, we’re going to create a new Powershell HTTP Trigger Azure Function.

Upload the Lithnet RMA PS Module to your new Azure Function (as per blog post 1 in this series). You should also be using protected credentials now as well. So upload your username/password encryption key.

 

Here is the Azure Function Powershell Script that performs the process detailed above.

Test it out. Looks good. 88 users matched the value of Sydney in their location attribute.

Verify that the Set was created and the membership updated.

Test calling the Azure Function remotely

Now that it is all working in the Azure Function, lets try doing it from Powershell remotely. This time I’m again looking for Person objects that have Sydney in their location attribute and I’ll create a set named Sydney-NSW and put them in it.

Brilliant, that works nicely. Let’s verify that the Set was created and has the correct number of users in it. Yes, a perfect match.

Summary

Putting Azure Functions and Powershell together along with the Lithnet Powershell Modules opens up a world of possibilities for automation and integration of the MIM Service without the need for any additional infrastructure or any considerable effort.

Experiment and let me know what you do with this style of integration.

Follow Darren on Twitter @darrenjrobinson

solution-arch

Get Users/Groups/Objects from Microsoft/Forefront Identity Manager with Azure Functions and the Lithnet Resource Management Powershell Module

Introduction

As an Identity Management consultant if I had a $1 for every time I’ve been asked “what is user x’s current status in IDAM”, “is user x active?”, “does user x have an account in y?”, “what is user x’s primary email address?”, particularly after Go Live of an IDAM solution my holidays would be a lot more exotic.

From a Service Desk perspective IDAM implementations are often a black box in the middle of the network that for the most part do what they were designed and implemented to do. However as soon as something doesn’t look normal for a user the Service Desk are inclined to point their finger at that black box (IDAM Solution). And the “what is the current value of ..”, “does user x ..” type questions start flying.

What if we could give the Service Desk a simple query interface into FIM/MIM without needing to give them access to another complicated application?

This is the first (of potentially a series) blog post on leveraging community libraries and Azure PaaS services to provide visibility of FIM/MIM information. This first post really just introduces the concept with a working example in an easy way to understand and replicate. It is not intended for production implementation without additional security and optimisation. 

Overview

The following graphic shows the concept of using Azure Functions to take requests from a client (web app, powershell, some other script) query the FIM/MIM Service and return the result. This post details the setup and configuration for the section in the yellow shaded box with the process outlined in the numbers 1, 2 & 3. This post assumes you already have your FIM/MIM implementation setup and configured according to your connected integrated applications/services such as Active Directory. In my example my connected datasource is actually Twitter.

Prerequisites

The prerequisites for this solution are;

 

Creating your Azure App Service

First up you’ll need to create an Azure App Service in your Azure Tenant. To keep everything logically structured for this example I created an Azure App Service in the same resource group that contains my MIM IaaS infrastructure (MIM Sync Server, MIM Service Server, SQL Server, AD Domain Controller etc).

In the Azure Marketplace select New (+) and search for Function App. Select the Function App item from the results and select Create.

Give your Azure App Service a name, choose the Resource Group where you want to locate it. Choose Dynamic for the Hosting Plan. This means you don’t have to worry about resource management for your Web App and you only pay for execution time which unless you put this into production and have gone crazy with it your costs should be zero as they will (should) be well under the free grant tier.  Put the application in the appropriate location such as close to your FIM/MIM resources that it’ll be interacting with and select Create.

Now that you have your Azure App Service setup, you need to create your Azure Function.

Create an HTTP Trigger Powershell Function App

In the Azure Portal locate your App Services Blade and select the Function App created in the steps above. Mine was named MIMMetaverseSearch in the example above. Select PowerShell as the Language and HttpTrigger-Powershell as the Function type.

Give your Function a name. I’ve kept it simple in this example and named it the same as my App Service Plan. Select Create.

Adding the Lithnet Powershell Module into your Function App.

As you’d expect the Powershell Function App by default only has a handful of core Powershell Modules. As we’re using something pretty specific we’ll need put the module into our Function App so we can load it and use the library.

Download and save the Lithnet Resource Management Powershell Module to your local machine. Something like the Powershell command below will do that.

Next follow this great blog post here from Tao to upload the Lithnet RMA PS Module you downloaded earlier into your function directory. I used WinSCP as my FTP client as I’ve shown below to upload the Lithnet RMA PS Module.

FTP to the host for your App Service and navigate to the /site/wwwroot/

Create a bin folder and upload via FTP the Lithnet RMA PS Module.

Using Kudu navigate to the path and version of the Lithnet RMA PS Module.
I’m using v1.0.6088 and my app is named MIMMetaverseSearch so MY path is D:\home\site\wwwroot\MIMMetaverseSearch\bin\LithnetRMA\1.0.6088

Note: the Lithnet RMA PS Module is 64-bit so you’ll need to configure your Web App for 64-bit as per the info in the same blog you followed to upload the module here.

Test loading the Lithnet RMA PS Module in your Function App

In your Function App select </> Develop. Remove the sample script and in your first line import the Lithnet RMA PS Module using the path from the previous step. Then, to check that it loads add a line that references a cmdlet in the module. I used Help Get-Resource. Select Save then Run.

If you’ve done everything correct when you select Run and look at the Logs you’ll see the module was loaded and the Help Get-Resource command was run in the Logs.

Allow your Function App to access your FIM/MIM Service Server

Even though you have logically placed your Function App in the same Resource Group (if you did it like I have) you’ll need to actually allow the Function App that is running in a shared PaaS environment to connect to your FIM/MIM Service Server.

Create an inbound rule in your Network Security Group to allow access to your FIM/MIM Service Server. The example below isn’t as secure as it could (and should) be as it allows access from anywhere. You should restrict the source of the request(s) accordingly. I’m just showing how to quickly get a working example. TCP Port 5725 is required to access your MIM Service Server. Enter the details as per below and select Ok.

Using an Azure Function to query FIM/MIM Service

Note: Again, this is an example to quickly show the concept. In the script below your credentials are in the script in clear text (and of course those below are not valid). For anything other than validating the concept you must protect your credentials. A great example is available here in Tao’s post.

The PS Azure Function gets the incoming request and converts it from JSON. In my request which you’ll see in the next step I’m passing in “displayName” and “objectType”.

In this example I’m using Get-Resource from the Lithnet RMA to get an object from the FIM/MIM Service. First you need to open a connection to the FIM/MIM Service Server. On my Azure IaaS MIM Service Server I’ve configured a DNS name so you can see I’m using that name in line 17 to connect to it using the unsecured credentials from earlier in the script. If you haven’t set up a DNS name for your FIM/MIM Service Server you can use the Public IP Address instead.

Line 20 queries for the ObjectType and DisplayName passed into the Function (see calling the Function in the next step) and returns the response in line 22. Again this is just an example. There is no error checking, validation or anything. I’m just introducing the concept in this post.

Testing your Function App

Now that you have the function script saved, you can test it from the Function App itself. Select Test from the options up in the right from your function. Change the Request Body for what the Function is expecting. In my case displayname and objectType. Select Run and in the Logs if you’ve got everything configured correctly (like inbound network rules, DNS name, your FIM/MIM Service Server is online, your query is for a valid resource) you should see an object returned.

Calling the Function App from a Client

Now that we have our Function App all setup and configured (and tested in the Function App) let’s send a request to the Azure Function using the Powershell Invoke-RestMethod function. The following call I did from my laptop. It is important to note that there is no authN in this example and the function app will be using whatever credentials you gave it to execute the request. In a deployed solution you’ll need to scope who can make the requests, limit on the inbound network rules who can submit requests and of course further protect the account credentials used to connect to your FIM/MIM Service Server.

Successful Response

The following screenshot shows calling the Function App and getting the responding object. Success. In a couple of lines I created a hashtable for the request, converted it to JSON and submitted it and got a response. How powerful is that!?

 

Summary

Using the awesome Lithnet Resource Management PowerShell Module with Azure Functions it is pretty quick and flexible to access a wealth of information we may want to expose for business benefit.

Now if only there was an affiliation program for Azure Functions that could deposit funds for each IDAM request to an Azure Functions App into my holiday fund.

Stay tuned for more posts on taking this concept to the next level.

Follow Darren on Twitter @darrenjrobinson

Active Directory – What are Linked Attributes?

A customer request to add some additional attributes to their Azure AD tenant via Directory Extensions feature in the Azure AD Connect tool, lead me into further investigation. My last blog here set out the customer request, but what I didn’t detail in that blog was one of the attributes they also wanted to extend into Azure AD was directReports, an attribute they had used in the past for their custom built on-premise applications to display the list of staff the user was a manager for. This led me down a rabbit hole where it took a while to reach the end.

With my past experience in using Microsoft Identity Manager (formally Forefront Identity Manager), I knew directReports wasn’t a real attribute stored in Active Directory, but rather a calculated value shown using the Active Directory Users and Computers console. The directReports was based on the values of the manager attribute that contained the reference to the user you were querying (phew, that was a mouthful). This is why directReport and other similar type of attributes such as memberOf were not selectable for Directory Extension in the Azure AD Connect tool. I had never bothered to understand it further than that until the customer also asked for a list of these type of attributes so that they could tell their application developers they would need a different technique to determine these values in Azure AD. This is where the investigation started which I would like to summarise as I found it very difficult to find this information in one place.

In short, these attributes in the Active Directory schema are Linked Attributes as detailed in this Microsoft MSDN article here:

Linked attributes are pairs of attributes in which the system calculates the values of one attribute (the back link) based on the values set on the other attribute (the forward link) throughout the forest. A back-link value on any object instance consists of the DNs of all the objects that have the object’s DN set in the corresponding forward link. For example, “Manager” and “Reports” are a pair of linked attributes, where Manager is the forward link and Reports is the back link. Now suppose Bill is Joe’s manager. If you store the DN of Bill’s user object in the “Manager” attribute of Joe’s user object, then the DN of Joe’s user object will show up in the “Reports” attribute of Bill’s user object.

I then found this article here which further explained these forward and back links in respect of which are writeable and which are read-only, the example below referring to the linked attributes member/memberOf:

Not going too deep into the technical details, there’s another thing we need to know when looking at group membership and forward- and backlinks: forward-links are writable and backlinks are read-only. This means that only forward-links changed and the corresponding backlinks are computed automatically. That also means that only forward-links are replicated between DCs whereas backlinks are maintained by the DCs after that.

The take-out from this is the value in the forward-link can be updated, the member attribute in this case, but you cannot update the back-link memberOf. Back-links are always calculated automatically by the system whenever an attribute that is a forward-link is modified.

My final quest was to find the list of linked attributes without querying the Active Directory schema which then led me to this article here, which listed the common linked attributes:

  • altRecipient/altRecipientBL
  • dLMemRejectPerms/dLMemRejectPermsBL
  • dLMemSubmitPerms/dLMemSubmitPermsBL
  • msExchArchiveDatabaseLink/msExchArchiveDatabaseLinkBL
  • msExchDelegateListLink/msExchDelegateListBL
  • publicDelegates/publicDelegatesBL
  • member/memberOf
  • manager/directReports
  • owner/ownerBL

There is further, deeper technical information about linked attributes such as “distinguished name tags” (DNT) and what is replicated between DCs versus what is calculated locally on a DC, which you can read in your own leisure in the articles listed throughout this blog. But I hope the summary is enough information on how they work.

Office365 Licensing Management Agent for Microsoft Identity Manager

Licensing for Office365 has always been a moving target for enterprise customers. Over the years I’ve implemented a plethora of solutions to keep licensing consistent with entitlement logic. For some customers this is as simple as everyone gets say, an E3 license. For other institutions there are often a mix of ‘E’ and ‘K’ licenses depending on EmployeeType.

Using the Granfeldt PowerShell Management Agent to import Office365 Licensing info

In this blog post I detail how I’m using Søren Granfeldt’s extremely versatile PowerShell Management Agent yet again. This time to import Office365 licensing information into Microsoft Identity Manager.

I’m bringing in the licenses associated with users as attributes on the user account. I’m also bringing in the licenses from the tenant as their own ObjectType into the Metaverse. This includes the information about each license such as how many licenses have been purchased, how many licenses have been issued etc.

Overview

I’m not showing assigning licenses. In the schema I have included the LicensesToAdd and LicensesToRemove attributes. Check out my Adding/Removing User Office365 Licences using PowerShell and the Azure AD Graph RestAPI post to see how to assign and remove licenses using Powershell. From that you can workout your logic to implement an Export flow to manage Office365 licenses.

Getting Started with the Granfeldt PowerShell Management Agent

If you don’t already have it, what are you waiting for. Go get it from here. Søren’s documentation is pretty good but does assume you have a working knowledge of FIM/MIM and this blog post is no different.

Three items I had to work out that I’ll save you the pain of are;

  • You must have a Password.ps1 file. Even though we’re not doing password management on this MA, the PS MA configuration requires a file for this field. The .ps1 doesn’t need to have any logic/script inside it. It just needs to be present
  • The credentials you give the MA to run this MA are the credentials for the account that has permissions to the Office365 Tenant. Just a normal account is enough to enumerate it, but you’ll need additional permissions to assign/remove licenses.
  • The path to the scripts in the PS MA Config must not contain spaces and be in old-skool 8.3 format. I’ve chosen to store my scripts in an appropriately named subdirectory under the MIM Extensions directory. Tip: from a command shell use dir /x to get the 8.3 directory format name. Mine looks like C:\PROGRA~1\MICROS~2\2010\SYNCHR~1\EXTENS~2\O365Li~1

Schema.ps1

My Schema is based around the core Office365 Licenses function. You’ll need to create a number of corresponding attributes in the Metaverse Schema on the Person ObjectType to flow the attributes into. You will also need to create a new ObjectType in the Metaverse for the O365 Licenses. I named mine LicensePlans. Use the Schema info below for the attributes that will be imported and the attribute object types to make sure what you create in the Metaverse aligns, so you can import the values. Note the attributes that are multi-valued.

Import.ps1

The logic which the Import.ps1 implements I’m not going to document here as this post goes into all the details Enumerating all Users/Groups/Contacts in an Azure tenant using PowerShell and the Azure Graph API ‘odata.nextLink’ paging function

Password Script (password.ps1)

Empty as not implemented

Export.ps1

Empty as not implemented

Management Agent Configuration

As per the tips above, the format for the script paths must be without spaces etc. I’m using 8.3 format and I’m using an Office 365 account to connect to Office365 and import the user and license data.

As per the Schema script earlier in this post I’m bringing user licensing metadata as well as the Office365 Tenant Licenses info.

Attributes to bring through aligned with what is specified in the Schema file.

Flow through the attributes to the attributes I created in the Metaverse on the Person ObjectType and to the new ObjectType LicensePlans.

Wiring it up

To finish it up you’ll need to do the usual tasks of creating run profiles, staging the connector space from Office365 and importing into the Metaverse.

Enjoy.

Follow Darren on Twitter @darrenjrobinson

datasample1

A Twitter Management Agent for Microsoft Identity Manager

In the last couple of weeks I’ve been evaluating a number of different approaches/concepts for some upcoming MIM development projects. Some of these I’ve blogged about already.

Having an Identity Manager Metaverse with identity data is a key dependency to being able to validate ideas and concepts. So what’s a good source of some interesting and varied identity data with string, integer, reference, and boolean attributes? Twitter? Yeah why not. There’s an API. Should be pretty quick to get some sample data right?

In this blog post I’m going to give an overview of creating a PowerShell Management Agent to consume Twitter identities and their data into Microsoft Identity Manager. I’ll cover;

  • Obtaining Twitter user data from Twitter using Powershell and the Twitter RestAPI
  • Using Søren’s Powershell Management Agent to import Twitter user data obtained via the RestAPI
  • Manipulating the Twitter data into the MIM Synchronisation Engine

Twitter Data

Here’s an overview of my approach/rationale of what data I was looking for and how I got it from Twitter;

  • I don’t need real-time data. Just identity data
  • I need data of all different data-types
  • I need data with all the randomness that identity data often contains
  • I created a standalone script that took a seed Twitter identity (one of my accounts) and;
    • obtained the Twitter account info including the list of the Twitter accounts it followed
    • the Twitter accounts that follows it
  • The standalone script uses the Twitter RestAPI to obtain the data and respected the service and rate-limits
  • To make the Twitter API calls easy I leveraged the awesome InvokeTwitterAPI Powershell Module from Shannon Conley & Mehmet Kaya available here https://github.com/MeshkDevs/InvokeTwitterAPIs . I notice that there is an updated version from Marc R Kellerman available here https://www.powershellgallery.com/packages/InvokeTwitterAPIs/2.1/Content/InvokeTwitterAPIs.psm1 that was released after I had done most of my work. Notably it supports having multiple OAuth keys and the ratelimit restrictions. The details below leverage this updated version.

Here is what a sample of some of the data looks like in the Metaverse.

Pre-requisites

You need to enable your Twitter Account for API access. Follow the details here 

Getting the Seed Twitter account info

By now you should have downloaded the Twitter PowerShell API Modules and installed them. If you haven’t get WMF5 installed and run the install-module command as shown below in Line 1.

Modify the script below to give the;

  • Seed Twitter Account you want to bring in the Friends and Followers for as users into the MIM Metaverse
  • The API keys asscoiated with your Twitter account(s) you’re going to use to query the Twitter API
  • The directory you want to dump the account info out to

….. and let it loose.

Now we have two XML files with a whole bunch of Twitter accounts and their metadata. There is an almost certainty that the seed account you’ve used is both followed by twitter accounts that you also follow. We’re going to need to remove the duplicates so that when we import the Twitter accounts into MIM we don’t have duplicates.

Basic, basic script to read in both files and spit out the unique Twitter Accounts is shown below.

Using the Granfeldt PowerShell Management Agent to import Twitter Identities

Consuming data into the MIM Sync Engine obtained via PowerShell is quick and simple utilising Søren Granfeldt’s extremely versatile PowerShell Management Agent. I’m just going to cover importing the data from the XML file we generated above.

Getting Started with the Granfeldt PowerShell Management Agent

First up, you can get it from here. Søren’s documentation is pretty good but does assume you have a working knowledge of FIM/MIM and this blog post is no different.

A few items of note are;

  • You must have a Password.ps1 file. Even though we’re not doing password management on this MA, the PS MA configuration requires a file for this field. The .ps1 doesn’t need to have any logic/script inside it. It just needs to be present
  • Same for an Export.ps1 file. I’m not doing any exports on the MA, but an export script must be present.
  • The credentials you give the MA to run this MA are irrelevant as they aren’t used as part of the import as I’m bringing in data from files generated via separate PowerShell scripts
  • The path to the scripts in the PS MA Config must not contain spaces and be in old-skool 8.3 format. I’ve chosen to store my scripts in an appropriately named subdirectory under the MIM Extensions directory. Tip: from a command shell use dir /x to get the 8.3 directory format name. Mine looks like C:\PROGRA~1\MICROS~2\2010\SYNCHR~1\EXTENS~2\Twitter

Import Twitter Users into Microsoft Identity Manager

Using the guidance above on the Granfeldt PSMA here are the two key scripts for the Twitter MA.

The Schema Script to expose the core Twitter identity attributes.

Schema Script

Import Script

The Import Script that takes the rationalised XML file created earlier from the friends and followers queries and populates the connector space.

Password Script

Required by the PSMA but not used as detailed earlier

Export Script

Required by the PSMA but not used as detailed earlier

Creating the Management Agent

Path to the Schema Script in 8.3 format as detailed earlier.

Path to the Import, Export and Password scripts also in 8.3 format.

Select the attributes you want to bring in to the connector space.

Any Join logic, and a Projection Rule.

Import Flow Rules to bring in the Tweeters.

Create your Run Profiles, perform a Stage and Full Sync and BAM; Tweeters in the Metaverse. Real Word Identity Data Ahoy. Exception testing commences now.

Follow Darren on Twitter @darrenjrobinson

pbi4

Simple reporting from the FIM/MIM Metaverse to PowerBI using the Lithnet FIM/MIM Sync Service PowerShell Module

I have a customer that is looking to report on FIM/MIM identity information. The reports they are looking for aren’t overly complex and don’t necessarily justify the need the full FIM/MIM reporting infrastructure. So I spent a few hours over a couple of days looking at alternatives. In this blog post I give an overview of using the awesome Lithnet FIM/MIM Sync Service PowerShell Module recently released from Ryan Newington to do basic reporting on the Microsoft (Forefront) Identity Manager Metaverse into PowerBI.

I’ll briefly show how to leverage the Lithnet FIM/MIM Sync Service PowerShell Module to extract Person objects and their metadata (based on a search filter criteria) from the MIM/FIM Metaverse and output to a file for PowerBI.

I cover;

  • Building a query
  • Executing the query
  • Filtering the results for output to a file (CSV)
  • Importing to PowerBI as a dataset, creating a report showing results in a Dashboard

First up you’ll need to download and install the module from https://github.com/lithnet/miis-powershell

Using the FIM/MIM Sync Service PowerShell Module to query the Metaverse

What operators you can choose on your attribute types (boolean, string, integer, reference etc) in the Metaverse Search function in the Synchronisation Service Manager you can also perform using the Lithnet FIM/MIM Sync Service PowerShell Module.

By creating a search with multiple criteria in the Metaverse Search you can filter the results from the Metaverse.

As shown below you can see that we get 302 results.

So let’s import the Lithnet FIM/MIM Sync Service PowerShell Module, create a filter execute it and look at the results. As you’d expect we get the same result. Excellent.

Remember that using this PowerShell automation module, the backend is still the WMI interface behind the Synchronisation Service Manager. This means you can’t for example create a query filter using “greater than/less than” if you can’t do it in the UI.

Take my Twitter FriendsCount attribute of type Number/Integer as an example.

I can’t create a query filter that would return results where FriendsCount > 20,000. I can only use the IsPresent, IsNotPresent and Equals.

On a sidenote the PowerShell error message will give you a hint at what operators you can use as shown below.

However, if you try and use StartsWith for an Integer attribute the search will execute but just return no results. My tip then is define your query in the Metaverse Search GUI and when you get what results you want/expect, create the equivalent query in PowerShell and validate you get the same number of results.

Final note on query filters. Multiple criteria are an AND operation filter, NOT OR.

Let’s do something with the results

Now that we have a query sorted let’s do something with the results. The result set is the full attribute list and values for each associated object that matched our query from the Metaverse. That’s way more info than what I and probably you need as well. So iterate through the results, pull out the attribute values that we want to do something with and export them as a CSV file.

What to do with the output ?

For this overview I’ve just chosen the local file (CSV) that I exported as part of the script as the input dataset in PowerBI. https://app.powerbi.com

On the right hand side I’ve chosen the columns that were exported to the CSV and they appear in the main window.

Click Pin to Live Page. You’ll be prompted to save the report first so do that then I choose New Dashboard for the report. Click Pin live.

I can then refine and get some visual reports quickly using text based queries using keywords from the dataset columns. Like Top 10 by number of friends from the dataset.

Create a couple of queries and pin them to the Dashboard and the data comes to life.

Summary

The Lithnet FIM/MIM Sync Service PowerShell Module provides a really easy way to expose information from the Metaverse that may satisfy many reporting and other requirements. Taking the concept further it wouldn’t be too complex to export the data to an Azure SQL DB on a schedule and have the results dynamically update on a PowerBI Dashboard.
The concept of exporting data for reporting is just one practical example using the tools. Huge thanks to Ryan for creating the Lithnet tools and publishing to the community. Keep in mind the tools disclaimer too.

Here is the sample PowerShell.

Follow Darren on Twitter @darrenjrobinson

Exception from HRESULT 0x80230729 creating a new FIM/MIM Management Agent

Another day, another piece of FIM/MIM experimentation. I had built a fresh MIM 2016 environment in Azure to test a few scenarios out. That all went quick and seamlessly thanks to some great templates and a few scripts. Until I came to create the management agent (the purpose of today’s experimentation).

It didn’t matter if I tried to Create a New Management Agent or Import the Management Agent. I just got “Exception from HRESULT 0x80230729”. The common element however was that the Management Agent I was creating was based off a 3rd party MA based on Microsoft’s Extensible Connectivity Management Agent (ECMA). Specifically I was using Soren Granfeldts PowerShell MA.

HResult 0x80230729

Now I’ve used this MA extensively and not had a problem previously.

So I retraced my steps, clean build, pre-requisites etc. All good. I then tried creating an MA from the out of the box connectors. That worked. I successfully created an Active Directory Management Agent.

In the Windows Application Log I found the following from when I was trying to create the PSMA. A little more to go on from that information.

AppLog

The link in the error message provides some info https://msdn.microsoft.com/en-us/library/dd409252(VS.100).aspx but it is a generic .NET article. Having experience with MIIS/ILM/FIM/MIM I figured the SyncEngine WebServices Config file would be appropriate place for the information provided in the MSDN link to go.

The Fix

The miiserver.exe.config file located in the default installation directory C:\Program Files\Microsoft Forefront Identity Manager\2010\Synchronization Service\Bin is what you need to edit.

Near the end of the misserver.exe.config file find the <runtime> section. Insert the line <loadFromRemoteSources enabled=”true”/> as shown below.

RemoteSources

Restart the Forefront Identity Manger Server Service from the Services Control Panel and you’re back in action.

Management Agent created and back to the task at hand. Happy days.

MACreated

Follow Darren on Twitter @darrenjrobinson