Planning Site structure and Navigation in SharePoint Modern Experience Communication and Team sites

If you are planning to implement or implementing Modern team sites or Communication sites, there is change in best practices for planning and managing the Sites structure, Site Hierarchy and Navigation. This is a very common question during my presentations – how do we manage site structures, navigation and content in Modern experiences.

So, in this blog, we will look at few strategies for planning Site structure and Navigation in Modern Experience sites.

1. First and foremost, get rid of nested subsites and Site hierarchy navigation. Recently Microsoft has been pushing for Site Collections flat structure with Modern Team and Communication sites, which adds a lot of benefit for managing isolation and content. So, the new approach – Flat Site Collections and no Subsites. (There are various advantages of flat structure site collections which will be listed in another upcoming blog)

2. Secondly, to achieve a hierarchy relationship among sites such as Navigation, news items, search etc, use Hub Sites. Hub sites are the new way of connecting SharePoint site collections together. Besides, they have added advantage of aggregating information such as News and Search results from related hub associated sites. So, create a Hub site for Navigation across related sites.HubSiteAssociatedTeam

3. A best candidate for Hub sites, in my opinion, is Communication sites. Communication sites have a top navigation that can be easily extended for Hub sites. They are perfect for publishing information and showing aggregrated content. However, it also depends on if the Hub site is meant for a team and business unit or company as a whole. So, use Communication as a Hub site if targeting all company or a major group.QuickLaunchNestedCommunicationSite

4. One Navigation structure – Quick launch (Left hand) is Top Navigation for Communication sites. So no need to maintain two navigations. If you ask me, this a big win and removes a lot of confusion for end users.QuickLaunchEdit_CommSite

5. Quick launch for Modern Team and Communication Sites allows three level sub hierarchy which allows to define a nested custom hierarchy structure for Navigation which could be different from the content structure and site structure.

6. Focus on Content, not on Navigation or location of Content, through new Modern web parts such as Highlighted content, Quick links etc. which allow you to find content anywhere easily.HighlightedContent

7. Finally, few limitations of Modern Site Structure and Navigation (as of June 2018) for reference. Hopefully, this will be changed soon.

    • Permissions management still needs to be managed at each Site Collection, no nested structure there yet. Yet it is possible to use AD groups for consistent permissions set up
    • Office 365 Unified Security Groups cannot have AD or other Office 365 groups nested for Modern Team sites. But SharePoint site permissions could be managed through AD groups
    • Contextual Navigation bolding is missing in Hub sites i.e. if you click on the link to move to a child site then navigation is not automatically bolded, but this might be coming soon.
    • Navigation headers in Modern sites cannot be blank and needs to be set to a link

Conclusion:

Hence in this blog, we looked at an approach for Modern site structures, hierarchy and navigation.

Automate network share migrations to Sharepoint Online using ShareGate PowerShell

Sharegate supports PowerShell scripting which can be used to automate and schedule migrations. In this post, I am going to demonstrate an example of end to end automation to migrate network Shares to SharePoint Online. The process effectively reduces the task of executing migrations to “just flicking a switch”.

Pre-Migration

The following pre-migration activities were conducted before the actual migration:

  1. Analysis of Network Shares
  2. Discussions with stakeholders from different business units to identify content needs
  3. Pilot migrations to identify average throughput capability of migration environment
  4. Identification of acceptable data filtration criteria, and prepare Sharegate migration template files based on business requirements
  5. Derive a migration plan from above steps

Migration Automation flow

The diagram represents a high-level flow of the process:

 

The migration automation was implemented to execute the following steps:

  1. Migration team indicates that migration(s) are ready to be initiated by updating the list item(s) in the SharePoint list
  2. Updated item(s) are detected by a PowerShell script polling the SharePoint list
  3. The list item data is downloaded as a CSV file. It is one CSV file per list item. The list item status is updated to “started”, so that it would not be read again
  4. The CSV file(s) are picked up by another migration PowerShell script to initiate migration using Sharegate
  5. The required migration template is selected based on the options specified in the migration list item / csv to create a migration package
  6. The prepared migration task is queued for migration with Sharegate, and migration is executed
  7. Information mails are “queued” to be dispatched to migration team
  8. Emails are sent out to the recipients
  9. The migration reports are extracted out as CSV and stored at a network location.

Environment Setup

Software Components

The following software components were utilized for the implementing the automation:

  1. SharePoint Online Management shell
  2. SharePoint PnP PowerShell
  3. Sharegate migration Tool

Environment considerations

Master and Migration terminals hosted as Virtual machines – Each terminal is a windows 10 virtual machine. The use of virtual machines provides following advantages over using desktops:

  • VMs are generally deployed directly in datacenters, hence, near the data source.
  • Are available all the time and are not affected by power outages or manual shutdowns
  • Can be easily scaled up or down based on the project requirements
  • Benefit from having better internet connectivity and separate internet routing can be drawn up based on requirements

Single Master Terminal – A single master terminal is useful to centrally manage all other migration terminals. Using a single master terminal offers following advantages:

  • Single point of entry to migration process
  • Acts as central store for scripts, templates, aggregated reports
  • Acts as a single agent to execute non-sequential tasks such as sending out communication mails

Multiple Migration terminals – It would be optimal to initiate parallel migrations and use multiple machines (terminals) to expedite overall throughput by parallel runs in an available migration window (generally non-business hours).  Sharegate has option to use either 1 or 5 licenses at once during migration. We utilized 5 ShareGate licenses on 5 separate migration terminals.

PowerShell Remoting – Using PowerShell remoting allows opening remote PowerShell sessions to other windows machines. This will allow the migration team to control and manage migrations using just one terminal (Master Terminal) and simplify monitoring of simultaneous migration tasks. More information about PowerShell remoting can be found here.

PowerShell execution policy – The scripts running on migration terminals will be stored at a network location in Master Terminal. This will allow changing / updating scripts on the fly without copying the script over to other migration terminals. The script execution policy of the PowerShell window will need to be set as “Bypass” to allow execution of scripts stored in network location (for quick reference, the command is “Set-ExecutionPolicy -ExecutionPolicy Bypass”.

Windows Scheduled Tasks – The PowerShell scripts are scheduled as tasks through Windows Task schedulers and these tasks could be managed remotely using scripts running on the migration terminals. The scripts are stored at a network location in master terminal.

Basic task in a windows scheduler

PowerShell script file configured to run as a Task

Hardware specifications

Master terminal (Manage migrations)

  • 2 cores, 4 GB RAM, 100 GB HDD
  • Used for managing scripts execution tasks on other terminals (start, stop, disable, enable)
  • Used for centrally storing all scripts and ShareGate property mapping and migration templates
  • Used for Initiating mails (configured as basic tasks in task scheduler)
  • Used for detecting and downloading migration configuration of tasks ready to be initiated (configured as basic tasks in task scheduler)
  • Windows 10 virtual machine installed with the required software.
  • Script execution policy set as “Bypass”

Migration terminals (Execute migrations)

  • 8 cores, 16 GB RAM, 100 GB HDD
  • Used for processing migration tasks (configured as basic tasks in windows task scheduler)
  • Multiple migration terminals may be set up based on the available Sharegate licenses
  • Windows 10 Virtual machines each installed with the required software.
  • Activated Sharegate license on each of the migration terminals
  • PowerShell remoting needs to be enabled
  • Script execution policy set as “Bypass”

Migration Process

Initiate queueing of Migrations

Before migration, migration team must perform manual pre – migration tasks (if any as defined by the migration process as defined and agreed with stake holders). Some of the pre-migration tasks / checks may be:

  • Inform other teams about a possible network surge
  • Confirming if another activity is consuming bandwith (scheduled updates)
  • Inform the impacted business users about the migration – this would be generally set up as the communication plan
  • Freezing the source data store as “read-only”

A list was created on a SharePoint online site to enable users to indicate that the migration is ready to be processed. The updates in this list shall trigger the actual migration downstream. The migration plan is pre-populated in this list as a part of migration planning phase. The migration team can then update one of the fields (ReadyToMigrate in this case) to initiate the migration. Migration status is also updated back to this list by the automation process or skip a planned migration (if so desired).

The list provides as a single point of entry to initiate and monitor migrations. In other words, we are abstracting out migration processing with this list and can be an effective tool for migration and communication teams.

The list was created with the following columns:

  • Source => Network Share root path
  • Destination site => https://yourteanant.sharepoint.com/sites/<Sitename>
  • Destination Library => Destination library on the site
  • Ready to migrate => Indicates that the migration is ready to be triggered
  • Migrate all data => Indicate if all data from the source is to be migrated (default is No). Only filtered data based on the predefined options will be migrated. (more on filtered options can be found here)
  • Started => updated by automation when the migration package has been downloaded
  • Migrated => updated by automation after migration completion
  • Terminal Name => updated by automation specifying the terminal being used to migrate the task

 

Migration configuration list

 

After the migration team is ready to initiate the migration, the field “ReadyToMigrate” for the migration item in the SharePoint list is updated to “Yes”.


“Flicking the switch”

 

Script to create the migration configuration list

The script below creres the source list in sharepoint online.

Script to store credentials

The file stores the credentials and can be used subsequent scripts.

Queuing the migration tasks

A PowerShell script is executed to poll the migration configuration list in SharePoint at regular intervals to determine if a migration task is ready to be initiated. The available migration configurations are then downloaded as CSV files, one item / file and stored in a migration packages folder on the master terminal. Each CSV file maps to one migration task to be executed by a migration terminal and ensures that the same migration task is not executed by more than one terminal. It is important that this script runs on a single terminal to ensure only one migration is executed for one source item.

 

 

Execute Migration

The downloaded migration configuration CSV files are detected by migration script tasks executing on each of the migration terminals. Based on the specified source, destination and migration options the following tasks are executed:

  1. Reads the item from configuration list to retrieve updated data based on item ID
  2. Verify Source. Additionally, sends a failure mail if source is invalid or not available
  3. Revalidates if a migration is already initiated by another terminal
  4. Updates the “TerminalName” field in the SharePoint list to indicate an initiated migration
  5. Checks if the destination site is created. Creates if not already available
  6. Checks if the destination library is created. Creates if not already available
  7. Triggers an information mail informing migration start
  8. Loads the required configurations based on the required migration outcome. The migration configurations specify migration options such as cut over dates, source data filters, security and metadata. More about this can be found here.
  9. Initiates the migration task
  10. Extracts the migration report and stores as CSV
  11. Extracts the secondary migration report as CSV to derive paths of all files successfully migrated. These CSV can be read by an optional downstream process.
  12. Triggers an information mail informing migration is completed
  13. Checks for another queued migration to repeat the procedure.

The automatioin script is given below –

 


Additional Scripts

Send Mails

The script triggers emails to required recipients.

This script polls a folder:   ‘\masterterminal\c$\AutomatedMigrationData\mails\input’ to check any files to be send out as emails. The csv files sepcify subject and body to be send out as emails to recipients configured in the script. Processed CSV files are moved to final folder.

 

Manage migration tasks (scheduled tasks)

The PowerShell script utilizes PowerShell remoting to manage windows Task scheduler tasks configured on other terminals.

 


Conclusion

The migration automation process as described above helps in automating the migration project and reduces manual overhead during the migration. Since the scripts utilize pre-configured migration options / templates, the outcome is consistent with the plan. Controlling and monitoring migration tasks utilizing a SharePoint list introduces transparency in the system and abstracts the migration complexity. Business stakeholders can review migration status easily from the SharePoint list and this ensures an effective communication channel. Automated mails informing about migration status provide additional information about the migration. The migration tasks are executed in parallel across multiple migration machines which aids in a better utilization of available migration window.

 

 

SBC Direct Routing For Teams – How are you affected?

What exactly is Direct Routing for Microsoft Teams

If you’ve been following along at home, you’ll have heard about the recent announcement with Microsoft Teams now allowing you to directly connect your existing on premise Sonus (Ribbon) or Audiocodes SBC to Microsoft Teams across the internet, to allow your Teams users to make and receive calls from the cloud.

Here’s a handy diagram direct from Microsoft that shows how it works

Hold on a minute Craig, Isn’t PSTN calling already available in Teams?

Er, yes. Sort of. You see, If you were one of the lucky countries to have Microsoft Phone System (CloudPBX for you old skoolers), you can already make and receive calls directly from the Cloud from both Skype for Business and Microsoft Teams. All you need is:

  • E5 licensing (or E3 licensing with the Microsoft Phone System Add On) Per user
  • A calling plan per user

If you’ve got both of those things, you can select and assign numbers to your cloud users, and as if by magic, they can make and receive calls from the cloud! Your calls simply route via Microsoft’s PSTN calling service.


I’m not in one of those lucky PSTN cloud calling countries though.

Well that’s ok too. Previously, if you weren’t in one of the lucky countries where Microsoft has enabled PSTN calling directly from the cloud, you could do one of two things to allow you to make calls from Skype for Business:

  1. Move countries
  2. Use Cloud Connector Edition

Cloud Connector Edition (or CCE for short) is a cut down version of Skype for Business that runs on either a server or a special edition of an AudioCodes or Sonus SBC. This box lives on premise with you – although most of the time it’ll sit in your data centre. You connect your existing ISDN, SIP or PSTN services from your carrier to the SBC, and then connect the SBC across the internet to Office 365.

Doing this then enables your Skype for Business Cloud homed users to make and receive PSTN calls via your existing carrier services.

What’s also really great about this setup is that you can connect your existing PABX to the SBC too, effectively allowing you to perform a staged migration of users from your current legacy PABX to Skype for Business in the cloud, all while allowing calls between the PSTN, legacy PABX and Skype for Business.

Lastly, if you’re running CCE right now on premise, you’re 3/4 of the way towards having PSTN calling from Microsoft Teams via this infrastructure. More on this below!


What if I’m running Skype for Business on premise in Hybrid mode with Skype for Business Online?

That’s cool, and it certainly means you’re able to make calls right now from the cloud via your on premise equipment (servers, SBC and SIP trunks).

The great thing about this setup is that you’re also 3/4 the way there towards having PSTN calling from Microsoft Teams!


What if I’m building a greenfield site?

Starting from scratch is an added bonus, and means that you get to decide the direction that you want telephony (and UC as a whole!) to go in.

If you want to explore Teams:
If you’re in one of the lucky countries that has direct PSTN calling from the cloud for Skype for Business and Teams, purchase your E5 licensing and PSTN calling plans and assign the numbers to your users.

If you’re not in one of the lucky direct PSTN calling countries, go down the direct routing path. Purchase a certified SBC from Sonus or AudioCodes, get yourself a SIP trunk, Office 365 E5 licensing and away you go!

If you want to explore Skype for Business AND Teams
Lucky country – Buy those E5 licenses and PSTN calling plans!

Other countries – Consider setting up CCE with a certified Sonus or AudioCodes SBC, then create two SIP trunks – One to Skype for Business online, and the other to Microsoft Teams. You’d then route your inbound calls to either service.


I’m running Cloud Connector Edition, or Hybrid Mode right now. How do I start making calls from Teams?

As mentioned above, if you’re already up and running with CCE or Hybrid mode and your SBC is certified you can create a SIP trunk from the SBC to Microsoft Teams!

Doing so will allow you to route calls between Teams, Skype for Business Online and On Premise, Legacy PABX’s and the PSTN via your existing ISDN, SIP or PSTN carrier trunks.


This all sounds great, but I have questions. Lots of questions!

Completely understandable. This is a moving topic at present and so information is always subject to change!

Microsoft have a great blog post about the topic here: https://techcommunity.microsoft.com/t5/Microsoft-Teams-Blog/Direct-Routing-enables-new-enterprise-voice-options-in-Microsoft/ba-p/170450

You can always post your question in the comments section below, and i’ll give you a hand!

Utilizing Sharegate migration Templates for Network share migrations

Sharegate supports PowerShell based scripting which can be used to automate and schedule migrations. The purpose of this post is to demonstrate the use of pre-created migration templates to initiate migration tasks in Sharegate using PowerShell scripts. In one of my previous project, we were migrating network shares to SharePoint Online using Sharegate as the migration tool of choice.

Based on our discussions with business divisions and IT department, the following requirements were identified for most of the divisions:

  1. Office documents, PDFs, Image files will be migrated
  2. Include only documents modified after a date for e.g. January 1, 2016
  3. Permissions should be preserved
  4. There were some exceptions where the identified divisions requested all available data to be migrated.

To address the above business requirements, we created custom migration templates and these templates were used to trigger the migration tasks. This allowed us to do use pre-created configurations for each migration and eliminate of manual overhead and risk of errors. Sharegate migration templates (.sgt files) are xml based configuration documents and encapsulate the configuration options such as:

  • Copy options
  • Content type mappings
  • Allowed extensions
  • Cut-off dates

Migration Templates

I’ll present two (2) migration templates, that were prepared to cover off the business requirements:

Template 1 – All data This migration template configures Sharegate to migrate all data to the destination library. The configuration values to be considered are:

  • CopyPermissions Specifies that all permissions on the files / folders be over [configured as “true”]
  • KeepAuthorsAndTimestamps – Specifies that the file / folder metadata such as Authors, associated time stamps (created date, modified date) be copied over [configured as “true”]
  • ContentFileExtensionFilterType – specifies file extension type filter to copy data [configured as “AllExtensions”]

 

Template 2 – Filtered Data This migration template configures Sharegate to migrate all the documents that was modified / created after 1st January, 2016 and matches one of the configured extensions to the destination library. The configuration values to be considered are:

  • CopyPermissions – Specifies that all permissions on the files / folders be over [configured as “true”]
  • ContentFrom – specifies the date cutover anything modified after this date shall be copied [configured as “01/01/2016 10:00:00”]
  • KeepAuthorsAndTimestamps – Specifies that the file / folder metadata such as Authors, associated time stamps (created date, modified date) be copied over [configured as “true”]
  • ContentFileExtension – specifies that all allowed file type to be copied over [configured as “pdf; xlsx; xls; doc; pptx; docx; ppt; xlsm; rtf; vsd; pub; docm; xlsb; mpp; one; pps; dotx; dot; pot; xlk; vdx; pptm; xlt; xlam; potx; odt; xla; wk4; dic; dotm; xltm; xltx; onetoc2; vss; thmx; vsdx; xps; msg; eml; jpg; cr2; png; tif; psd; bmp; eps; ai; jpeg; gif; indd; dwg; svg; djvu; ico; wmf; dcx; emf; xpm; pdn; cam; ”]
  • ContentFileExtensionFilterType – specifies file extension type filter to copy data [configured as “LimitTo”]

 

Migration templates in Sharegate user interface

The migration templates presented as XML files before may be imported into Sharegate user interface to inspect and test. Migration options are available while configuring a migration as demonstrated in the screen shot provided below:

 

The screen shots below represent the migration options as visible in Sharegate user interface, when the templates are imported into the tool.

Copy Options Screen – Template 1

Content type mapping screen – Template 1

Filter options screen – Template 1

 

Copy Options Screen – Template 2

Content type mapping screen – Template 2

Filter options screen – Template 2

 

PowerShell scripts to initiate migrations

Migrations can be initiated from PowerShell console by running the following script

 

The coded migration templates as demonstrated above has merits in terms of eliminating the need to manually configure options for each migration using Sharegate user interface.

To extend this to a whole new level, the overall migration project can further be split into multiple migration tasks and scheduled as sequential migrations across multiple machines utilizing common migration templates and scripts stored in a network location, but that is something I’ll cover off as a part of separate blog post.

Happy Migrating…

Surface Hub: Notes from the field

I’ve been working with Surface Hub for around a year, and have ran into a few quirky things that are worth pointing out.


The display

Let’s get this one out of the way straight away. The display is beautiful. Especially when you connect up an Xbox one to It via HDMI! Perfect for kicking back during a deployment.


You broke it

Alright. I’ll hold my hand up and admit that I was the first person globally to break a surface hub (so I’m told, anyways). You see, when you first turn the hub on it’s going to want to do Windows Updates. This is highly recommended and i’d definitely advise letting the device do its thing.

This is particularly important, because if you interrupt the update process, you’ll break it. I learned this one the hard way.

Long story short, we were in the middle of deploying a surface hub and had left the device to update, but at some point all we got was a black screen. We left the device to work itself out whilst we went for an extended dinner break and came back to .. a black screen. I made the (stupid) decision to power cycle the device and when it attempted to boot back up it wouldn’t boot into Windows.

No problem, i’ll just F8 into safe mode. There’s no safe mode.

Ok, then I’ll download an image from Microsoft and reimage the Surface Hub. There’s no image available to download.

To recover the device, We had to ask Microsoft to go and grab a spare pre-imaged hard drive from the warehouse floor, and bring it to us so we could swap it out for the “broken” one. Once that was swapped over, we were back up and running!

The long story short – Don’t interrupt the Surface Hub whilst its doing updates – including if you get a black screen for many hours.

 

Skype Meetings

The surface hub is a fantastic device for hosting and joining Skype for Business meetings. The two cameras work perfectly together to show everyone in the room and around the device, and the ability to draw on and present the whiteboard is awesome.

There are a couple of things to note though:

Only todays meetings appear
The home screen on the surface hub will only display todays scheduled meetings, and on top of that will only show 8 scheduled meetings from today. Meetings scheduled for tomorrow will not appear at all.

Only the next 3 scheduled meetings appear in Skype
Within Skype itself, only the next 3 scheduled meetings will appear in the list. The list is not scrollable either.

Inviting two or more surface hubs to draw on the whiteboard has its limitations
Whilst it is true that you can invite more than one surface hub to draw on the same whiteboard during a Skype for Business call, there are some prerequisites:

  • Office 365 with cloud-based Azure Active Directory (Azure AD) for all users
  • OneDrive for Business deployed for all users who intend to collaborate
  • Currently not utilizing Office 365 Germany or Office 365 operated by 21Vianet
  • Surface Hub needs to be updated to Windows 10, version 1607 or newer
  • Port 443 needs to be open since Whiteboard makes standard https requests
  • Whiteboard.ms, wbd.ms, *.onenote.com, and your company’s SharePoint tenant domain URLs need to be whitelisted for proxies

https://docs.microsoft.com/en-us/surface-hub/whiteboard-collaboration has all the information you need to get this set up and working.


Microsoft Teams

If your organisation is currently trialling Microsoft Teams along side Skype for Business, you’ll be glad to know that at least some of the features of Teams will work on Surface Hub right now via the built in Edge browser.

Microsoft are working on a app for the Surface Hub as you read this. If you’d like to know more about what you can do right now on Surface Hub, check out my blog post on the topic.


Surface Hub 2
Lastly, there has some really exciting news on the new Surface Hub 2 from Microsoft. If you haven’t already seen the video, go and check it out here: https://www.youtube.com/watch?v=7DbslbKsQSk

I for one cannot wait to see one in person!

Surface Hub and Microsoft Teams: What can you do right now?

If you’re lucky to have access to a Microsoft Surface Hub in your organisation, you may be wondering if Microsoft Teams will work on the device.

Microsoft are working on a Teams app for the surface hub, but whilst we wait for that to become available, what features are available to us right now?

 

Signing In to Teams

First things first; If we don’t have a Teams app on the device how do we sign in and use Teams? The answer is to simply browse to http://teams.microsoft.com from the built in Edge browser on the surface hub.

Sign in with your organisation credentials, and you’re in!

 

The interface

If you’ve used the web version of Teams before, you’ll feel right at home. Everything looks and feels exactly the same on the big screen as it would on the small screen. You get easy access to the Teams activity feed via the left hand navigation, as well as chats, teams, meetings and files.

Using the chat screen, you can easily invite team mates into a chat, just as you would on your desktop. Gifs work too!

 

What does work

Surprisingly, quite a lot!

  • The activity feed, and chat work great
  • Voice and Video calls with the built in camera and mic/speakers works
  • Adding Gifs to chat works
  • Scheduling and joining Microsoft Teams meetings works

 

 

What doesn’t work

Right now, the following things don’t work in the web client on the Surface Hub:

  • Adding additional attendees to a meeting once the meeting has started
  • Presenting your desktop, or other content
  • Adding content to a chat

I imagine that a whole bunch of other things don’t work as expected too, but am yet to test every feature.

 

What about the Microsoft Teams surface hub app

It’s coming! We don’t know when, but when it does become available it’ll most certainly be available for download from the App store.

Keep those eyes peeled to Twitter to stay up to date with the latest Surface Hub and Teams announcements!

 

 

Notes From The Field – Enabling GAL Segmentation in Exchange Online

First published at https://nivleshc.wordpress.com

Introduction

A few weeks back, I was tasked with configuring Global Address List (GAL) Segmentation for one of my clients. GAL Segmentation is not a new concept, and if you were to Google it (as you would do in this day and age), you will find numerous posts on it.

However, during my research, I didn’t find any ONE article that helped me. Instead I had to rely on multiple articles/blogposts to guide me into reaching the result.

For those that are new to GAL Segmentation, this can be a daunting task. This actually is the inspiration for this blog, to provide the steps from an implementers view, so that you get the full picture about the preparation, the steps involved and the gotchas so that you feel confident about carrying out this simple yet scary change.

This blog will be focus on GAL Segmentation for an Exchange Online hybrid setup.

So what is GAL Segmentation?

I am glad you asked 😉

By default, in Exchange Online (and On-Premises Exchange environment as well), a global address list is present. This GAL contains all mail enabled objects contained in the Exchange Organisation. There would be mailboxes, contacts, rooms, etc.

This is all well and good, however, at times a company might not want everyone to see all the objects in the Exchange environment. This might be for various reasons, for instance, the company has too many employees and it won’t make sense to have a GAL a mile long. Or, the company might have different divisions, which do not require to correspond to each other. Or the company might be trying to sell off one of its divisions, and to start the process, is trying to separate the division from the rest of the company.

For this blog, we will use the last reason, as stated above. A “filter” will be applied to all users who are in division to be sold off, so that when they open their GAL, they only see objects from their own division and not everyone in the company. In similar fashion, the rest of the company will see all objects except the division that will be sold off. Users will still be able to send/receive emails with that particular division, however the GAL will not show them.

I would like to make it extremely clear that GAL Segmentation DOES NOT DELETE any mail enable objects. It just creates a filtered version of the GAL for the user.

Introducing the stars

Lets assume there was once a company called TailSpin Toys. They owned the email namespace tailspintoys.com and had their own Exchange Online tenant.

One day, the board of TailSpin Toys decided to acquire a similar company called WingTip ToysWingTip Toys had their own Exchange Online Tenant and used the email namespace wingtiptoys.com. After the acquisition, WingTip Toys email resources were merged into the TailSpin Toys Exchange Online tenant, however WingTip Toys still used their wingtiptoys.com email namespace.

After a few years, the board of TailSpin Toys decided it was time to sell of WingTip Toys. As a first step, they decided to implement GAL Segmentation between TailSpin Toys and WingTip Toys users.

Listed below is what was decided

  • TailSpin Toys users should only see email objects in their GAL corresponding to their own email namespace (any object with the primary smtp address of @tailspintoys.com). They should not be able to see any WingTip Toys email objects.
  • Only TailSpin Toys users will be able to see Public Folders in their GAL
  • WingTip Toys users should only see email objects in their GAL corresponding to their own email namespace (any object with the primary smtp address of @wingtiptoys.com). They should not be able to see any TailSpin Toys email objects.
  • The All Contacts in the GAL will be accessible to both WingTip Toys and TailSpin Toys users.

The Steps

Performing a GAL Segmentation is a very low risk change. The steps that will be carried out are as follows

  • Create new Global Address Lists, Address Lists, Offline Address Book and Address Book Policy for TailSpin Toys and WingTip Toys users.
  • Assign the respective policy to TailSpin Toys users and WingTip Toys users

The only issue is that by default, no users are assigned an Address Book Policy (ABP) in Exchange Online (ABPs are the “filter” that specifies what a user sees in the GAL).

Due to this, when we are creating the new address lists, users might see them in their GAL as well and get confused as to which one to use. If you wish to carry out this change within business hours, the simple remedy to the above issue is to provide clear communications to the users about what they could expect during the change window and what they should do (in this case use the GAL that they always use). Having said that, it is always a good practice to carry carry out changes out of business hours.

Ok, lets begin.

  • By default, the Address Lists Management role is not assigned in Exchange Online. The easiest way to assign this is to login to the Exchange Online Portal using a Global Administrator account and add this role to the Organization Management role group. This will then provide all the Address List commands to the Global Administratos.
  • Next, connect to Exchange Online using PowerShell
  • For TailSpin Toys
    • Create a default Global Address List called Default TST Global Address List
    • New-GlobalAddressList -Name "Default TST Global Address List" -RecipientFilter {((Alias -ne $null) -and (((ObjectClass -eq 'user') -or (ObjectClass -eq 'contact') -or (ObjectClass -eq 'msExchSystemMailbox') -or (ObjectClass -eq 'msExchDynamicDistributionList') -or (ObjectClass -eq 'group') -or (ObjectClass -eq 'publicFolder'))) -and (WindowsEmailAddress -like "*@tailspintoys.com") )}
    • Create the following Address Lists
      • All TST Distribution Lists
      • New-AddressList -Name "All TST Distribution Lists" -RecipientFilter {((Alias -ne $null) -and (ObjectCategory -like 'group') -and (WindowsEmailAddress -like "*@tailspintoys.com"))}
      • All TST Rooms
      • New-AddressList -Name "All TST Rooms" -RecipientFilter {((Alias -ne $null) -and (((RecipientDisplayType -eq 'ConferenceRoomMailbox') -or (RecipientDisplayType -eq 'SyncedConferenceRoomMailbox'))) -and (WindowsEmailAddress -like "*@tailspintoys.com"))}
      • All TST Users
      • New-AddressList -Name "All TST Users" -RecipientFilter {((Alias -ne $null) -and (((((((ObjectCategory -like 'person') -and (ObjectClass -eq 'user') -and (-not(Database -ne $null)) -and (-not(ServerLegacyDN -ne $null)))) -or (((ObjectCategory -like 'person') -and (ObjectClass -eq 'user') -and (((Database -ne $null) -or (ServerLegacyDN -ne $null))))))) -and (-not(RecipientTypeDetailsValue -eq 'GroupMailbox')))) -and (WindowsEmailAddress -like "*@tailspintoys.com"))}
    • Create an Offline Address Book called TST Offline Address Book (this uses the Default Global Address List that we had just created)
    • New-OfflineAddressBook -Name "TST Offline Address Book" -AddressLists "Default TST Global Address List"
    • Create an Address Book Policy called TST ABP
    • New-AddressBookPolicy -Name "TST ABP" -AddressLists "All Contacts", "All TST Distribution Lists", "All TST Users", “Public Folders” -RoomList "All TST Rooms" -OfflineAddressBook "TST Offline Address Book" -GlobalAddressList "Default TST Global Address List"
  • For WingTip Toys
    • Create a default Global Address List called Default WTT Global Address List
    • New-GlobalAddressList -Name "Default WTT Global Address List" -RecipientFilter {((Alias -ne $null) -and (((ObjectClass -eq 'user') -or (ObjectClass -eq 'contact') -or (ObjectClass -eq 'msExchSystemMailbox') -or (ObjectClass -eq 'msExchDynamicDistributionList') -or (ObjectClass -eq 'group') -or (ObjectClass -eq 'publicFolder'))) -and (WindowsEmailAddress -like "*@wingtiptoys.com") )}
    • Create the following Address Lists
      • All WTT Distribution Lists
      • New-AddressList -Name "All WTT Distribution Lists" -RecipientFilter {((Alias -ne $null) -and (ObjectCategory -like 'group') -and (WindowsEmailAddress -like "*@wingtiptoys.com"))}
      • All WTT Rooms
      • New-AddressList -Name "All WTT Rooms" -RecipientFilter {((Alias -ne $null) -and (((RecipientDisplayType -eq 'ConferenceRoomMailbox') -or (RecipientDisplayType -eq 'SyncedConferenceRoomMailbox'))) -and (WindowsEmailAddress -like "*@wingtiptoys.com"))}
      • All WTT Users
      • New-AddressList -Name "All WTT Users" -RecipientFilter {((Alias -ne $null) -and (((((((ObjectCategory -like 'person') -and (ObjectClass -eq 'user') -and (-not(Database -ne $null)) -and (-not(ServerLegacyDN -ne $null)))) -or (((ObjectCategory -like 'person') -and (ObjectClass -eq 'user') -and (((Database -ne $null) -or (ServerLegacyDN -ne $null))))))) -and (-not(RecipientTypeDetailsValue -eq 'GroupMailbox')))) -and (WindowsEmailAddress -like "*@wingtiptoys.com"))}
    • Create an Offline Address Book called WTT Offline Address Book (this uses the Default Global Address List that we had just created)
    • New-OfflineAddressBook -Name "WTT Offline Address Book" -AddressLists "Default WTT Global Address List"
    • Create an Address Book Policy called WTT ABP
    • New-AddressBookPolicy -Name "WTT ABP" -AddressLists "All Contacts", "All WTT Distribution Lists", "All WTT Users" -RoomList "All WTT Rooms" -OfflineAddressBook "WTT Offline Address Book" -GlobalAddressList "Default WTT Global Address List"
  • Once you create all the Address Lists, after a few minutes, you will be able to see them using Outlook Client or Outlook Web Access. One of the obvious things you will notice is that they are all empty! If you are wondering if the recipient filter is correct or not, you can use the following to confirm the membership
  • Get-Recipient -RecipientPreviewFilter (Get -AddressList -Identity {your address list name here}).RecipientFilter

    Aha, you might say at this stage. I will just run the Update-AddressList cmdlet. Unfortunately, this won’t work since this cmdlet is only available for On-Premises Exchange Servers. There is none for Exchange Online. Hmm. How do I update my Address Lists ? Its not too difficult. All you have to do is change some attribute of the members and they will start popping into the Address List! For a hybrid setup, this means we will have to change the setting using On-Premise Exchange Server and use Azure Active Directory Connect Server to replicate the changes to Azure Active Directory, which in turn will update Exchange Online objects, thereby updating the newly created Address Lists. Simple? Yes. Lengthly? Yes indeed

  • I normally use CustomAttribute for such occasions. Before using any CustomAttribute, ensure it is not used by anything else. You might    be able to ascertain this by checking if for all objects, that CustomAttribute currently holds any value or not. Lets assume CustomAttribute10 can be used.
    #Get all On-Premise Mailboxes
    $OnPrem_MBXs = Get-Mailbox -Resultsize unlimited
    
    #Get all Exchange Online Mailboxes
    $EXO_MBXs = Get-RemoteMailbox -Resultsize Unlimited
    
    #Get all the Distribution Groups
    $All_DL = Get-DistributionGroup -Resultsize unlimited
    
    #Update the CustomAttribute10 Value
    #Since Room mailboxes are a special type of Maibox, the following update will
    #address Room Mailboxes as well
    
    $OnPrem_MBXs | Set-Mailbox -CustomAttribute10 “GAL”
    $EXO_MBXs | Set-RemoteMailbox -CustomAttribute10 “GAL”
    
    $All_DL | Set-DistributionGroup -CustomAttribute10 “GAL”
  • Using your Azure Active Directory Connect server run a synchronization cycle so that the updates are synchronized to Azure Active Directory and subsequently to Exchange Online
  • One Gotcha here is if you have any Distribution Groups that are not synchronised from OnPremises. You will have to find these and update their settings as well. One simple way to find them is to use the property isDirSynced. Connect to Exchange Online using PowerShell and then use the following command
  • $All_NonDirsyncedDL = Get-DistributionGroup -Resultsize unlimited| ?{$_.isdirsynced -eq $FALSE}   
    
    #Now, we will update CustomAttribute10 (please check to ensure this customAttribute doesn't have any values)
     
    $All_NonDirSyncedDL | Set-DistributionGroup -CustomAttribute10 "GAL"
  • Check using Outlook Client or Outlook Address Book to see that the new Address Lists are now populated
  • Once confirmed that the new Address Lists have been populated, lets go assign the new Address Book Policies to TailSpin Toys and WingTip Toys users It can take anywhere from 30min – 1hr for the Address Book Policy to take effect
  • $allUserMbx = Get-Mailbox -RecipientTypeDetails UserMailbox -Resultsize unlimited
    
    #assign "TST ABP" Address Book Policy to TailSpin Toys users
    
    $allUserMbx | ?{($_.primarysmtpaddress -like "*@tailspintoys.com")} | Set-Mailbox -AddressBooksPolicy “TST ABP”
    
    #assign "WTT ABP" Address Book Policy to WingTip Toys users
    $allUserMbx | ?{($_.primarysmtpaddress -like "*@wingtiptoys.com")} | Set-Mailbox -AddressBooksPolicy “WTT ABP”
  • While waiting, remove the CustomAttribute10 values you had populated. Using PowerShell on On-Premises Exchange Server, run the following
  • #Get all On-Premise Mailboxes
    
    $OnPrem_MBXs = Get-Mailbox -Resultsize unlimited
    
    #Get all Exchange Online Mailboxes
    
    $EXO_MBXs = Get-RemoteMailbox -Resultsize Unlimited
    
    #Get all the Distribution Groups
    
    $All_DL = Get-DistributionGroup -Resultsize unlimited
    
    #Set the CustomAttribute10 Value to null
    
    #Since Room mailboxes are a special type of Maibox, the following update will
    
    #address Room Mailboxes as well
    
    $OnPrem_MBXs | Set-Mailbox -CustomAttribute10 $null
    
    $EXO_MBXs | Set-RemoteMailbox -CustomAttribute10 $null
    
    $All_DL | Set-DistributionGroup -CustomAttribute10 $null
  • Connect to Exchange Online using PowerShell and remove the value that was set for CustomAttribute10 for nonDirSynced Distribution Groups
  • $All_NonDirsyncedDL = Get-DistributionGroup -Resultsize unlimited| ?{$_.isdirsynced -eq $FALSE}   
    
    #Change CustomAttribute10 to $null
    
    $All_NonDirSyncedDL | Set-DistributionGroup -CustomAttribute10 $null

     

    Thats it folks! Your GAL Segmentation is now complete! Users from TailSpin Toys will only see TailSpin Toys mail enabled objects and WingTip Toys users will only see WingTip Toys mail enabled objects

A few words of wisdom

In the above steps, I would advise that once the new Address Lists have been populated

  • apply the Address Book Policy to a few test mailboxes
  • wait between 30min – 1 hour, then confirm that the Address Book Policy has been successfully applied to the test mailboxes and has the desired result
  • once you have confirmed that the test mailboxes had the desired result for ABP, then and ONLY then continue to the rest of the mailboxes

This will give you confidence that the change will be successful. Also, if you find that there are issues, the rollback is not too difficult and time consuming.

Another thing to note is that when users have their Outlook client configured to use  cached mode, they might notice that their new GAL is not fully populated. This is because their Outlook client uses the Offline Address Book to show the GAL and at that time, the Offline Address Book would not have regenerated to include all the new members. Unfortunately in Exchange Online, the Offline Address Book cannot be regenerated on-demand and we have to wait for the the Exchange Online servers to do this for us. I have noticed the regeneration happens twice in 24 hours, around 4am and 4pm AEST (your times might vary). So if users are complaining that their Outlook Client GAL doesn’t show all the users, confirm using Outlook Web Access that the members are there (or you can run Outlook in non-cached mode) and then advise the users that the issue will be resolved when the Offline Address Book gets re-generated (in approximately 12 hours). Also, once the Offline Address Book has regenerated, it is best for users to manually download the latest Offline Address Book, otherwise Outlook client will download it at a random time in the next 24 hours.

The next gotcha is around which Address Lists are available in Offline mode (refer to the screenshot below)

GAL01

When in Offline mode, the only list available is Offline Global Address List . This is the one that is pointed to by the  green arrow. Note that the red arrow is pointing to Offline Global Address List as well however this is an “Address List” that has been named Offline Global Address List by Microsoft to confuse people! To repeat, the Offline Global Address List pointed to by the green arrow is available in Offline mode however the one pointed to by red is not!

In our case, the Offline Global Address List is named Default TST Global Address List and Default WTT Global Address List).

If you try to access any others in the drop down list when in Offline mode, you will get the following error

AddressListError

This has always been the case, unfortunately hardly anyone tries to access all the Address Lists in Offline mode. However, after GAL Segmentation, if users receive the above error, it is very easy to blame the GAL Segmentation implementation 😦 Rest assured, this is not the case and this “feature” has always been present.

Lastly, the user on-boarding steps will have to be modified to ensure that when their mailbox is created, the appropriate Address Book Policy is applied. This will ensure they only see the address lists that they are supposed to (on the flip side, if no address book policy is applied, they will see all address lists, which will cause a lot of confusion!)

With these words, I will now stop. I hope this blog comes in handy to anyone trying to implement GAL Segmentation.

If you have any more gotchas or things you can think of regarding GAL Segmentation, please leave them in the comments below.

Till the next time, Enjoy 😉

 

 

 

 

 

 

Automation and Creation of Office 365 groups using Flow, Microsoft Graph and Azure Function – Part 2

In the Part 1 blog here, we discussed an approach for the Group creation process and important considerations for provisioning groups. In this blog, we will look at getting a Graph App ID and App secret for invoking the graph service and then implementation of the group provisioning process.

MS Graph App Set up

Before we start creating groups we will need to set up a Graph App that will be used to create the group in the Office 365 tenancy. The details are in this blog here on how to create a Microsoft Graph app.

Regarding the permissions, below are the settings that are necessary to allow creating groups through the graph service.

GroupApp_Rights

Creating a Group

As discussed in Part 1 here, below are the broad level steps for automating group creation using a SharePoint inventory list, Microsoft Flow and Azure Function

1. Create a SharePoint list, with the metadata necessary for Group and SharePoint assets provisioning

We can use a SharePoint list to act as a trigger to create groups with the custom metadata necessary for provisioning the groups such as Owners and metadata necessary for creating site assets for SharePoint sites. As a best practice, I recommend you create multiple master lists to manage the details separately if there are too many to manage. In our case, we have created three separate lists for managing the Group details.

1. Group details and metadata
2. Owners and Team Members List
3. Site Assets configuration list

2. Create a Microsoft flow. The flow will validate a new or existing group and pick the unique Group Alias from the list which will allow us to find the group if it exists.

The flow will act as a trigger to start the provisioning process and call the Azure function passing the appropriate metadata as shown below. The flow also allows error handling scenarios as described in the Part 1 blog here

Note: The GroupAlias is the unique name of the Group and is not necessarily the SharePoint URL. For example, in the case where a group was created and subsequently deleted, the unique alias could be used again but the Site URL will be different (unless cleared from the SharePoint recycle bin).

Group_FlowAzureFunctionCall

3. Create the Group in an Azure Function using SharePoint Online CSOM

In order to create the group, we will need to authenticate to the Graph service using the Graph App created earlier. For authenticating the app through Azure AD, please install the NuGet Package for Microsoft.IdentityModel.Clients.ActiveDirectory.

After authenticating, we will create the group using the UnifiedGroup Utility provided through the SharePoint Online CSOM.

Below is a quick snapshot of the code. Note the inclusion of Graph module of the OfficeDevPnP class.

Note: One important bit to note is that, in the above code owners and members email array is the same. If the owners and members email array differ, then the group provisioning delays significantly. Also, it is important to keep the other parameters same as during creation in the below method because it might reset the other properties to default otherwise. For eg. if isPrivate is not set, then the group becomes public.

4. After the group is created, we can fetch the details of the group as below.

5. The group provisioning generally takes about 2-3 mins to provision. However, if there are multiple hits, then one request might override the second request causing the group creation to fail. In such cases, we can sequence the Azure Functions to run by modifying the host.json file. A quick blog covering this can be found here.

Provisioning SharePoint Assets in Azure Function after Group Creation

1. For provisioning of the SharePoint assets, we might have to wait for the Office 365 AD sync to finish granting access to the Admin account.

Sometimes, the AD sync process takes much longer, so to grant direct access to the SharePoint Site Collection using tenant admin, we could use the below code. Recommendation: Only proceed with the below code approach if the access fails for more than few mins.

Note: As a best practice, I would recommend using a Service Account when working on the SharePoint Site. We could also use an App as suggested in the Site Scripting blog here.

2. Once you have access, you can use the normal SharePoint CSOM to do the activities that are pertaining to SharePoint asset provisioning such as Libraries, Site Pages content, Lists, etc.

3. After you’re done, you can return the success from the Azure function as below.

Note: Use HttpStatusCode.Accepted instead of HttpStatusCode.Error in case there is error handling in the Flow or else Flow will trigger another instance of the flow when the Azure Function fails

Conclusion:

Above we saw how we can have a SharePoint Inventory list and create groups using Flow and Azure Functions. For a quick reference, below are the links to the other related blogs.

Part 1 – Automation and Creation of Office 365 groups approach

How to create a Microsoft Graph App

Sequencing calls in Azure Functions

Latest updates to Modern Libraries experience in SharePoint Communication sites (Apr 2018)

Modern Libraries in Communication Sites have got some welcome facelift during the last few months (Apr 2018) and there have been many great changes. I am going to list of few of these updates here.

Note: Some of these updates might be limited to Targeted release (or First release) versions only. In case these changes are not available then they might be not in Standard release ( or GA release) yet.

1. Full page view of SharePoint libraries

The SharePoint libraries now have a full page view, which provides it to use the full home page layout of Communication sites. It looks great 🙂

ModernLibExperienceSitePages

2. Custom Metadata support for Site Pages.

Now it is possible for newly created Communication sites (after Mar 2018) to have custom metadata updates with Site Pages content type.

Modern_Site_Pages_with_custom_content_type

Few catches in this scenario are:

1. It is still not possible to create a page by selecting a Child Site Page content type unless the child content type is set to default. When a page is created, it is set to default content type of the Site Pages library

2. Any communication sites, created prior to March 2018 mayn’t get this update. For associating metadata to site pages prior to Mar 2018, please check this blog for a custom approach to associate custom metadata to Site Pages. This will require custom code build for the same.

3. Support for more columns types through Modern UI Panel

Now we can create columns of additional metadata types such as Date, Choice and Picture with Modern Libraries, so don’t have to go to classic experience which is great from a UX and usability prespective.

ModernLibExperience_O3651

4. New command bar on Modern Libs

The Library command bar now provides a seemless experience of search and command items at the same level. Though small this is a great change because it would drive user to search content prior to creating new one.

NewCommandBar_SitePages

Conclusion

The above are some great updates for SharePoint modern libraries.

Set up a Microsoft Graph App for Office 365 and SharePoint Online management to use in Azure Functions, Flow, .Net solutions and much more

Microsoft Graph API can be used to connect and manage the Office 365 SaaS platforms such as SharePoint Online, Office 365 Groups, One Drive, OneNote, Azure AD, Teams (in beta) and much more.

A Graph app is an Azure AD app that has privileges (with provided permissions) to authenticate and then execute operations when using PowerShell, Azure Functions, Flow, Office Online CSOM, SharePoint Online and many other tools.

It is quite easy to set up a graph app, below is a brief preview of the process.

1. Go to the following link in your tenancy – https://apps.dev.microsoft.com/ and create an App. Below a brief screenshot of the App registration page.

GraphAppRegistrationScreen1

  1.  Then, first create a password and make sure to copy the password because it will be shown that time only. Also copy the App ID later for any future use.

GraphAppRegistrationScreen2

3. Then select the platforms that will be used to call the Graph app. For web calls use Web and for PowerShell scripts use Native as platform. You can leave the fields in the apps blank unless there is a specific endpoint that you would like to refer to. More information is provided at this link – https://developer.microsoft.com/en-us/graph/docs/concepts/auth_register_app_v2

Note: Turn off “Allow Implicit Flow” for web calls

4. Add the owners that could manage the App in Azure AD.
5. Next, select the proper application permissions that the App will need to run the actions. These settings are very important for your app to do the right calls, so try to set the appropriate settings. In some cases, it might be necessary to trial various app permission levels till you get it correct.

Note: For admin programs or scripts, it will be necessary to get Admin consent to the url below

https://login.microsoftonline.com//adminconsent?client_id=[clientid]&state=[something]

GraphAppRegistrationScreen7

6. Leave the other fields as is and create the App. We can turn off the ‘Live SDK support’ if not needed

GraphAppRegistrationScreen6

7. The App will show up in the Apps home page once created

GraphAppMyApplications

After the Graph app is created, we can use to perform various operations on Office 365 platforms. More details of the various operations are detailed here – https://developer.microsoft.com/en-us/graph/docs/concepts/overview.

There is also the beta release (https://graph.microsoft.com/beta/) which has more features upcoming in Graph Api.

Conclusion:

In the above blog, how we can create an Graph App that will allow us to connect to Graph Api and do operations with it.