Starting Microsoft Teams and Skype for Business IM sessions from a link

An interesting question was raised by a colleague recently:

Craig, How do you start an IM session from a link on a web page?

We all know that you can create a mailto: link that users can click to open their Outlook (or other mail) client and automagically pre-populate the to: field with your email address, but did you know you can do the same thing with Skype for Business?

Skype for Business

To create a clickable link that automatically opens a Skype for Business chat window to you, or anyone else for that matter, use the following format:

<a href=”sip:craig.chiffers@contoso.com“>Click here to IM me via Skype for Business</a>

Here’s how it looks: Click here to IM me via Skype for Business

The text in purple is the SIP address where you want to send the IM. The orange text is what will appear on the web page or within your email signature as the clickable link. You can even get fancy and use a picture or Skype logo if you wish! To do that, simply use the following:

<a href=”sip:craig.chiffers@contoso.com“><img src=”skype-logo.png“> Click here to IM me via Skype for Business</a>

This time, the green text points to the location of your Skype logo, or any other icon that you wish to use.

Here’s how that looks:


Microsoft Teams

Right now, Microsoft Teams doesn’t support the ability to directly start a chat session with a user from a link. I imagine that if/when this is introduced, it will look something like the following:

<a href=”msteams:craig.chiffers@contoso.com”>Click here to IM me via Microsoft Teams</a>

If you’d like to see this feature within Teams, vote for it over on UserVoice!


A Few Things To Note

For the above to work, the user will need to ensure they have Skype for Business installed. If they don’t the links won’t work. Federation will also need to be enabled if the user is outside your organisation.

How to quickly copy Azure Functions between Azure Tenants and implement ‘Run From Zip’

As mentioned in this post yesterday I needed to copy a bunch of Azure WebApps and Functions from one Tenant to another. With anything cloud based, things move fast. Some of the methods I found were too onerous and more complex than they needed to be. There is of course the Backup option as well for Azure Functions. This does require a storage account associated with the Function App Plan. My Functions didn’t have the need for storage and the plan tier they were on meant that wasn’t a prerequisite. I didn’t have the desire to add a storage account to backup to then migrate.

Overview

In this post I show my method to quickly copy Azure Functions from one Azure Tenant to another. My approach is;

  • In the Source Tenant from the Azure Functions App
    • Using Kudu take a backup of the wwwroot folder (that will contain one or more Functions)
  • In the Target Tenant
    • Create an Azure Function App
    • Using Kudu locate the wwwroot archive in the new Azure Function App
    • Configure Azure Function Run From Zip

Backing up the Azure Functions in the Source Tenant

Using the Azure Portal in the Source Tenant go to your Function App => Application Settings and select Advanced Tools. Select Debug Console – Powershell and navigate to the Site Folder. Next to wwwroot select the download icon to obtain an archive of your functions.

Download WWWRoot Folder 2.PNG

Copying the Azure Functions to the Target Tenant

In the Target Tenant first create a New Azure Function App. I did this as I wanted to change the naming, the plan and a few other configuration items. Then using the Azure Portal go to your new Function App, Application Settings and select Advanced Tools.

Function Advanced Tools

Create a folder under D:\home\data named SitePackages.

Create Site Packages Folder

Drag and drop your wwwroot.zip file into the SitePackages Folder.

Drag Drop wwwroot

In the same folder select the + icon to create a file named siteversion.txt

Site Packages

Inside the file give the name of your archive file e.g.  wwwroot.zip Select Save.

Siteversion.txt.png

Back in your new Function App select Application Settings

Application Settings

Under Application Settings add a new setting for Website_Use_Zip with a setting value of ‘1’.

Website Use Zip.PNG

Refresh your Function App and you’ll notice it is now Read Only as it is running from Zip. All the Functions that were in the Zip are displayed.

Functions Migrated.PNG

Summary

This is a quick and easy method to get your functions copied from one Tenant to another. Keep in mind if your functions are using Application Settings, KeyVaults, Managed Service Identity type options you’ll need to add those settings, certificates, credentials in the target environment.

How to quickly copy an Azure Web App between Azure Tenants using ‘Zip Push Deploy’

In the last couple of weeks I’ve had to copy a bunch of Azure WebApps and Functions from one Azure Tenant to another. I hadn’t had to do this for a while and went looking for the quickest and easiest way to accomplish it. As with anything cloud based, things move fast. Some of the methods I found were too onerous and more complex than they needed to be. There is of course the Backup option as well. However for WebApps that is only available if you are on a Standard or above tier Plan. Mine weren’t and I didn’t have the desire to uplift to get that feature.

Overview

In this post I show my method to quickly copy an Azure WebApp from one Azure Tenant to another. I cover copying Azure Functions in another post. My approach is;

  • In the Source Tenant from the WebApp
    • Download the Automation Scripts for the WebApp
    • Using Kudu take a backup of the wwwroot folder
  • In the Target Tenant
    • Create a new Resource from a Template
    • Import the Deployment Automation Scripts from above
    • Modify for any changes, Resource Group, Location etc
    • Use Zip Push Deploy to upload the wwwroot archive and deploy it

Backing up the WebApp in the Source Tenant

Open your WebApp in the Azure Portal. Select Automation Script

WebApp Deployment Script

Download the Automation Script

Save Deployment Script

Select Advanced Tools

Kudu Adv Tools

Select the Site Folder then on the right menu of wwwroot select the download icon and save the backup of the WebApp.

Download WWWRoot Folder 3.png

Expand the Deployment Script archive file from the first step above. The contents will look like those below.

Expand the Deploy Script Archive.PNG

Deploy the WebApp to another Tenant

In the Azure Portal select Create a Resource from the top of the menu list on the left hand side. Type Template in the search box and select Template Deployment then select Create. Select Build your own template in the editor. Select Load File and select the parameters.json file. Then select Load File again and select the template.json file. Select Save.

Load Parameters then Template JSON Files

Make any changes to naming, and provide an existing or new Resource Group for the WebApp. Select Purchase.

New Template Deployment - Change Parameters

The WebApp will be created. Once completed select it from the Resource Group you specified and select Advanced Tools. From the Tools menu select Zip Push Deploy.

Tools Zip Push Deploy

Drag and drop the Zip file with the archive of the wwwroot folder you created earlier.

Drop WebApp ZipFile Export via Kudu

The zip will be processed and the WebApp deployed.

Deployed WebApp

Selecting the App in the new Tenant we can see it is deployed and running.

App Running.PNG

Hitting the App URL we can see that is being served.

Deployed App.PNG

This WebApp is the Microsoft Identity Manager User Object Report that I detailed in this post here.

Summary

In less that 10 minutes the WebApp is copied. No modifying JSON files, no long command lines, no FTP clients. Pretty simple. In the next post I’ll detail how I copied Azure Functions using a similar process.

Keep in mind if your WebApp is using Application Settings, KeyVaults, Managed Service Identity type options you’ll need to add those settings, certificates/credentials in the target environment.

Demystifying Managed Service Identities on Azure

Managed service identities (MSIs) are a great feature of Azure that are being gradually enabled on a number of different resource types. But when I’m talking to developers, operations engineers, and other Azure customers, I often find that there is some confusion and uncertainty about what they do. In this post I will explain what MSIs are and are not, where they make sense to use, and give some general advice on how to work with them.

What Do Managed Service Identities Do?

A managed service identity allows an Azure resource to identify itself to Azure Active Directory without needing to present any explicit credentials. Let’s explain that a little more.

In many situations, you may have Azure resources that need to securely communicate with other resources. For example, you may have an application running on Azure App Service that needs to retrieve some secrets from a Key Vault. Before MSIs existed, you would need to create an identity for the application in Azure AD, set up credentials for that application (also known as creating a service principal), configure the application to know these credentials, and then communicate with Azure AD to exchange the credentials for a short-lived token that Key Vault will accept. This requires quite a lot of upfront setup, and can be difficult to achieve within a fully automated deployment pipeline. Additionally, to maintain a high level of security, the credentials should be changed (rotated) regularly, and this requires even more manual effort.

With an MSI, in contrast, the App Service automatically gets its own identity in Azure AD, and there is a built-in way that the app can use its identity to retrieve a token. We don’t need to maintain any AD applications, create any credentials, or handle the rotation of these credentials ourselves. Azure takes care of it for us.

It can do this because Azure can identify the resource – it already knows where a given App Service or virtual machine ‘lives’ inside the Azure environment, so it can use this information to allow the application to identify itself to Azure AD without the need for exchanging credentials.

What Do Managed Service Identities Not Do?

Inbound requests: One of the biggest points of confusion about MSIs is whether they are used for inbound requests to the resource or for outbound requests from the resource. MSIs are for the latter – when a resource needs to make an outbound request, it can identify itself with an MSI and pass its identity along to the resource it’s requesting access to.

MSIs pair nicely with other features of Azure resources that allow for Azure AD tokens to be used for their own inbound requests. For example, Azure Key Vault accepts requests with an Azure AD token attached, and it evaluates which parts of Key Vault can be accessed based on the identity of the caller. An MSI can be used in conjunction with this feature to allow an Azure resource to directly access a Key Vault-managed secret.

Authorization: Another important point is that MSIs are only directly involved in authentication, and not in authorization. In other words, an MSI allows Azure AD to determine what the resource or application is, but that by itself says nothing about what the resource can do. For some Azure resources this is Azure’s own Identity and Access Management system (IAM). Key Vault is one exception – it maintains its own access control system, and is managed outside of Azure’s IAM. For non-Azure resources, we could communicate with any authorisation system that understands Azure AD tokens; an MSI will then just be another way of getting a valid token that an authorisation system can accept.

Another important point to be aware of is that the target resource doesn’t need to run within the same Azure subscription, or even within Azure at all. Any service that understands Azure Active Directory tokens should work with tokens for MSIs.

How to Use MSIs

Now that we know what MSIs can do, let’s have a look at how to use them. Generally there will be three main parts to working with an MSI: enabling the MSI; granting it rights to a target resource; and using it.

  1. Enabling an MSI on a resource. Before a resource can identify itself to Azure AD,it needs to be configured to expose an MSI. The way that you do this will depend on the specific resource type you’re enabling the MSI on. In App Services, an MSI can be enabled through the Azure Portal, through an ARM template, or through the Azure CLI, as documented here. For virtual machines, an MSI can be enabled through the Azure Portal or through an ARM template. Other MSI-enabled services have their own ways of doing this.

  2. Granting rights to the target resource. Once the resource has an MSI enabled, we can grant it rights to do something. The way that we do this is different depending on the type of target resource. For example, Key Vault requires that you configure its Access Policies, while to use the Event Hubs or the Azure Resource Manager APIs you need to use Azure’s IAM system. Other target resource types will have their own way of handling access control.

  3. Using the MSI to issue tokens. Finally, now that the resource’s MSI is enabled and has been granted rights to a target resource, it can be used to actually issue tokens so that a target resource request can be issued. Once again, the approach will be different depending on the resource type. For App Services, there is an HTTP endpoint within the App Service’s private environment that can be used to get a token, and there is also a .NET library that will handle the API calls if you’re using a supported platform. For virtual machines, there is also an HTTP endpoint that can similarly be used to obtain a token. Of course, you don’t need to specify any credentials when you call these endpoints – they’re only available within that App Service or virtual machine, and Azure handles all of the credentials for you.

Finding an MSI’s Details and Listing MSIs

There may be situations where we need to find our MSI’s details, such as the principal ID used to represent the application in Azure AD. For example, we may need to manually configure an external service to authorise our application to access it. As of April 2018, the Azure Portal shows MSIs when adding role assignments, but the Azure AD blade doesn’t seem to provide any way to view a list of MSIs. They are effectively hidden from the list of Azure AD applications. However, there are a couple of other ways we can find an MSI.

If we want to find a specific resource’s MSI details then we can go to the Azure Resource Explorer and find our resource. The JSON details for the resource will generally include an identity property, which in turn includes a principalId:

Screenshot 1

That principalId is the client ID of the service principal, and can be used for role assignments.

Another way to find and list MSIs is to use the Azure AD PowerShell cmdlets. The Get-AzureRmADServicePrincipal cmdlet will return back a complete list of service principals in your Azure AD directory, including any MSIs. MSIs have service principal names starting with https://identity.azure.net, and the ApplicationId is the client ID of the service principal:

Screenshot 2

Now that we’ve seen how to work with an MSI, let’s look at which Azure resources actually support creating and using them.

Resource Types with MSI and AAD Support

As of April 2018, there are only a small number of Azure services with support for creating MSIs, and of these, currently all of them are in preview. Additionally, while it’s not yet listed on that page, Azure API Management also supports MSIs – this is primarily for handling Key Vault integration for SSL certificates.

One important note is that for App Services, MSIs are currently incompatible with deployment slots – only the production slot gets assigned an MSI. Hopefully this will be resolved before MSIs become fully available and supported.

As I mentioned above, MSIs are really just a feature that allows a resource to assume an identity that Azure AD will accept. However, in order to actually use MSIs within Azure, it’s also helpful to look at which resource types support receiving requests with Azure AD authentication, and therefore support receiving MSIs on incoming requests. Microsoft maintain a list of these resource types here.

Example Scenarios

Now that we understand what MSIs are and how they can be used with AAD-enabled services, let’s look at a few example real-world scenarios where they can be used.

Virtual Machines and Key Vault

Azure Key Vault is a secure data store for secrets, keys, and certificates. Key Vault requires that every request is authenticated with Azure AD. As an example of how this might be used with an MSI, imagine we have an application running on a virtual machine that needs to retrieve a database connection string from Key Vault. Once the VM is configured with an MSI and the MSI is granted Key Vault access rights, the application can request a token and can then get the connection string without needing to maintain any credentials to access Key Vault.

API Management and Key Vault

Another great example of an MSI being used with Key Vault is Azure API Management. API Management creates a public domain name for the API gateway, to which we can assign a custom domain name and SSL certificate. We can store the SSL certificate inside Key Vault, and then give Azure API Management an MSI and access to that Key Vault secret. Once it has this, API Management can automatically retrieve the SSL certificate for the custom domain name straight from Key Vault, simplifying the certificate installation process and improving security by ensuring that the certificate is not directly passed around.

Azure Functions and Azure Resource Manager

Azure Resource Manager (ARM) is the deployment and resource management system used by Azure. ARM itself supports AAD authentication. Imagine we have an Azure Function that needs to scan our Azure subscription to find resources that have recently been created. In order to do this, the function needs to log into ARM and get a list of resources. Our Azure Functions app can expose an MSI, and so once that MSI has been granted reader rights on the resource group, the function can get a token to make ARM requests and get the list without needing to maintain any credentials.

App Services and Event Hubs/Service Bus

Event Hubs is a managed event stream. Communication to both publish onto, and subscribe to events from, the stream can be secured using Azure AD. An example scenario where MSIs would help here is when an application running on Azure App Service needs to publish events to an Event Hub. Once the App Service has been configured with an MSI, and Event Hubs has been configured to grant that MSI publishing permissions, the application can retrieve an Azure AD token and use it to post messages without having to maintain keys.

Service Bus provides a number of features related to messaging and queuing, including queues and topics (similar to queues but with multiple subscribers). As with Event Hubs, an application could use its MSI to post messages to a queue or to read messages from a topic subscription, without having to maintain keys.

App Services and Azure SQL

Azure SQL is a managed relational database, and it supports Azure AD authentication for incoming connections. A database can be configured to allow Azure AD users and applications to read or write specific types of data, to execute stored procedures, and to manage the database itself. When coupled with an App Service with an MSI, Azure SQL’s AAD support is very powerful – it reduces the need to provision and manage database credentials, and ensures that only a given application can log into a database with a given user account. Tomas Restrepo has written a great blog post explaining how to use Azure SQL with App Services and MSIs.

Summary

In this post we’ve looked into the details of managed service identities (MSIs) in Azure. MSIs provide some great security and management benefits for applications and systems hosted on Azure, and enable high levels of automation in our deployments. While they aren’t particularly complicated to understand, there are a few subtleties to be aware of. As long as you understand that MSIs are for authentication of a resource making an outbound request, and that authorisation is a separate thing that needs to be managed independently, you will be able to take advantage of MSIs with the services that already support them, as well as the services that may soon get MSI and AAD support.

Xamarin Forms – Platform Specifics (iOS) : Blur Effect

As a Xamarin mobile developer ever wonder, why we need write some much of code in PCL and iOS projects to do simple Native feature, some of which are usually one-liners code natively.

Xamarin has now introduced a nice nifty feature that helps us to write code in Xamarin Forms in the form of Platform Specifics.

In short, Platform Specifics helps us to consume the features or functionalities that are only available on iOS, without needing to implement custom renderers or effects in the required platform project.

One of the best example to understand this features is Blur effect. Platform specifics are baked into the Xamarin and it is ready to use.

Below are the steps to test this feature

Create a Xamarin Forms projectScreen Shot 2018-04-12 at 22.28.54

Namespaces: It is important to understand Xaml namespaces to get to know about Xamarin specifics. Below is the required namespace on each page.

xmlns:ios=clr-namespace:Xamarin.Forms.PlatformConfiguration.iOSSpecific;assembly=Xamarin.Forms.Core 

Blur Effect

Below the way we can define Blur effect on a boxview, this effect can be implemented on any visual elements in Xamarin Forms.

<BoxView x:Name=boxView ios:VisualElement.BlurEffect=Dark HeightRequest=50 WidthRequest=50 />

Blur effect have enumeration options to be set to it.

  1. Dark – applies a Dark blur effect
  2. Light – applies the light blur effect
  3. ExtraLight – applies an extra light blur effect
  4. None – applies no blur effect

Below is sample code for various blur effects we can  notice

<StackLayout>
    <Image Source=Aus.png HeightRequest=50 WidthRequest=50 />
    <BoxView x:Name=boxView ios:VisualElement.BlurEffect=Dark HeightRequest=50 WidthRequest=50 />
</StackLayout>
<StackLayout>
    <Image Source=Aus.png HeightRequest=50 WidthRequest=50 />
    <BoxView x:Name=boxView1 ios:VisualElement.BlurEffect=Light HeightRequest=50 WidthRequest=50 />
</StackLayout>
<StackLayout>
    <Image Source=Aus.png HeightRequest=50 WidthRequest=50 />
    <BoxView x:Name=boxView2 ios:VisualElement.BlurEffect=ExtraLight HeightRequest=50 WidthRequest=50 />
</StackLayout>

Below is the sample output on iOS

Screen Shot 2018-04-12 at 22.50.55

Here is the sample available in Github

https://github.com/divikiran/PlatformSpecficBlurEffect.git

 

 

 

 

Deploy active/active FortiGate NGFW in Azure

I recently was tasked with deploying two Fortinet FortiGate firewalls in Azure in a highly available active/active model. I quickly discovered that there is currently only two deployment types available in the Azure marketplace, a single VM deployment and a high availability deployment (which is an active/passive model and wasn’t what I was after).

FG NGFW Marketplace Options

I did some digging around on the Fortinet support sites and discovered that to you can achieve an active/active model in Azure using dual load balancers (a public and internal Azure load balancer) as indicated in this Fortinet document: https://www.fortinet.com/content/dam/fortinet/assets/deployment-guides/dg-fortigate-high-availability-azure.pdf.

Deployment

To achieve an active/active model you must deploy two separate FortiGate’s using the single VM deployment option and then deploy the Azure load balancers separately.

I will not be going through how to deploy the FortiGate’s and required VNets, subnets, route tables, etc. as that information can be found here on Fortinet’s support site: http://cookbook.fortinet.com/deploying-fortigate-azure/.

NOTE: When deploying each FortiGate ensure they are deployed into different frontend and backend subnets, otherwise the route tables will end up routing all traffic to one FortiGate.

Once you have two FortiGate’s, a public load balancer and an internal load balancer deployed in Azure you are ready to configure the FortiGate’s.

Configuration

NOTE: Before proceeding ensure you have configured static routes for all your Azure subnets on each FortiGate otherwise the FortiGate’s will not be able to route Azure traffic correctly.

Outbound traffic

To direct all internet traffic from Azure via the FortiGate’s will require some configuration on the Azure internal load balancer and a user defined route.

  1. Create a load balance rule with:
    • Port: 443
    • Backend Port: 443
    • Backend Pool:
      1. FortiGate #1
      2. FortiGate #2
    • Health probe: Health probe port (e.g. port 22)
    • Session Persistence: Client IP
    • Floating IP: Enabled
  2. Repeat step 1 for port 80 and any other ports you require
  3. Create an Azure route table with a default route to the Azure internal load balancer IP address
  4. Assign the route table to the required Azure subnets

IMPORTANT: In order for the load balance rules to work you must add a static route on each FortiGate for IP address: 168.63.129.16. This is required for the Azure health probe to communicate with the FortiGate’s and perform health checks.

FG Azure Health Probe Cfg

Once complete the outbound internet traffic flow will be as follows:

FG Internet Traffic Flow

Inbound traffic

To publish something like a web server to the internet using the FortiGate’s will require some configuration on the Azure public load balancer.

Let’s say I have a web server that resides on my Azure DMZ subnet that hosts a simple website on HTTPS/443. For this example the web server has IP address: 172.1.2.3.

  1. Add an additional public IP address to the Azure public load balancer (for this example let’s say the public IP address is: 40.1.2.3)
  2. Create a load balance rule with:
    • Frontend IP address: 40.1.2.3
    • Port: 443
    • Backend Port: 443
    • Backend Pool:
      1. FortiGate #1
      2. FortiGate #2
    • Session Persistence: Client IP
  3. On each FortiGate create a VIP address with:
    • External IP Address: 40.1.2.3
    • Mapped IP Address: 172.1.2.3
    • Port Forwarding: Enabled
    • External Port: 443
    • Mapped Port: 443

FG WebServer VIP Cfg

You can now create a policy on each FortiGate to allow HTTPS to the VIP you just created, HTTPS traffic will then be allowed to your web server.

For details on how to create policies/VIPs on FortiGate’s refer to the Fortinet support website: http://cookbook.fortinet.com.

Once complete the traffic flow to the web server will be as follows:

FG Web Traffic Flow

Adding a Display to the Teenager Notification Service Azure IoT Device

Overview

A couple of weeks back I wrote this post that detailed Building a Teenager Notification Service using Azure IoT an Azure Function, Microsoft Flow, Mongoose OS and a Micro Controller. 

Over the Easter break I enhanced it with the inclusion of a display. I was rummaging around in a box of parts when I found a few LCD displays I’d purchased on speculation some time ago. They are SSD1306 LCD driven units that can be found on Amazon here. A quick upgrade later and …

… scrolling text to go with rotating lights. The addition of the display requires the following changes to the previous project which are detailed in this post;

  • inclusion of the SSD1306 library
  • configure your micro controller for the display
  • a few changes in the Mongoose OS Init.JS file to have the appropriate text displayed for the notification
  • change to the Notifier Base case to integrate the display
    • it is available in the Thingiverse Project for this thing here and named NodeMCU with Display Window.stl

Incorporating the SSD1306 Library

Before starting, with your micro controller connected and using the MOS UI, take a copy of your Init.js configuration file by selecting Device Files, then Init.js and copying the content to somewhere safe. Also the Device Config by choosing Device Config, Expert View and Save Configuration.

From the MOS UI select Projects, select the AzureIoT-Neopixel-js project then from the drop down menu select mos.yml.

Add the line  – origin: https://github.com/mongoose-os-libs/arduino-adafruit-ssd1306 then select the Spanner icon to Rebuild the App. Once completed select the Flash icon to update your micro controller.

Include SSD1306 Library.PNG

Once written to your micro controller check your Init.js and copy back your backup. Check your Configuration and make sure your MQTT settings are still present. Copy your previous config back if required.

Configure your Micro Controller for the SSD1306 Display

We need to tell your micro controller which GPIO Pins we have attached the display too. I actually also moved the GPIO Pin I attached for the Neopixel as part of this. The configuration is;

  • Neopixel connected to GPIO 12
  • SSD1306 SDA connected to GPIO 4
  • SSD1306 SCL connected to GPIO 5

In the Expert Device Config mode update the I2C section as shown below. Save the configuration.

 "i2c": {
 "enable": true,
 "freq": 100000,
 "debug": false,
 "sda_gpio": 4,
 "scl_gpio": 5
 },

Wiring the SSD1306 to the Micro Controller

Looking at the NodeMCU diagram you can see where the connections need to be made for the NeoPixel and SSD1306 display. SSD1306 SCL to D1, SDA to D2. The Neopixel data connection is now on D6. Power and GND using the PWR and GND pins. I’m using them all on the same side of the NodeMCU to make it fit cleanly into the case later.

NodeMCU.png

Init.js code additions

Incorporate the display library in your Init.js by including the line below.

load('api_arduino_ssd1306.js');

With that done we to initialize the display also in the Init.js. The following lines initialize the display address, SCL pin the display is connected to, the size of the text we are going to display and color. Put them before or after the initialization for the Neopixel.

//------------ Setting up Display ----------------
let oled_addr = 0x3C; // I2C Address for SSD1306let 
oled = Adafruit_SSD1306.create_i2c(5 /* RST GPIO */, Adafruit_SSD1306.RES_128_32);

// Initialize the display. 
oled.clearDisplay();
oled.setTextSize(2);
oled.setTextColor(Adafruit_SSD1306.WHITE);

In the MQTT Subscriber section where you are looking at the MQTT message being sent from the Microsoft Flow and displaying a color on the Neopixel add the following lines to send output to the display. The following below outputs Pink to the display. If Pink indicates some task then change oled.write(‘PINK’); to oled.write(‘TASK’); or similar.

 if (msg === "Pink"){
 // PINK 
 oled.clearDisplay();
 oled.setTextSize(2);
 oled.setCursor(1, 10);
 oled.write('PINK');
 oled.display();
 oled.startScrollLeft(0x00, 0x0F);

Following the Neopixel loop after

 strip.clear();
 strip.show(strip);

add the following to clear the display as the the Neopixel has finished displaying its color notification.

 oled.clearDisplay();
 oled.display();

Repeat for the differing colors and their tasks/meanings.

Summary

Now the notifier includes both a visual color notification AND the text associated with the notification. No confusion here, or does it need a buzzer as well?

 

Amazon QuickSight – An elegant and easy to use business analytics tool

First published at https://nivleshc.wordpress.com

Introduction

Recently, I had a requirement for a tool to visualise some data I had collected. My requirements were very simple. I didn’t want something that would cost me a lot, and at the same time I wanted the reports to be elegant and informative. Most of all, I didn’t want to have to go through pages and pages of documentation to learn how to use it.

As my data was within Amazon Web Services (AWS), I thought to check if AWS had any such offerings. Guess what, there was indeed a tool just for what I wanted, and after using it, I was amazed at how simple and elegant it is.

In this blog, I will show how you can easily get started with Amazon QuickSight. I will take you through the steps to import your data into Amazon QuickSight and then create some informative visualisations.

Some background on Amazon QuickSight

Pricing

Amazon QuickSight is very inexpensive, infact, if your data is not too much, you won’t have to pay anything!

For standard edition use, Amazon QuickSight provides 1GB of SPICE for the first user free per month. SPICE is an acronym for Super-fast, Parallel, In-memory, Calculation Engine and it uses a combination of columnar storage, in-memory technologies enabled through the latest hardware innovations, machine code generation, and data compression to allow users to run interactive queries on large datasets and get rapid responses.  SPICE is the calculation engine that Amazon QuickSight uses.

Any additional SPICE is priced at $USD0.25 per GB/month. For the latest pricing, please refer to https://aws.amazon.com/quicksight/#Pricing

Data Sources

Currently Amazon QuickSight supports the following data sources

  • Relational Data Sources
    • Amazon Athena
    • Amazon Aurora
    • Amazon Redshift
    • Amazon Redshift Spectrum
    • Amazon S3
    • Amazon S3 Analytics
    • Apache Spark 2.0 or later
    • Microsoft SQL Server 2012 or later
    • MySQL 5.1 or later
    • PostgreSQL 9.3.1 or later
    • Presto 0.167 or later
    • Snowflake
    • Teradata 14.0 or later
  • File Data Sources
    • CSV/TSV – (comma separated, tab separated value text files)
    • ELF/CLF – Extended and common log format files
    • JSON – Flat or semi-structured data files
    • XLSX – Microsoft Excel files

Unfortunately, currently Amazon DynamoDB is not supported as a native data source. Since my data is in Amazon DynamoDB, I had to write some custom lambda functions to export it to a csv file, so that it could be imported into Amazon QuickSight.

Ok, time for that walk-through I promised earlier.  For this blog, I will be using an S3 bucket as my data source. It will contain the CSV files that I will use for analysis in Amazon QuickSight.

Step 1 – Create S3 buckets

If you haven’t already done so, create an S3 bucket that will contain the csv files. The S3 bucket does not have to be publicly accessible. Once created, upload the csv files into the S3 bucket.

In my case, the csv file is called orders.csv and its location is https://s3.amazonaws.com/sample/orders.csv (to get the URL to your S3 file, login to the S3 console and navigate to the S3 bucket that contains the file. Click the S3 bucket to open it, then click the file name to open its properties. Under Overview you will see Link. This is the URL to the file)

Step 2 – Create an Amazon QuickSight Account

Before you start using Amazon QuickSight, you must create an account. Unfortunately, I couldn’t find a way for creating an Amazon QuickSight account without creating an Amazon AWS account. If you don’t have an existing Amazon AWS account, you can create an AWS Free Tier account. Once you have got an AWS account, go ahead and create an Amazon QuickSight account at https://aws.amazon.com/quicksight/.

While creating your Amazon QuickSight account, you will be asked if you would like Amazon QuickSight to auto-discover your Amazon S3 buckets. Enable this and then click to Choose S3 buckets. Choose the S3 bucket that you created in Step 1 above. This will give Amazon QuickSight read-only access to the S3 bucket, so that it can read the data for analysis.

Step 3 – Create a manifest file

A manifest file is a JSON file that provides the location and format of the data files to Amazon QuickSight. This is required when creating a data set for S3 data sources. Please refer to https://docs.aws.amazon.com/quicksight/latest/user/supported-manifest-file-format.html if you would like more information about manifest files.

Below is my manifest file, which I have affectionately named ordersmanifest.json.

{
   "fileLocations": [
      {
         "URIs": [
            "https://s3.amazonaws.com/sample/orders.csv"
         ]
      },
   ],
   "globalUploadSettings": {
      "format": "CSV",
      "delimiter": ",",
      "textqualifier": "'",
      "containsHeader": "true"
   }
}

Once created, upload the manifest file into the same S3 bucket as to where the csv file is stored.

Step 4 – Create a data set

  • Login to your Amazon QuickSight account. From the top right, click on Manage data
  • In the next screen, click on New data set
  • In the next screen, for Create a Data Set FROM NEW DATA SOURCES, click on S3
  • In the next screen
    • provide a name for the data source
    • for Upload a manifest file ensure URL is clicked and enter the URL to the manifest file (you can get the url by logging into the S3 console, and then clicking on the manifest file to reveal its properties. Under the Overview tab, you will see Link. This is the URL to the manifest file).NewS3DataSource
    • Click Connect
    • Amazon QuickSight will now read the manifest file and then import the csv file to SPICE. You will see the following screenFinishDataSetCreation
    • Click on Edit/Preview data.
    • In the next screen, you will see the contents of the data file that was imported, along with the Fields name on the left. If you want to exclude any columns from the analysis, simply untick them (I unticked orderTime (S) since I didn’t need it) EditPreviewDataSet
    • By default, the data is called Group 1. To customise the name, replace Group 1 with a text of your choice (I have renamed my data to Orders Data)RenameGroup1Label
    • Click Save & visualize from the top menu

Step 5 – Create Visualisations

Now that you have imported the data into SPICE, you can start analysing it and creating visualisations.

After step 4, you should be in the Analysis section.

  • Depending on which visualisation you want, you can select the respective type under Visual types from the bottom left hand side of the screen. For my visualisations, I chose Pie Chart (side note – you will notice that orderTime (S)  isn’t listed under Fields list. This is because we had unticked it in the previous screen)OrdersDataAnalysis-01
  • I want to create two Pie Charts, one to show me analysis about what is the most popular foodName and another to find out what is the most popular drinkName. For the first Pie Chart, drag foodName (S) from the Fields list to the Value – Add a measure here box  in the top of the screen. Then drag foodName (S) from the Fields list to the Group/Color – Add a dimension here box in the top of the screen. You will see the followingOrdersDataAnalysis-02
  • You can customise the visualisation title Count of Foodname (S) by Foodname (S) by clicking it and then changing the text (I have changed the title to Popularity of Food Types)FoodNamePopularity
  • If you look closely, the legend on the right hand side doesn’t serve much purpose since the pie slices are already labelled quite well. You can also get rid of the legend and get more space for your visual. To do this, click on the down arrow above FoodName (S) on the right and then select Hide legend FoodNameHideLengend
  • Next, lets create a Pie Chart visualisation for drinkName. From the top menu, click on Add and then Add visual drinkNameAddVisual
  • You will now have another Canvas at the bottom of the first Pie Chart. Click this new canvas area to select it (a blue border will appear to show that it is selected). From Visual types at the bottom left hand side, click on the Pie Chart visual. Then from the top, click on Field wells to expose the Value and Group/Color boxes for the second canvas drinkNameCanvas
  • From the Field list on the left, drag drinkName (S) to the Value – Add a measure here box  in the top of the screen. Then drag drinkName (S) from the Fields list to the Group/Color – Add a dimension here box in the top of the screen. You will now see the following foodanddrinkvisual
  • We are almost done. I actually want the two Pie Charts to sit side by side, instead of one ontop ofthe other. To do this, I will show you a neat trick. In each of the visuals, at the bottom right border, you will see two diagonal lines. If you move your mouse pointer over them, they change to a resizing cursor. Use this to resize the visual’s canvas area. Also, in the middle of the top border of the visual, you will see two rows of gray dots. Click your mouse pointer on this and drag to the location you want to move the visual to.VisualResizeandMove
  • I have hidden the legend for the second visual, customised the title and resized both the visuals and moved them side by side. Viola! Below is what I get. Not bad aye!BothVisualsSidebySide

Step 6 – Create a dashboard

Now that the visuals have been created, they can be shared it with others. This can be done by creating a dashboard. A dashboard is a read-only snapshot of the analysis. When you share the dashboard with others, they can view and filter the dashboard data, however any filters applied to the dashboard visual exist only when the user is viewing the dashboard, and aren’t saved once it is closed.

One thing to note about sharing dashboards – you can only share dashboards with users who have an Amazon QuickSight account.

Creating a dashboard is very easy.

  • In the Analysis screen, on the top right corner, click on Share and then select Create dashboardCreateDashboard
  • You can either replace an existing dashboard or create a new one. In our case, since we are creating a new dashboard, select Create a new dashboard as and enter a name for the dashboard. Once finished, click Create dashboardCreateDashboard-Name
  • You will then be asked to enter the username or email address of those you want to share the dashboard with. Enter this and click on Share ShareDashboard
  • That’s it, your dashboard is now created. To access it, go to the Amazon QuickSight home screen (click on the Amazon QuickSight icon on the top left hand side of the screen) and then click on All dashboards. Those that you have shared the dashboard with will also be able to see it once they login to their Amazon QuickSight account.AllDashboards

Step 6 – Refreshing the Data Set

If your data set continually changes, your visualisations/dashboards will not show the updated information. This can be done by refreshing the data set. Doing this will import the new data into SPICE, which will then automatically update the analysis/visualisations and dashboards

Note: you will have to manually reload the webpage to see the updated visualisations and dashboard

There are two ways of refreshing data sets. One is to do it manually while the other is to use a schedule. The scheduled data refresh allows for the data to be automatically refreshed at a certain time daily, weekly or monthly. A maximum of five scheduled refreshes can be configured.

The steps below show how you can manually refresh the data or create schedules to refresh the data

  • From the Amazon QuickSight main screen, click on Manage data from the top left of the screen ManageData
  • In the next screen, you will see all your currently configured data sets. Click the Orders Data dataset (this is the one we had created previously).
  • In the next screen, you will see Refresh Now and Schedule refreshManualScheduleDataRefresh
  • Clicking on Refresh Now will manually refresh the data. Clicking on Schedule refresh will bring up the screen where you can configure a schedule for refreshing the data automatically.

 

That’s it folks! Wasn’t that simple? If you already have an Amazon AWS account, I would strongly recommend giving Amazon QuickSight a try for all your analytics needs. Even if you don’t have an Amazon AWS account, I would still suggest getting an AWS free tier account to try it out.

Enjoy 😉

 

Azure Update Management

How do you patch/update your infrastructure in Azure, AWS, On-Premises? There are many ways, of course, including manually, built-in scheduled update, Group Policy, locally scripted, ConfigMgr, custom Azure Automation, WSUS, and so on.

Somewhat recently, another option “Azure Update Management” has become available, and it is FREE*. This is an expanded offering of what used to be OMS Update Management, integrated into the main Azure Portal and visible on each VM under the “Update Management” node.

Rather than regurgitate the existing documentation and tutorial, I want to highlight some of the finer points:

  • Yes, supports Windows and Linux
  • Requires ‘supported’ versions of Windows or Linux
  • Does not support ‘client’ versions e.g. Windows 7/8/8.1/10
  • Requires .NET Framework 4.5 and Windows Management Framework 5.0 or later on Windows 2008R2 SP1
  • Windows Server 2008 or 2008 R2 without SP1 won’t apply updates, just scan/assess
  • Update targets must have access to an update repository
    • WSUS, ConfigMgr SUP, or Microsoft Update
    • Linux package repository, either locally managed, or the OS default
  • Integration with ConfigMgr requires current branch 1606 or newer

Caveats

  • If an update reports that it requires a reboot, the VM will reboot. Currently there appears to be no way to avoid/defer a reboot
  • Windows VMs only scan for updates every 12 hours
  • Linux VMs scan for updates every 3 hours

I was about to build a WSUS server in an Azure subscription to address a number of manually updated or otherwise unmanaged Azure VMs, which was going to cost a minimum of about AUD $200 per month. This appears to be a nearly ideal solution to me and very attractive to the client at the ‘nearly free’ price point.

This is my new ‘go-to’ for update management; I hope it looks as good to you, and simplifies an important part of your environment.

FREE*

  • Requires a Log Analytics storage account, which does cost a small amount for ingestion and storage for 31 days (https://azure.microsoft.com/en-us/pricing/details/log-analytics/). The first 5 GB ingestion per month is free, which should be good for 50-100 VMs, leaving about 20c/GB/Month for storage costs – so maybe $1 per month for up to 100 VMs!
  • No further costs involved for Azure VMs
  • Non-Azure VMs/Computers could incur further charges, depending on your environment, use of extra Azure Automation/Configuration Management features, etc

Sample Update Status (Server name column removed)

AzureUpdateManagement

“Cannot complete this action” error in Sharepoint team site

Lately I was assigned a long standing issue in a well known organisation with Sharepoint 2013 on premise, in which some of its users have been getting this weird “Cannot complete this action” error screen whenever they delete a document from a file or modify a list view in their team sites.

Lots and lots of testings were done throughout a few days and I came out with the following an analysis summary:

  • Issue exists in some sub-sites of a team site under a team site collection (Sharepoint 2013 on-premise)
  • Error occurring consistently for team site users (including site collection admin), changes did get actioned/ saved
  • Users have to click back to get back to the previous screen to get back to the site
  • Error didn’t occur for some other team sites
  • No specific error correlation ID, nor anything suspicious in ULS log

Luckily i was able to find an answer from Microsoft. This was due to the fact that there was a load balanzer running which hasn’t had HTTP compression enabled plus these sub-sites having Minimal Download Strategy (MDS) site feature enabled.

Minimal Download Strategy feature was supposed to optimize loading speed for users as it allows the browser to only retrieve the changes required for a web page, rather than the whole page. However, when you have a load balanzer running for Sharepoint, HTTP compression will need to be enabled in order for MDS to work properly. Otherwise, simply turn off the site feature and this error will disappear.

See: https://support.microsoft.com/en-au/help/2934590/modifying-a-list-view-returns-cannot-complete-this-action-when-mds-is