HoloLens – Using the Windows Device Portal

Windows Device Portal is a web-based tool which was first introduced by Microsoft in the year 2016. The main purpose of the tool is to facilitate application management, performance management and advanced remote troubleshooting for Windows devices.  Device portal is a lightweight web server built into the Windows Device which can be enabled in the developer mode. On a HoloLens, developer mode can be enabled from the settings application (Settings->Update->For Developers->Developer Mode).

Connecting the Device

Once the developer mode on the device is enabled, the following steps must be performed to connect to the device portal.

  1. Connect the device to the same Wi-Fi network as the computer used to access the portal. Wi-Fi settings on the HoloLens can be reached through the settings application (Settings > Network & Internet > Wi-Fi).
  2. Note the IP address assigned to the HoloLens. This can be found in the ‘Advance Options’ screen under the Wi-Fi settings (Settings > Network & Internet > Wi-Fi > Advanced Options).
  3. On the web browser of your computer, navigate to “https://<HoloLens IP Address>”
  4. As you don’t have a valid certificate installed, you will be prompted with a warning. Proceed to the website by ignoring the warning.

Holo WDP1

  1. For the first connection, you will be prompted with a credential reset screen. Your HoloLens should now blackout with only a number displayed in the centre of your viewport. Note this number and enter this in the textbox highlighter below

Holo WDP2

  1. You should also enter a ‘username’ and ‘password’ for the portal to access the device in future.
  2. Click ‘Pair’ button
  3. You will now be prompted with a form to enter the username and password. Fill this form and click on the ‘Login’ button.Holo WDP3
  4. Successful login will take you to the home screen of the Windows Device Portal

Holo WDP4

Portal features

The portal has a main menu (towards the left of the screen) and a header menu (which also serves as a status bar). Following diagram elaborates on the controls available in the header menu

Holo WDP5

The main menu is divided into three sections

  • Views – Groups the Home page, 3D view and the Mixed Reality Capture page which can be used to record the device perspective.
  • Performance – Groups the tools which are primarily used for troubleshooting applications installed on the device or the device itself (hardware and software)
  • System – functionalities to retrieve and control the device behaviour ranging from the managing the applications installed on the device to control the network settings.

A detailed description of the functionalities available under each menu item can be found on the following link:

https://developer.microsoft.com/en-us/windows/mixed-reality/using_the_windows_device_portal

Tips and tricks

Windows device portal is an elaborate tool and some of its functions are tailor-made for advanced troubleshooting. Following are few features of interest to get started with the development.

  • Device name – Device name can be access from the home screen under the view category. It is helpful in the long run to meaningfully name your HoloLens for easier identification over a network. You can change this from the Device Portal.

Holo WDP6

  • Sleep setting – HoloLens has a very limited battery life which ranges from 2 to 6 hours based on the applications we are running on it. This makes the sleep settings important. You can control the sleep setting from home screen of the Device Portal. Keep in mind that during development, you may usually leave the device plugged in (using UBS cable to your laptop/desktop). To avoid the pain of starting the device repeatedly, it is recommended to configure the ‘sleep time when plugged’ into a larger number.

Holo WDP7

  • Mixed reality capture – This feature is very handy to capture the device perspective for application demonstrations. You can capture the vide through the PV (Picture/Video) camera, the audio from the microphones, audio from the app and the holograms projected by the app on a collated media stream. The captured media content is saved on the device and can be downloaded to your machine from the portal. However, the camera channel cannot be shared between two applications. So, if you have an application using the camera of the device your recording will not work by default.
  • Capturing ETW trace – Performance tracking feature under the performance category is very handy for collecting ETW traces for a specific period. The portal enables you to start and stop the trace monitoring. The ETL events captured during this period is logged into a file which can be downloaded for further analysis.
  • Application management – Device portal provides a set of features to manage the applications installed on the HoloLens. These features include deploying, starting and stopping applications. This feature is very useful when you want to remotely initiate an application launch for a user during a demo.

Holo WDP8

  • Virtual Keyboard – HoloLens is not engineered to efficiently capture inputs from the keyboard. Pointing at a virtual keyboard using HoloLens and making an ‘airtap’ for every key press can be very stressful. Device portal gives you an alternate way to input text into your application from the ‘virtual input’ screen.

Holo WDP9

Windows Device Portal APIs

Windows Device Portal is built on top of a collection of public REST APIs. The API collection consists of a set of core APIs common to all Windows devices and specialised sets of APIs for specific devices such as HoloLens and Xbox. The HoloLens Device Portal APIs are bundled under the following controllers

  • App deployment – Deploying applications on HoloLens
  • Dump collection – Managing crash dumps from installed applications
  • ETW – Managing ETW events
  • Holographic OS – Managing the OS level ETW events and states of the OS services
  • Holographic Perception – Manage the perception client
  • Holographic Thermal – Retrieve thermal states
  • Perception Simulation Control – Manage perception simulation
  • Perception Simulation Playback – Manage perception playback
  • Perception Simulation Recording – Manage perception recording
  • Mixed Reality Capture – Manage A/V capture
  • Networking – Retrieve information about IP configuration
  • OS Information – Retrieve information about the machine
  • Performance data – Retrieve performance data of a process
  • Power – Retrieve power state
  • Remote Control – Remotely shutdown or restart the device
  • Task Manager – Start/Stop an installed application
  • Wi-Fi Management – Manage Wi-Fi connections

The following link captures the documentation for HoloLens Device Portal APIs.

https://developer.microsoft.com/en-us/windows/mixed-reality/device_portal_api_reference.

The library behind this tool is an open source project hosted on GitHub. Following is a link to the source repository

https://github.com/Microsoft/WindowsDevicePortalWrapper

HoloLens – Setting up the Development environment

HoloLens is undoubtedly a powerful invention in the field of Mixed reality. Like any other revolutionary inventions, the success of a technology largely depends upon its usability and its ease of adoptability. This is what makes software development kits (SDK) and application programming interfaces (API) associated with a technology super critical. Microsoft has been very smart on this front when it comes to HoloLens. Rather than re-inventing the wheel, they have integrated HoloLens development model with the existing popular gaming platform ‘Unity’ for modelling a Mixed reality application frontend.

Unity is a cross-platform gaming engine which was first released in the year 2005. Microsoft’s collaboration with Unity was first announced in the year 2013 when it started supporting Windows 8, Windows Phone 8 and Xbox One development tools. Unity’s support for Microsoft HoloLens was announced in 2015, since then, it has been the Microsoft recommended platform for developing game worlds for Mixed reality applications.

With Unity in place for building the game world, the next piece of the puzzle was to choose the right platform for scripting the business logic, deploying and debugging the application. For obvious reasons such as flexibility in code editing and troubleshooting, Microsoft Visual Studio was chosen as the platform for authoring HoloLens applications. Visual Studio Tools for Unity is integrated into to the Unity platform which enables it to generate a Visual Studio Solutions for the game world. Scripts can be coded using C# in Visual Studio to contain the operational logic for the Mixed reality application.

HoloLens Application Architecture

A typical Mixed reality application development had multiple tiers. The frontend game world which is built using Unity, the application logic used to power the game objects which is coded using C# in Visual Studio and backend services which encapsulated the business logic. Following diagram illustrates the architecture of such an application.

hldev1

In the above diagram, the box towards the right (Delegated services) is an optional tier for a Mixed reality application. However, many real-world scenarios in the space of Augmented and Mixed reality are powered by strong backend systems such as machine learning and data analytics services.

Development Environment

Microsoft does not use a separate SDK for HoloLens development. All you will require is Visual Studio and Windows 10 SDK. The following section lists the ideal hardware requirements for a development machine.

Hardware requirements

  • Operating system – 64 Bit Windows 10 (Pro, Enterprise, or Education)
  • CPU – 64 Bit, 4 cores. GPU supporting Direct X 11.0 or later
  • RAM – 8 GB or more
  • Hypervisor – Support for Hardware-assisted virtualization

Once the hardware is sorted, you will need to install the following tools for HoloLens application development:

Software requirements

  • Visual Studio 2017 or Visual Studio 2015 (Update 3) – While installing Visual Studio, make sure that you have selected the Universal Windows Platform development workload and the Game Development with Unity workload. You may have to repair/upgrade visual studio is its already installed on your machine without these workloads.
  • HoloLens Emulator – The HoloLens emulator can be used to test a Mixed reality application without deploying it on the device. The latest version of the emulator can be downloaded from the Microsoft web site for free. One thing to remember before you install the emulator is to enable the hypervisor on your machine. These may require changes in your system BIOS
  • Unity – Unity is your front-end game world modelling tool. Unity 5.6 (2017.1) or above supports HoloLens application development. Configuring the Unity platform for a HoloLens application is covered in later sections of this blog.

Configuring Unity

It is advisable to take a basic training in Unity before jumping onto HoloLens application development. Unity offers tons of free training materials online. The following link should lead you to a good starting point

https://unity3d.com/learn/tutorials

Once you are comfortable with basics of Unity, within a Unity project, follow the steps listed in the below URL to configure it for a Microsoft Mixed reality application.

https://developer.microsoft.com/en-us/windows/mixed-reality/unity_development_overview

Following are few things to note while we perform the configuration

  • Build settings – While configuring the build settings, it is advisable to enable the ‘Unity C# project’ checkbox to generate a Visual Studio solution which can be used for debugging your application.

hldev2

  • Player settings – apart from the publishing capabilities (Player settings->Publish settings->Capabilities) listed in the link above, there may be the requirement for your application to have specific capabilities such as an ability to access the picture library or the Bluetooth channel. Make sure that the capabilities required by your application are selected before building the solution.

hldev3

  • Project settings – Although the above-mentioned link recommends the quality to be set to fastest, this might not be the appropriate setting for all applications. It may be good to start with the quality flag set to ‘Fastest’ and then update the flag if required based on your application needs.

hldev4

One these settings are configured, you are good to create your game world for the HoloLens application. The unity project can then be built to generate a Visual Studio solution. The build operation will pop up a file dialog box to shoes a folder where you want the visual studio solution to be created. It is recommended to create a new folder for the Visual Studio solution.

Working with Visual Studio

Irrespective of the frontend tool used to create the game world, a HoloLens application will require Visual Studio to deploy and debug the application. The application can be deployed on a HoloLens emulator for development purpose.

The below link details the steps for deploying and debugging an Augmented/Mixed reality application on HoloLens using Visual Studio.

https://developer.microsoft.com/en-us/windows/mixed-reality/using_visual_studio

Following are few tips which may help set-up up and deploy the application effectively:

  • Optimizing the deployment – Deploying the application over USB is multiple time faster than deploying it over Wi-Fi. Ensure that the architecture is chosen as ‘X86’ before you fire the deploy.

hldev5

  • Application settings – You will observe that the solution has a Universal Windows Project set as your start-up project. To change the application properties such as application name, description, visual assets, application capabilities, etc., you can update the package manifest.

Following are the steps:

  1. Click on project properties

hldev6

2. Click on Package Manifest

hldev7

3. Use the ‘Application’, ‘Visual Assets’ and ‘Capabilities’ tabs to manage your application properties.

hldev8

HoloLens application development toolset comes with another powerful tool called ‘Windows Device Portal’ to manage your HoloLens and the applications installed on it. More about this in my next blog.

 

 

HoloLens – understanding the device

HoloLens is without doubt the next coolest product launched by Microsoft after Kinect. Before understanding the device lets quickly familiarize ourselves with the domain of Mixed reality and how it is different from Virtual and Augmented reality.

VR, AR and MR

Virtual reality, the first of the lot, is a concept of creating a virtual world about the user. This means all that the user sees or hears is simulated. The concept of virtual reality is not new to us. A simpler form of virtual reality was achieved back in 18th century using panoramic paintings and stereoscopic photo viewers. Probably the first implementation of a commercial virtual reality application was the “Link Trainer”, a flight simulator invented in the year 1929.

The 1st head mounted headset for Virtual reality was invented in the year 1960 by Morton Heilig. This invention enabled the user to be mobile, thereby introducing possibilities of better integrating with their surroundings. The result was the era of sensor packed headsets which can track movement, sense depth, heat, geographical coordinates and so on.

These head mounted devices then became capable of projecting 3D and 2D information on see- through screens. This concept of overlaying the content on the real world was termed as Augmented reality. The concept of Augmented Reality was first introduced in the year 1992 by Boeing where it was implemented to assist workers for assembling wire bundles.

The use cases around Augmented reality started strengthening over the following years. When virtual objects were projected on the real world, the possibilities of these objects interacting with the real-world objects started gaining focus. This brought in the invention of the concept called Mixed reality. Mixed reality can be portrayed as the successor of Augmented reality where the virtual objects projected in the real world are anchored to and interact with the real-world objects. HoloLens is one of the most powerful devices in the market today which can cater to Augmented and Mixed reality applications.

Birth of the HoloLens – Project Baraboo

After Windows Vista, repairing amazon forest and project Natal (popularly known as Kinect) Alex Kipman (Technical fellow, Microsoft) decided to focus his time on a machine which could not only see what a person sees but also understand his environment and project things on his line of vision. While building this device Kipman was keen on preserving the perpetual vision of the user to ensure that he or she does not feel blindfolded. He used his knowledge from his previous invention, Kinect, around depth sensing and recognizing objects.

The end product was a device with an array of cameras, microphones and other smart sensors all feeding information to a specially crafted processing module which Microsoft calls a Holographic Processing Unit (HPU). The device is capable of mapping its surroundings and understating the depth of the world in its field of vision. It can be controlled by gestures and by voice. The user’s head acts like the point device with a cursor which shines in the middle of his view port. HoloLens is also a fully functional Windows 10 computer.

The Hardware

Following are the details of the sensors built into the HoloLens:

HoloSense

  • Inertial measurement unit (IMU) – The HoloLens IMU consists of an accelerometer, gyroscope, and a magnetometer to help track the movements of the user.
  • Environment sensing cameras – The devices come with 4 environment sensing cameras used to recognize the orientation of the device and for spatial mapping
  • Depth Camera – The depth camera in this device is used for finger tracking and for spatial mapping
  • HD Video camera – A generic high definition camera which can be used by applications to capture video stream
  • Microphones – The device is fitted with an array of 4 microphone to capture voice commands and sound from 360 degrees
  • Ambient light sensor – A sensor used to capture the light intensity from the surrounding environment

The HoloLens also comes with the following two built-in processing units and storage

HoloChip

  • Central Processing Unit – Intel Atom 1.04 GHz processor with 4 logical processors.
  • Holographic Processing Unit – HoloLens Graphics processor based on Intel 8086h architecture
  • High Speed memory – 2 GB RAM and 114 MB dedicated Video Memory
  • Storage – 64 GB flash memory.

HoloLens supports Wi-Fi (802.11ac) and Bluetooth (4.1 LE) communication channels. The headset also comes with a 3.5 mm audio jack and a Micro USB 2.0 multi-purpose port, the device has a battery life of nearly 3 hours when fully charged.

More about the software and development tools in my next blog.

Enterprise Application platform with Microservices – A Service Fabric perspective

An enterprise application platform can be defined as a suite of products and services that enables development and management of enterprise applications. This platform should be responsible of abstracting complexities related to application development such as diversity of hosting environments, network connectivity, deployment workflows, etc. In a traditional world, applications are monolithic by nature. A monolithic application is composed of many components grouped into multiple tiers bundled together into a single deployable unit. Each tier here can be developed using a specific technology and will have the ability to scale independently. Monolithic application usually persists data in one common data store.

MicroServices - 1

Although a monolithic architecture logically simplifies the application, it introduces many challenges as the number of applications in your enterprise increases. Following are few issues with a monolithic design

  • Scalability – The unit of scale is scoped to a tier. It is not possible to scale bundled within an application tier without scaling the whole tier. This introduces massive resource wastage resulting in increase in operational expense.
  • Reuse and maintenance – The components within an application tier cannot be consumed outside the tier unless exposed as contracts. This forces development teams to replicate code which becomes very difficult to maintain.
  • Updates – As the whole application is one unit of deployment, updating a component will require updating the whole application which may cause downtime thereby affecting the availability of the application.
  • Low deployment density – The compute, storage and network requirements of an application, as a bundled deployable unit may not match the infrastructure capabilities of the hosting machine (VM). This may lead to wastage of shortage of resources.
  • Decentralized management – Due to the redundancy of components across applications, supporting, monitoring and troubleshooting becomes expensive overheads.
  • Data store bottlenecks – If there are multiple components accessing the data store, it becomes the single point of failure. This forces the data store to be highly available.
  • Cloud unsuitable – The hardware dependency of this architecture to ensure availability doesn’t work well with cloud hosting platforms where designing for failure is a core principle.

A solution to this problem is an Application platform based on Microservices architecture. Microservice architecture is a software architecture pattern where applications are composed of small, independent services which can be aggregated using communication channels to achieve an end to end business use case. The services are decoupled from one another in terms of the execution space in which they operate. Each of these services will have the capability to be scaled, tested and maintained separately.

MicroServices - 2

Microsoft Azure Service Fabric

Service Fabric is a distributed application platform that makes it easy to package, deploy, and manage scalable and reliable Microservices. Following are few advantages of a Service Fabric which makes it the ideal platform to build a Microservice based Application Platform

  • Highly scalable – Every service can be scaled without affecting other services. Service Fabric will support scaling based on VM scale sets which means that these services will have the ability to be auto-scales based on CPU consumption, memory usage, etc.
  • Updates – Services can be updated separately and different versions of a service can be co-deployed to support backward compatibility. Service Fabric also supports automatic rollback during updates to ensure consistency of an application deployment.
  • State redundancy – For state full Microservices, the state can be stored alongside compute for a service. If there are multiple instances of a service running, the state will be replicated for every instance. Service Fabric takes care of replicating the state changes through the stores.
  • Centralized management – The service can be centrally managed, monitored and diagnosed outside application boundaries.
  • High density deployment – Service Fabric supports high density deployment on a virtual machine cluster while ensuring even utilization of resources and distribution of work load.
  • Automatic fault tolerance – The cluster manager of Service Fabric ensures failover and resource balancing in case of a hardware failure. This ensures that your services are cloud ready.
  • Heterogeneous hosting platforms – Service Fabric supports hosting your Microservices across Azure, AWS, On premises or any other datacenter. Cluster manager is capable of managing service deployments with instances spanning multiple datacenters at a time. Apart from Windows, Service Fabric also supports Linux as a host operating system for your micro services.
  • Technology agnostic – Services can be written in any programming language and deployed as executables or hosted within containers. Service Fabric also supports a native Java SDK for Java developers.

Programming models

Service Fabric supports the following four programming models for developing your service:

  • Guest Container – Services packaged into Windows or Linux containers managed by Docker.
  • Guest executables – Services can be packaged as guest executables which are arbitrary executables, written in any language.
  • Reliable Services – An application development framework with fully supported application lifecycle management. Reliable services can be used to develop stateful as well as stateless services and supports transactions.
  • Reliable Actors – A virtual actor based development framework with built-in state management and communication management capabilities. Actor programming model is single threaded and is ideal for hyper scale out scenarios (1000s or instances)

More about Service Fabric programming models can be found here

Service Type

A service type in Service Fabric consists of three components

MicroServices - 3

Code package defines an entry point to the service. This can be an executable or a dynamic linked library. Config package specifies the service configuration for your services and the data package holds static resources like images. Each package can be independently versioned. Service fabric supports upgrade of each of these packages separately. Allocation of a service in a cluster and reallocation of a service on failure are responsibilities of Service Fabric cluster manager. For stateful services, Service Fabric also takes care of replicating the state across multiple instances of a service.

Application Type

A Service Fabric application type is composed of one or more service types.

MicroServices - 4

An application type is a declarative template for creating an application. Service fabric uses application types for packaging, deployment and versioning Microservices.

State stores – Reliable collections

Reliable Collections are highly available, scalable and high performance state store which can be used to store states alongside compute for Microservices. The replication of state and persistence of state on secondary storage is taken care of by the Service Fabric framework. A noticeable difference between Reliable Collections and other high-availability state store (such as cache, tables, queues, etc.) is that the state is kept locally in the service hosting instance while also being made highly available by means of replication. Reliable collections also support transactions and are asynchronous by nature while offering strong consistency.

More about reliable collection can be found here

Management

Service Fabric offers extensive health monitoring capabilities with built-in health status for clusters and services and custom app health reporting. Services are continuously monitored for real-time alerting on problems in production. Performance monitoring overheads are diluted with rich performance metrics for actors and services. Service Fabric analytics is capable of providing repair suggestion thereby supporting preventive healing of services. Custom ETW logs can also be captured for guest executables to ensure centralized logging for all your services. Apart from support for Microsoft tools such as Windows Azure Diagnostics, Operational Insights, Windows Event Viewer and Visual studio diagnostics events viewer, Service Fabric also supports easy integration with third party tools like Kibana and Elastic search as monitoring dashboards.

Conclusion and considerations

Microsoft Service Fabric is potential platform capable of hosting enterprise grade Microservices. Following are few considerations to be aware of while using Service Fabric as your Microservice hosting platform.

  • Choosing a programming model is critical for optimizing the management of Microservices hosted on Service Fabric. Reliable services and Reliable actors are more thickly integrated with the Service Fabric cluster manager compared to guest containers and guest executables.
  • The support for containers on Service Fabric is in an evolving stage. While Windows containers and Hyper-V containers are on the release roadmap, Service Fabric only supports Linux containers as of today.
  • The only two native SDKs supported by Service fabric as of today is based on .net and Java.

 

Cursor-Scoped Event Aggregation Pattern – for high performance aggregation queries

Often, stateful applications executing enterprise scenarios are not just interested in the current state of the system, but also in the events which caused the transition that resulted in the particular state. This requirement has exponentially increased the popularity of the Event Sourcing Pattern.

Event Sourcing Patterns ensure that all changes/events that caused a transition in application state are stored as a sequence of events in an append-only store. These events can then be queried for tracking history or even be used to reconstruct system state in case of an unexpected application crash.

The Event Sourcing design pattern is best suited to decouple commands and queries to improve efficiency of the system. An implicit dimension which is recorded in this pattern along with the events of interest is the ‘time’ when the event occurred.  This post talks about a design pattern which enhances event sourcing pattern making it capable of portraying views based on a time range.

Problem

Apart from the requirement of replaying events to recreate state, some systems also have requirements around aggregating information drawn from events which occurred during a specific timeframe.

Running a query against the event source table to retrieve this information can be an expensive operation. The event source table has a tendency to grow quickly if it is designated to capture events related to multiple entity types. In this case the performance of querying this table to retrieve this information will not be very performant.

Apart from the requirement of replaying events to recreate state, some systems also have requirements around aggregating information drawn from events which occurred during a recent time frame. Running a query against the event source table to retrieve this information will be an expensive operation. The event source table also has a tendency to grow very fast if it is designated to capture events related to multiple entity types. In which case the efficiency of querying the event store to retrieve this information will be sub-optimal.

Event Sourcing Pattern

Solution

A solution to this problem is to introduce a Cursor-Scoped Event Aggregation Pattern on top of the Event Sourcing Pattern. The cursor here will be associated with a configurable time range which will define the scope of events the query is interested in. The pattern replicates the tuples of relevance (based on the cursor) in an in-memory data dictionary based on the filtering criteria defined in the query. The Aggregation function will collate the properties of interest within this dictionary to produce results for the query.

Cursor-Scoped Event Aggregation Pattern

The following flowchart captures the activities which are performed on the cursor data store when a new event is appended in the store or when a new query is fired against it.

CA-flowchart

 

Related Cloud Design Patterns

  • Event Sourcing Pattern – This pattern assumes an event sourcing pattern to be implemented by the system.
  • Index Table Pattern – Index table pattern is an alternative to improve performance around querying event sources.

Considerations

  • Eventual consistency – Event Sourcing Pattern is usually implemented in scenarios which can support eventual consistency. As we are querying the event source directly rather than the entities, there is a chance of inconsistency between the state of the entities and the result of the query.
  • Immutability – The event store is immutable, the only way to reverse a transaction is to introduce a compensation event. The Aggregator should be capable of handling compensation events.

Usage Scenarios

  • High performance systems – Systems which required real time response on aggregation queries.

Inside Azure – Deployment workflow with Fabric Controller and Red Dog Front End

Abstracting complexities around developing, deploying and maintaining software applications have diminished the importance of understanding underlying architecture. While this may work well for today’s aggressive delivery cycles, at the same time, it impacts the ability of engineers to build an efficient, optimal solution which aligns with the internal architecture of the hosting platform. Architects and engineers should be cognizant of the architecture of the hosting environment to better design a system. The same holds good for Microsoft Azure as a hosting provider.

This post is an attempt to throw light on the workflow around deploying workload on Microsoft Azure, the systems involved in this process and their high-level architecture.

 Architecture

To start with let’s look at the high level architecture of an Azure datacenter. The following diagram illustrates the physical layout of an Azure Quantum 10 V2 datacenter.

FC1

Figure 1.0

Parking the physical layers for a later post, we shall focus on last layer termed as ‘Clusters’ to understand the logical architecture of Microsoft Azure datacenters.

Logical Architecture

Clusters are logical group of server racks. A typical cluster will include 20 server racks hosting approximately 1000 servers. Clusters are also known as ‘Stamps’ internally within Microsoft. Every cluster is managed by a Fabric Controller. Fabric Controller, often considered as the brain of the entire Azure ecosystem is deployed on dedicated resources with in every cluster.

FC2

Figure 1.1

Fabric Controller is a highly available, stateful, replicated application. Five machines on every Azure Datacenter cluster are dedicated for Fabric Controller deployment. One server out of the five servers acts as the primary and replicates the state to the other four secondary servers at regular intervals. This is to ensure high availability of this application. Fabric controller is an auto healed system. Hence, when a server goes down, one of the active servers will take charge as the primary and spin up another instance of Fabric Controller.

Fabric controller is responsible for managing the Network, Hardware and the Provisioning of servers in an Azure datacenter. If we visualize the whole datacenter as a machine we can map the server (hardware) as the datacenter itself, kernel (of the operating system) as Fabric controller and processes running on the machine as services hosted on the datacenter. The following image illustrates this perspective:

FC3

Figure 1.2

Fabric controller controls the allocation of resources (such as compute and storage), both for your deployed custom applications as well as for built-in Azure services. This provides a layer of abstraction to the consumer there by ensuring better security, scalability and reliability.

Fabric controller takes inputs from following two systems:

  • Physical Datacenter deployment manager – The physical layout of the machines on the racks, their IP addresses, routers address, certificates for accessing the router, etc. in XML format.
  • RDFE (discussed later in the post) – The deployment package and configuration.

Following flowchart captures the boot up tasks performed by the Fabric Controller when it instantiates a cluster:

FC6

Figure 1.3

 Deployment

The following diagram illustrates a high-level view of the deployment process:

FC4

Figure 2.0

Before we understand the deployment workflow, it’s important to familiarize with another software component in the Azure ecosystem which is primarily responsible for triggering a deployment. RDFE (Red Dog Front End), named after the pre-release code name for Microsoft Azure (Red Dog) is a highly available, Azure deployed application which feeds deployment instructions to the Fabric controller. It is responsible for collecting the deployment artefacts, making copies of it, choosing the target cluster for deployment and triggering the deployment process by sending instructions to the Fabric Controller. The following flowchart details the workflow handled by RDFE:

FC5

Figure 2.1

A fact to keep in mind is that Azure portal is stateless and so are the management APIs. During a deployment, the uploaded artefacts are passed on to RDFE which stores these artefacts in a staging store (RDFE store). RDFE chooses five clusters for every deployment. If a cluster fails to complete the deployment operation, RDFE picks another cluster from the other four and restarts the deployment operation.

Once a cluster is picked, the deployment workflow is handed over to the Fabric controller. Fabric controller performs the following tasks to provision the required hardware resources as per the application/service configuration.

FC7

Figure 2.2

Things to remember

Following are few key learnings from my understanding of Azure Architecture:

  • Keep your workload small – As discussed in the Deployment section (Figure 2.1, 2.2) , the uploaded workload gets copied multiple times before its deployed on a virtual machine. While deploying an application package, for better efficiency, it is recommended that the code is separated from the other dependent artefacts and only code is packaged as the workload. The dependent artefacts can be separately uploaded into Azure storage or any other persistent store.
  • Avoid load balancer whenever you canFigure 2.2 illustrates steps to update the Azure load balancer post deployment. By default all traffic to a node (server) is routed through the load balancer. Every node(server) is associated with a virtual IP address when it is deployed. Using this to directly communicate with the server reduces one hop in the network route. However, this is not advisable under all circumstances. This may be well suited if the Server is hosting a singleton workload.
  • Deploy the largest resource first – The choice of clusters to deploy is made my RDFE in the early stages of deployment as illustrated in Figure 2.1. Not all clusters have all machine configurations available. So if you want co-allocation of servers on a cluster choose to deploy the largest resource first. This will force RDFE to pick a more capable cluster to deploy the workload. All subsequent workloads will be deployed on the same cluster if they are placed under the one affinity group.
  • Create syspreped master image VHDs with drivers persisted – Azure optimizes the ISO download operation (on a node) by referring to a pre-fetched cache as shown in Figure 2.2.  While using custom images for Virtual Machines, it is advisable to persist drivers so that Azure can utilize pre-fetched driver caching to optimize your deployment

WebHook your WebJob – With Visual Studio Team Services integration example

WebHooks, also known as ‘HTTP(S) call backs’ are becoming very popular for reporting asynchronous events to trigger business workflows. The latest release of Microsoft Azure WebJobs can now be triggered using WebHooks. In this post I will cover the configuration of a WebJob to use a WebHook as a trigger using a sample scenario of integrating a WebJob with Visual Studio Team Services (VSTS) to explain the workflow.

Support for WebHooks is packaged as an extension to WebJobs and is currently in a pre-release state. You must install the following NuGet package to start using WebHooks with WebJobs:

  • Package name – Microsoft.Azure.WebJobs.Extensions.WebHooks
  • Verion – 1.0.0-beta4
  • Author – Microsoft.

To install this using the NuGet Package Manager, make sure that you have checked the Include prerelease as shown below.

Package Manager with prereleases

As I said previously, the WebHook extension is currently prerelease as Microsoft is adding capabilities to it, one of which is to enrich its security model. Presently, WebHooks only support basic authentication using the Azure App Services’ publishing credentials.

Using the WebHook trigger in WebJob

The first step after installing the extension is to update the WebJob’s startup code to load the extension

Now we can write a function which can be triggered by a WebHook

In the above code, I have a private method (SendEmail) which is called from the ReportBug method to send an email (using SendGrid) when the WebHook fires the function. The code uses an API Key to connect to the SendGrid service. You can provision a free SendGrid service through Microsoft Azure Portal.

SendGrid configuration

The WebJob can now be published in a Microsoft Azure App Service container.

Configuring Visual Studio Online

To register this WebHook in Visual Studio Team Services, use the following steps:

  1. Login to the visual studio online web portal
  2. Navigate to settings.

VSTS settings

  1. Click on the ‘Service Hooks’ tab and add a new service hook

VSTS Service Hooks

  1. Pick WebHook service and click next

VSTS Service Hook Selection

  1. Pick the ‘Work item created’ trigger. We can leave the filters open for this example. Click next

VSTS Service Hook Trigger

  1. Construct your WebHook URL.

The WebHook URL for the WebJob hosted on Azure within an App Service should be in the following format:

https://Site/api/continuouswebjobs/Job/passthrough/Path

  • Site – App Service’s  SCM site (e.g. yoursite.scm.azurewebsites.net)
  • Job – The name of your WebJob
  • Path – Path to the WebHook function. This should be of the format ClassName/MethodName

In our case this would look like

https://appservicename.scm.azurewebsites.net/api/continuouswebjobs/WebJobName/passthrough/Functions/ReportBug

Extract the credentials from your App Services Publish Profile.

  1. Fill the URL, Basic Authentication username and password and then click on ‘Test’.

VSTS Service Hook Test

  1. If all configurations are right and the WebJob is running, you will see a pop-up similar to that below.

VSTS Service Hook Test

  1. Click on finish

Testing the integration

Creating any new work item in Visual Studio Team Services should now trigger an email with a JSON describing the work item in the body of the email.

Sample email message

You should also be able to see the trace messages in the WebJob Dashboard.

Key learnings

  • WebHook triggers are best suited for integrating heterogeneous workflows
  • This feature is still in preview. This should be considered before using it for any production solutions
  • Only Basic Authentication is supported for WebHooks today, but this is expected to change in future releases. As we are sharing the publishing credentials with the calling application, it’s important to ensure that the calling application is trusted.

Highly available WordPress deployment on Azure

WordPress is the leading content management system today owning more than 50% of the market share. WordPress on Microsoft Azure is becoming a very popular offering with the ability to host WordPress as an Azure WebApp. While Microsoft has made the process of spinning up a WordPress site very easy with built in gallery images, running business critical applications on a cloud platform also introduces challenges in terms on availability and scalability.

WordPress Architecture

A typical WordPress deployment consists of the following two tiers:

  • Web Frontend – PHP web site
  • Backend Data store
    • Relational data store – Hierarchical entity store
    • Object data store – Used to store uploaded images and other artefacts

In order to guarantee high availability and scalability, we need to ensure that each of these service tiers are decoupled and can be separately managed.

 Web frontend

Windows Azure supports WordPress to be deployed as an Azure WebApp. Azure WebApp is a part of the Azure App Services offering, a fully managed service which can host mission critical web applications.

Azure WebApps natively supports scaling which can be achieved by increasing the number of instances of the hosting WebApp. The native Azure load balancer takes care of distributing the traffic amongst these instances in a ‘round robin’ manner. WebApps also support schedule-driven and automatic scaling.

HAWP1

Azure Portal can be used to configure scaling rules for the WebApp.

Azure WebApps offers an uptime of 99.95% for basic, standard and premium tiers even with a single instance of deployment . Azure Load Balancer takes care of managing failover amongst instances within a region. To achieve higher availability, the application front end can be deployed across different geographical regions and Azure Traffic manager can be employed to handle load balancing, network performance optimization and failover.

HAWP2.PNG

Backend Store

WordPress back consists of two data stores. One, a relational data store which is used to store WordPress entities along with their hierarchies and second, an object store used to persist artefacts.

Azure Gallery hosts a prebaked image for Scalable WordPress, which lets you configure a scalable MySQL backend as the relational store and uses Azure Storage as the object store for your WordPress deployment. Scalable WordPress uses Azure Blob store to host uploaded artefacts.  This works well for most of the scenarios.

HAWP3.PNG

It is important to understand that by using a MySQL backend for your WordPress site you are engaging with a third party database-as-a-service provider (which in this case is Clear DB). This means that the availability SLA associated with your WordPress backend is not provided by Microsoft. Always check the SLAs associated with your chosen pricing tier with the third party provider.

An alternative is to use Project Nami, which offers a WordPress image configured to run against Azure SQL Database as the back end. You can deploy the WordPress site with a Project Nami image either from the Azure gallery or from project’s website. Nami supports WordPress version 4.4.2 with a fully configurable Azure SQL Database backend. Once deployed, the WordPress instance can be configured to use Azure Storage as the object store by employing Windows Azure Storage for WordPress plugin.

HAWP4.PNG

Using SQL Azure and Azure Storage as backend for your WordPress site has the following key advantages

  • Eligibility for Microsoft SLA – A minimum uptime of 99.9% is guaranteed on Azure Storage and an uptime of 99.99% is guaranteed on Basic, Standard, or Premium tiers of Microsoft Azure SQL Database.
  • Easily manageable – Azure SQL databased can be provisioned and managed using Azure Resource Manager templates or through the portal.
  • On-demand scaling – Azure SQL database supports on-demand scaling to meet changing business demands and traffic.
  • More secure – Azure SQL databases offers better support around auditing and compliance apart from offering highly secure connections.

 Conclusion

To conclude, you can deploy a highly available WordPress site in Microsoft Azure by ensuring that the front end and backend tiers are fault tolerant and designed for failure. The deployment strategy can be influenced by the following factors:

  • Choice of technology for the backend – Microsoft (SQL Azure)/Non-Microsoft (MySQL)
  • SLA requirements
  • Logging and Auditing requirements
  • Scaling, security and manageability requirements

References

Project Nami – http://projectnami.org/

Scalable WordPress – https://azure.microsoft.com/en-us/marketplace/partners/wordpress/scalablewordpress/

 

Azure WebJob logs demystified

Asynchronous jobs are usually hard to troubleshoot due to the very nature of its execution. This post talks about how we can monitor and trouble shoot Azure WebJobs both during development and when it is deployed on an Azure Web App. The key is to understand the layout of the logs the WebJob runtime creates during its execution.

WebJob storage accounts

To enable logging, WebJob needs two Azure storage account connection strings to be configured:

  • AzureWebJobsDashboard
  • AzureWebJobsStorage

AzureWebJobsDashboard

This storage account is primarily used by Azure WebJob SDK to store logs from the WebJobs Dashboard. This connection string is optional and is required only if you are planning to use the dashboard for monitoring WebJobs.

The WebJob runtime creates two containers under this storage account with the names ‘azure-webjobs-dashboard’ and ‘azure-jobs-host-archive’. The azure-webJobs-dashboard container is used by the WebJob dashboard to store host and execution endpoint (function) details. Azure-jobs-host-archive is used as an archive for execution logs.

AzureWebJobsStorage

AzureWebJobsStorage should point to a storage account which will be primarily used for logging. WebJob runtime creates two containers in this storage account with the names ‘azure-jobs-host-output’ and ‘azure-webjobs-host’. If you point AzureWebJobsDashboard and AzureWebJobsStorage  at two different storage accounts, you will notice that these two containers are duplicated in both the storage accounts.

azure-webjobs-host container in-turn hosts three directories:

  • Heartbeats – Containing 0 byte blogs for every heartbeat check performed on the service.
  • Ids – Containing the directory with a single blog holding a unique identifier for this service.
  • Output-logs – Hosts the output of the explicit logs for each run. Explicit logs being logs introduced by WebJob developers within the execution code.

azure-jobs-host-output container is the key for troubleshooting web jobs. This container hosts logs created by the WebJob runtime during initialization and termination of every execution.

Understanding azure-jobs-host-output (Storage Container)

To understand output logs, let’s consider the following code which defines a task to be executed by the WebJob.

When the WebJob is executed (either on a dev environment from Visual studio or by hosting within a WebApp), an output log is created in the azure-jobs-host-output container in the following format.

WebJob start log

Note: The maroon text below are comments explaining the JSON elements and are not part of the actual log file.

Once the execution is complete, a blob file in the following format is created by the WebJob runtime.

Using WebJob Dashboard

WebJob dashboard provides a detailed, user friendly interface to manage your WebJobs hosted on Microsoft Azure. The dashboard will be enabled once the Azure WebApp hosting the WebJob is deployed and will be accessible on the following path

https://<WebApp Name>.scm.azurewebsites.net/azurejobs/#/jobs

The dashboard can also be reached via Azure Portal.

namit_webjoblogs1

On the homes screen, dashboard will list the web jobs hosted within the selected WebApp and their current execution status.

namit_webjoblogs1

On clicking on a specific WebJob, dashboard will navigate you to a page displaying the list of executions for that WebJob along with their status.

namit_webjoblogs2

Each execution can be further expanded to see the explicit logging.

namit_webjoblogs4

For most cases, the logs displayed on the portal should supply sufficient information for us to troubleshoot the WebJob. Exploring the WebJob logs in Azure Storage Accounts can be used for deeper investigations.

References

Azure WebJobs SDK – https://github.com/Azure/azure-webjobs-sdk

MSDN Documentation – https://azure.microsoft.com/en-us/documentation/articles/app-service-webjobs-readme/

Cloud Cushioning using Azure Queues

The distributed world

The cloud revolution has revived the importance of distributed computing in today’s enterprise market with the distribution of compute and storage workloads across multiple decoupled resources helping corporates optimise their capital and operational expenditure.

While there are benefits of moving to the cloud, it’s important to understand the ground rules of the cloud platform. Running your business critical services on commodity hardware with a service SLA of three nines (99.9) against five nines (99.999) does call for some precautions. A key mitigation is to adhere to the recommendations provided by the cloud platform provider for application hosting.

The Microsoft published article on cloud development recommendations is a the perfect cheat sheet.

Amongst many design patterns and recommendations for designing and developing applications for the cloud, designing the right methodology for enabling asynchronous communication between software services ultimately plays a key role in determining the reliability, scalability and efficiency of your application.

Why employ a Queue?

Queuing is as an effective solution for enabling asynchronous communications between software services. The following are few benefits of employing a queuing model:

  1. Minimal dependency on service availability – As queues act as a buffer between software components, the availability of a service will not impact another as they can function in a disconnected fashion.
  2. High reliability – Queues uses transactions to manage the messages stored in them. In case of a failure, the transaction can be rolled back to recover the message.
  3. Load balancing – Queues can be used for load balancing work between software services. Microsoft recommends a Queue based load leveling pattern as an ideal implementation of this.

Microsoft Azure provides two queuing solutions which can be used to enable asynchronous communication between software services:

  • Azure Storage Queues – an early Azure feature which is a part of Azure storage service offer REST based reliable persistent messaging capability.
  • Azure Service Bus Queues – introduced as a part of Azure Service Bus services to support additional features such as publish/subscribe and topics.

Picking the right queuing technology plays a significant role in the efficiency of a distributed cloud application. In the rest of this post I will cover a few important factors you should consider when choosing one.

What is the size of messages being transferred?

The maximum message size supported by Azure Storage Queues is 64KB while Azure Service Bus Queues support messages up to 256KB. This becomes an important factor especially when the message format is padded (such as XML). An ideal pattern to use for transferring larger chunks of data is to use Azure Storage Blobs as a transient store. The data can be stored as a blob and the link to the blob can be communicated to the consuming service using queues.

Does your ‘message consuming service’ go offline?

This is mostly applicable for batch processing systems which are designed to be dormant/offline periodically. In such a scenario the maximum size of the queue becomes an important factor to consider when choosing a queuing technology. Azure Storage Queues can grow to a maximum size of 200TB while Azure Service Bus Queues can only hold a maximum 80GB of data.

Another factor which impacts the choice of technology is the message expiration duration. In case of batch processing systems it is likely that the system only consumes messages once every few days or weeks. The maximum message expiry period for Azure Storage Queues is 7 days after which the messages cannot be recovered. In case of Azure Service Bus Queues the message expiry duration is unlimited.

Does the order of messages matter?

Although all queues are expected to follow FIFO (first in first out) ordering, it is not guaranteed in the case of Azure Storage Queues. Azure Service Bus Queues, however, guarantees FIFO ordering of messages at all times.

Does your messaging infrastructure require auditing?

Server side logs for operations on the queues is only supported on Azure Storage Queues. A custom implementation is required to capture queuing events if Azure Service Bus Queues are used.

What is the preferred programming model for your applications?

The messages from a queue can be consumed by two methods. A push (publish/subscribe) or a pull (polling) action. Azure Service Bus Queues supports both push and pull models to consume messages while Azure Storage Queues support only a pull model.

Does your application require features like dead letter handling, grouping, scheduling, forwarding or support for transactions?

Azure Service Bus Queues supports advanced features such as dead letter queues, dead letter events, message grouping, message forwarding, duplicate detection, at most once delivery and transactions. These features are not supported by Azure Storage Queues.

Queue design patterns

Here are a few useful design patterns which can be used to leverage the potential of Azure Queues in a distributed application hosted on cloud

Detailed comparison

The following table compares the features of Azure Storage Queues, Azure Service Bus Queues and Amazon Simple Queuing Service (SQS) in detail.

Features Azure Service Bus Queues Azure Storage Queues
Provisioning
API support Yes Yes
PowerShell command lets Support Yes Yes
Local (Australia) availability Yes Yes
Security
Encryption No No
Authentication Symmetric key Symmetric key
Access control RBAC via ACS Delegated access via SAS tokens
Auditing No Yes
Identity provider federation Yes No
Scale
Max no: Queues per account 10,000 (per service namespace, can be increased) Unlimited
Max Queue size 1 GB to 80 GB 200 TB
Max message size 256 KB 64 KB
Max message expiration duration Unlimited 7 days
Max concurrent connections Unlimited Unlimited
Max no: records returned per call 5000
Poison Messages
Dead letter handling Yes No
Dead letter events Yes No
Consumption patterns
One-way messaging Yes Yes
Request response Yes Yes
Broadcast messaging Yes No
Publish-Subscribe Yes No
Batch processing
Message grouping Yes No
Scheduling
Message scheduling Yes No
Transactions
Transaction support Yes No
Delivery
Assured FIFO Yes No
Delivery guarantee At-Least-Once

At-Most-Once

At-Least-Once
Receive behaviour Blocking with/without timeout

Non-blocking

Non-blocking
Receive Mode Peek & Lease

Receive & Delete

Peek & Lease
Lease/Lock duration 60 seconds (default) 30 seconds (default)
Auto forwarding Yes No
Duplicate detection Yes No
Peek message Yes Yes
Monitoring
Server side logs No Yes
Storage metrics Yes Yes
State management Yes No
Management
Purge queue No Yes
Management Protocol REST over HTTPS REST over HTTP/HTTPS
Runtime Protocol REST over HTTPS REST over HTTP/HTTPS
Development
.Net managed API Yes Yes
Native C++ API No Yes
Java API Yes Yes
PHP API Yes Yes
Node.js API Yes Yes
Queue naming rules Yes Yes
Performance
Maximum throughput Up to 2,000 messages per second

(based on benchmark with 1 KB messages)

Up to 2,000 messages per second

(based on benchmark with 1 KB messages)

Average latency 20-25 ms 10 ms
Throttling behaviour Reject with exception/HTTP 503 Reject with HTTP 503

Useful links

Azure Queues and Service Bus Queues – compared and contrasted.

Overview of Service Bus Messaging Patterns.