Application Insights is a component of Azure Monitor for application-level instrumentation. It collects telemetry from your application infrastructure like web servers, App Services, and Azure Functions apps, and from your application code. In this post we’ll discuss how Application Insights can be automated in several key ways: first, by setting up an Application Insights instance in an ARM template; second, by connecting it to various types of Azure application components through automation scripts including Azure Functions, App Services, and API Management; and third, by configuring its smart detection features to emit automatic alerts in a configurable way. As this is the first time in this series that we’ll deploy instrumentation code, we’ll also discuss an approach that can be used to manage the deployment of different types and levels of monitoring into different environments.

This post is part of a series:

• Part 1 provides an introduction to the series by describing why we should instrument our systems, outlines some of the major tools that Azure provides such as Azure Monitor, and argues why we should be adopting an ‘infrastructure as code’ mindset for our instrumentation and monitoring components.

• Part 2 (this post) describes Azure Application Insights, including its proactive detection and alert features. It also outlines a pattern for deploying instrumentation components based on the requirements we might typically have for different environments, from short-lived development and test environments through to production.

• Part 3 discusses how to publish custom metrics, both through Application Insights and to Azure Monitor. Custom metrics let us enrich the data that is available to our instrumentation components.

• Part 4 covers the basics of alerts and metric alerts. Azure Monitor’s powerful alerting system is a big topic, and in this part we’ll discuss how it works overall, as well as how to get alerts for built-in and custom metrics.

• Part 5 (coming soon) covers log alerts and resource health alerts, two other major types of alerts that Azure Monitor provides. Log alerts let us alert on information coming into Application Insights logs and Log Analytics workspaces, while resource health alerts us when Azure itself is having an issue that may result in downtime or degraded performance.

• Part 6 (coming soon) describes dashboards. The Azure Portal has a great dashboard UI, and our instrumentation data can be made available as charts. Dashboards are also possible to automate, and I’ll show a few tips and tricks I’ve learned when doing this.

• Part 7 (coming soon) covers availability tests, which let us proactively monitor our web applications for potential outages. We’ll discuss deploying and automating both single-step (ping) and multi-step availability tests.

• Part 8 (coming soon) describes autoscale. While this isn’t exactly instrumentation in and of itself, autoscale is built on much of the same data used to drive alerts and dashboards, and autoscale rules can be automated as well.

• Finally, part 9 (coming soon) covers exporting data to other systems. Azure Monitor metrics and log data can be automatically exported, as can Application Insights data, and the export rules can be exported and used from automation scripts.

Setting up Application Insights

When using the Azure Portal or Visual Studio to work with various types of resources, Application Insights will often be deployed automatically. This is useful when we’re exploring services or testing things out, but when it comes time to building a production-grade application, it’s better to have some control over the way that each of our components is deployed. Application Insights can be deployed using ARM templates, which is what we’ll do in this post.

Application Insights is a simple resource to create from an ARM template. An instance with the core functionality can be created with a small ARM template:

There are a few important things to note about this template:

• Application Insights is only available in a subset of Azure regions. This means you may need to deploy it in a region other than the region your application infrastructure is located in. The template above includes a parameter to specify this explicitly.
• The name of your Application Insights instance doesn’t have to be globally unique. Unlike resources like App Services and Cosmos DB accounts, there are no DNS names attached to an Application Insights instance, so you can use the same name across multiple instances if they’re in different resource groups.
• Application Insights isn’t free. If you have a lot of data to ingest, you may incur large costs. You can use quotas to manage this if you’re worried about it.
• There are more options available that we won’t cover here. This documentation page provides further detail.

After an Application Insights instance is deployed, it has an instrumentation key that can be used to send data to the correct Application Insights instance from your application. The instrumentation key can be accessed both through the portal and programmatically, including within ARM templates. We’ll use this when publishing telemetry.

Publishing Telemetry

There are a number of ways to publish telemetry into Application Insights. While I won’t cover them all, I’ll give a quick overview of some of the most common ways to get data from your application into Application Insights, and how to automate each.

Azure Functions

Azure Functions has built-in integration with Application Insights. If you create an Azure Functions app through the portal it asks whether you want to set up this integration. Of course, since we’re using automation scripts, we have to do a little work ourselves. The magic lies in an app setting, APPINSIGHTS_INSTRUMENTATIONKEY, which we can attach to any function app. Here is an ARM template that deploys an Azure Functions app (and associated app service plan and storage account), an Application Insights instance, and the configuration to link the two:

App Services

If we’re deploying a web app into an app service using ASP.NET, we can use the ASP.NET integration directly. In fact, this works across many different hosting environments, and is described in the next section.

If you’ve got an app service that is not using ASP.NET, though, you can still get telemetry from your web app into Application Insights by using an app service extension. Extensions augment the built-in behaviour of an app service by installing some extra pieces of logic into the web server. Application Insights has one such extension, and we can configure it using an ARM template:

As you can see, similarly to the Azure Functions example, the APPINSIGHTS_INSTRUMENTATIONKEY is used here to link the app service with the Application Insights instance.

One word of warning – I’ve found that the site extension ARM resource isn’t always deployed correctly the first time the template is deployed. If you get an error the first time you deploy, try it again and see if the problem goes away. I’ve tried, but have never fully been able to understand why this happens or how to stop it.

ASP.NET Applications

If you have an ASP.NET application running in an App Service or elsewhere, you can install a NuGet package into your project to collect Application Insights telemetry. This process is documented here. If you do this, you don’t need to install the App Services extension from the previous section. Make sure to set the instrumentation key in your configuration settings and then flow it through to Application Insights from your application code.

API Management

If you have an Azure API Management instance, you might be aware that this can publish telemetry into Application Insights too. This allows for monitoring of requests all the way through the request handling pipeline. When it comes to automation, Azure API Management has very good support for ARM templates, and its Application Insights integration is no exception.

At a high level there are two things we need to do: first, we create a logger resource to establish the API Management-wide connection with Application Insights; and second, we create a diagnosticresource to instruct our APIs to send telemetry to the Application Insights instance we have configured. We can create a diagnostic resource for a specific API or to cover all APIs.

The diagnostic resource includes a sampling rate, which is the percentage of requests that should have their telemetry sent to Application Insights. There is a lot of detail to be aware of with this feature, such as the performance impact and the ways in which sampling can reduce that impact. We won’t get into that here, but I encourage you to read more detail from Microsoft’s documentationbefore using this feature.

Here’s an example ARM template that deploys an API Management instance, an Application Insights instance, and configuration to send telemetry from every request into Application Insights:

Smart Detection

Application Insights provides a useful feature called smart detection. Application Insights watches your telemetry as it comes in, and if it notices unusual changes, it can send an alert to notify you. For example, it can detect the following types of issues:

• An application suddenly sends back a higher rate of 5xx (error)-class status responses than it was sending previously.
• The time it takes for an application to communicate with a database has increased significantly above the previous average.

Of course, this feature is not foolproof – for example, in my experience it won’t detect slow changes in error rates over time that may still indicate an issue. Nevertheless, it is a very useful feature to have available to us, and it has helped me identify problems on numerous occasions.

Smart detection is enabled by default. Unless you configure it otherwise, smart detection alerts are sent to all owners of the Azure subscription in which the Application Insights instance is located. In many situations this is not desirable: when your Azure subscription contains many different applications, each with different owners; or when the operations or development team are not granted the subscription owner role (as they should not be!); or when the subscriptions are managed by a central subscription management team who cannot possibly deal with the alerts they receive from all applications. We can configure each smart detection alert using the proactiveDetectionConfigs ARM resource type.

Here is an example ARM template showing how the smart detection alerts can be redirected to an email address you specify:

In development environments, you may not want to have these alerts enabled at all. Development environments can be used sporadically, and can have a much higher error rate than normal, so the signals that Application Insights uses to proactively monitor for problems aren’t as useful. I find that it’s best to configure smart detection myself so that I can switch it on or off for different environments, and for those environments that do need it, I’ll override the alert configuration to send to my own alert email address and not to the subscription owners. This requires us to have different instrumentation configuration for different environments.

Instrumentation Environments

In most real-world applications, we end up deploying the application in at least three environments: development environments, which are used by software developers as they actively work on a feature or change; non-production environments, which are used by testers, QA engineers, product managers, and others who need to access a copy of the application before it goes live; and production environments, which are used by customers and may be monitored by a central operations team. Within these categories, there can be multiple actual environments too – for example, there can be different non-production environments for different types of testing (e.g. functional testing, security testing, and performance testing), and some of these may be long-lived while others are short-lived.

Each of these different environments also has different needs for instrumentation:

• Production environments typically need the highest level of alerting and monitoring since an issue may affect our customers’ experiences. We’ll typically have many alerts and dashboards set up for production systems. But we also may not want to collect large volumes of telemetry from production systems, especially if doing so may cause a negative impact on our application’s performance.
• Non-production environments may need some level of alerting, but there are certain types of alerts that may not make sense compared to production environments. For example, we may run our non-production systems on a lower tier of infrastructure compared to our production systems, and so an alert based on the application’s response time may need different thresholds to account for the lower expected performance. But in contrast to non-production environments, we may consider it to be important to collect a lot of telemetry in case our testers do find any issues and we need to diagnose them interactively, so we may allow for higher levels of telemetry sampling than we would in a production environment.
• Development environments may only need minimal instrumentation. Typically in development environments I’ll deploy all of the telemetry collection that I would deploy for non-production environments, but turn all alerts and dashboards off. In the event of any issues, I’ll be interactively working with the telemetry myself anyway.

Of course, your specific needs may be different, but in general I think it’s good to categorise our instrumentation across types of environments. For example, here is how I might typically deploy Application Insights components across environments:

Once we’ve determined those basic rules, we can then implement them. In the case of ARM templates, I tend to use ARM template parameters to handle this. As we go through this series we’ll see examples of how we can use parameters to achieve this conditional logic. I’ll also present versions of this table with my suggestions for the components that you might consider deploying for each environment.

Configuring Smart Detection through ARM Templates

Here’s a large ARM template that includes everything we’ve created in this post, and has the three environment type modes:

Summary

Application Insights is a very useful tool for monitoring our application components. It collects telemetry from a range of different sources, which can all be automated. It provides automatic analysis of some of our data and has smart detection features, which again we can configure through our automation scripts. Furthermore, we can publish data into it ourselves as well. In fact, in the next post this series, we’ll discuss how we can publish custom metrics into Application Insights.