I had promised a couple of blogs on dealing with the challenges of distance that are unavoidable as we adopt a variety of dispersed Cloud deployed services. The first was using a WCF Custom Channel to cache SharePoint content which is now a bit old school. This is the second.

The rate of change at the moment is astonishing. I’ve been sitting on blog number two for quite some time, but when I go to build an example and type it up, something new has come along that is a slightly better way of doing it. But then that’s why we are in this business, right? So it is with much haste that I blog this while it is still relevant, (I think it still is) or at the very least it is buzz word (Web API, SignalR, Owin and Katana) compliant.

Something in one of the previous blog comments got me thinking.
“Thanks peter for nice blog. It helps allot. I am planning to write Routing service which can route both SOAP and OData requests. I took your solution and tried to debug. I am getting empty string while reading message before sending to client.”

Turns out that while WCF is designed to be transport agnostic, it’s hard to get it to be message format agnostic. What would be good is something that doesn’t get in the way but sits off to the side of the messaging but can be used to get in and affect messages, like adding authentication, caching or some other functionality. What I need is a forward proxy, not a router.

From the great book “Succinctly” book series: A proxy server is a computer that sits between a client and server. A proxy is mostly transparent to end users. You think you are sending HTTP request messages directly to a server, but the messages are actually going to a proxy. The proxy accepts HTTP request messages from a client and forwards the messages to the desired server. The proxy then takes the server response and forwards the response back to the client. Before forwarding these messages, the proxy can inspect the messages and potentially take some additional actions.

Which got me wondering, what is it that happens when you tick that proxy box in your browser? Is there something special about those servers we use to browse traffic through?

So I built a simple webserver, set my local IIS as the proxy server and with some config and fiddling the IIS web server was receiving traffic on IIS when I was trying to browse sites on the Internet. OK, so what? What if we capture that traffic in a web service and send it on and hand back the replies to just as a proxy would?

Web API

Web API is great simple technology for building simple APIs that serve the HTTP protocol. How hard would it be to build a simple server using Web API that proxies traffic straight through? Turns out it’s easy. Way, way easier that in WCF! All you need to do is build a Web API DelegatingHandler like this:

public class ProxyHandler : DelegatingHandler
{
  protected override async System.Threading.Tasks.Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, System.Threading.CancellationToken cancellationToken)
  {
    UriBuilder forwardUri = new UriBuilder(request.RequestUri);
    //strip off the proxy port and replace with an Http port
    forwardUri.Port = 80;
    //send it on to the requested URL
request.RequestUri = forwardUri.Uri;
    HttpClient client = new HttpClient();
    var response = await client.SendAsync(request,HttpCompletionOption.ResponseHeadersRead);
    return response;
}
}

It’s worth taking a moment to understand what this code is doing. Every request that comes in on the proxy port (by convention I use port 8080) change it to port 80 and make the request then return the response. Simple. The other thing to note and the big change over my previous WCF version is the use of async. This enables the proxy to listen for new requests while previous requests are still pending responses, perfect for a proxy. Next is to configure the proxy:

config.Routes.MapHttpRoute(name: “Proxy”, routeTemplate: “{*path}”, handler: HttpClientFactory.CreatePipeline
  (
    innerHandler: new HttpClientHandler(), // will never get here if proxy is doing its job
    handlers: new DelegatingHandler[]
    { new ProxyHandler() }
  ),
  defaults: new { path = RouteParameter.Optional },
  constraints: null
);

For all requests ({*path}) use the default HttpClientHandler but put the ProxyHandler in the pipeline before it. All requests are actually handled by the ProxyHandler and should never fall through to the inner handler. There are couple of additional tricks to get it working in all cases. You’ll need this for some reason that I haven’t yet figured out but I think it’s a bug in Web API.

 //have to explicitly null it to avoid protocol violation
 if (request.Method == HttpMethod.Get)
   request.Content = null;

You’ll also need this in your web.config if you are going to host it in IIS (more on hosting options later) to tell IIS that you want the proxy code to handle all requests, no matter what!

<system.webserver>
<!–needed if you want to run in IIS and handle everything!–>
<handlers>
 <remove name=“ExtensionlessUrlHandler-Integrated-4.0“ />
<remove name=“OPTIONSVerbHandler“ />
<remove name=“TRACEVerbHandler“ />
<add name=“ExtensionlessUrlHandler-Integrated-4.0” path=“*.” verb=“*” type=“System.Web.Handlers.TransferRequestHandler” preCondition=“integratedMode,runtimeVersionv4.0“ />
</handlers>
</system.webServer>

And, most importantly, this will tell your proxy not to use a proxy when making requests to the Internet (to avoid an endless loop of proxy requests)

<system.net>
<defaultProxy enabled=“false“ />
</system.net>

Proxy done!

Hosting

The above code is hosting platform agnostic. This is one of the areas undergoing a lot of change in the industry right now. In what appears to be a new era of openness, open source and interoperability (even by Microsoft!) there are now a range of ways to host web services. Long story short .NET Web != IIS which is great. IIS is a very big, cumbersome engine if you don’t really need it. And this is a perfect example, as a simple forward proxy we don’t want or need any of that web serving functionality getting in the way, in fact we had to put special configuration into IIS to get it to leave our requests alone.

Since Web API is not tied to the web hosting framework in .NET, it supports self-hosting which is useful for running cheap API’s in Console apps or WorkerRoles. But now there is another option, Owin. Owin defines an open API for building web applications over a hosting platform by delivering well defined pluggable architecture. Katana is Microsoft’s implementation of OWIN. This article has a great explanation of how we got to such a complicated place with the pair of ASP.NET and IIS and how we are now finally winding it back out again now with Owin, Katana and various other supporting frameworks.

Like anything so new, this area is still in flux. The APIs are a bit green and changing and some simple modules (like Static File delivery) are still in beta. But it’s a great idea, and we can use it right away for our proxy with just a simple line of code we can spin up the proxy in a Console app anywhere.

public
static
void Start(string proxyAddress)
{
  try
  {
   // Start OWIN proxy host
   apps.Add(WebApp.Start<Proxy>(proxyAddress));
   Trace.TraceInformation(“Listening on:” + proxyAddress);
   Trace.WriteLine(“Set your IE proxy to:” + proxyAddress);
  }

The abstraction of Owin also enables us to just as easily deploy into Azure WorkerRole, again I prefer using a worker role so we don’t get all that annoying complexity and default modules that come with IIS and a Web Role.

foreach (var rie in RoleEnvironment.CurrentRoleInstance.InstanceEndpoints)
{
 var endpoint = rie.Value;
 if (endpoint.Protocol == “http”)
 {
  string baseUri = string.Format(“{0}://{1}”, endpoint.Protocol, endpoint.IPEndpoint);
  Trace.TraceInformation(String.Format(“Starting OWIN at {0}”, baseUri));
  Proxy.Start(baseUri);
 }
}
OK so now we have built a proxy server which can be used to channel all requests from your machine through the Web API proxy which is pretty powerful. We can deploy the proxy to Azure in US, UK or Asia and use it to browse sites unavailable in Australia. Try it out yourself by setting your IE proxy to busker.cloudapp.net port 8080. [The choice of the name “Busker” will become apparent in the next blog entry]

And try browsing www.google.com to see the results through the proxy (my US based proxy makes Google think I’m in Kendale Lakes Florida)

and direct ( that’s me with the with the proxy off getting Australian results)

But while that’s fun for tricking Google or for viewing NetFlix we could do the same with a publically available proxy or by deploying a commercial proxy server (TMG, Squid etc) into a virtual machine in the Cloud. The difference and power is in how we can inject custom behaviours using the WebAPI pipeline to get the proxy to affect the traffic.

Domain constrained handling

The idea of a proxy is to handle all traffic no matter what the domain. A web site or Web API is targeted at providing many urls under a single domain and it shows in the frameworks. They have great url mapping and filtering capability but largely ignore the domain. What if we want our proxy to behave differently for different domains? The answer is in this line:

constraints: null

We can add a custom constraint to ensure a handler is only invoked if the constraint is satisfied and in the constraint we can implement any sort of filtering we like.

constraints: new { isLocal = new HostConstraint { Host = “table.core.windows.net” } }

The HostConstraint class has a single method Match() which does something simple like:

public
bool Match(HttpRequestMessage request, IHttpRoute route, string parameterName, IDictionary<string, object> values, HttpRouteDirection routeDirection)
{ return request.RequestUri.Host.Contains(Host);}

So now I can make the proxy plug in different handlers and therefore behave differently for different domains!

Domain Specific Handlers

Previously I demonstrated using a WCF Router to modify the Azure Table API and make it a true oData interface which can be consumed by Excel. To do it we needed to create the Azure Table Storage authentication tokens and inject the oData paging elements over the vanilla table.core.net service. That same functionality has been ported to the proxy and plugged in as another handler which is constrained to just requests to the Azure Table Service (table.core.windows.net)

//now plug in some AzureAuth config handling using a Controller
//http://backtester.table.core.windows.net/config?connectionstring=DefaultEndpointsProtocol=https;AccountName=backtester;AccountKey=

config.Routes.MapHttpRoute(
  name: “ConfigAzureAuth”,
  routeTemplate: “config”,
  defaults: new { controller = “ConfigAzureAuth” },
  constraints: new { isLocal = new HostConstraint { Host = “table.core.windows.net” } }
);

Try setting the browser proxy to busker.cloudapp.net:8080 and use a browser or Excel to browse to http://backtester.table.core.windows.net/Quote(). This account holds a table with some stock quotes in it. Without the proxy in play this would fail because the AzureTableHandler injects the necessary authentication headers and paging over the straight Azure table API. Or submit your own storage keys to the Proxy to browse your own table storage in a browser or Excel using this config url

http://backtester.table.core.windows.net/config?connectionstring=DefaultEndpointsProtocol=https;AccountName=backtester;AccountKey=

Note here something a bit unusual, it looks like the Azure url http://backtester.table.core.windows.net supports a url path “config” but actually it doesn’t the Proxy picks up that url and handles it instead of sending on to Azure.

Client Specific Handlers

So now we can configure the Proxy to behave differently for different destinations, but what about different requesting clients? Yes we can, one (very simple) way is through ipaddress although we could also write domain specific cookies back to the user from the Proxy. I’ve used this ability to add some additional functionality using a handler in the pipeline if a user has registered for it. Try this (when your proxy is set to busker.cloudapp.net port 8080) http://www.buskerimage.com/flip or this http://www.buskerimage.com/bw. This website (www.buskerimage.com) doesn’t actually exist but the proxy is listening for that domain and using it to configure the proxy. In both cases the Proxy is remembering your ipaddress and manipulating the images on the way through either flipping them or going old school black and white (or both). Here’s what Bing Images looks like with both on.

Note only requests from and ip address registered for black and white and flipped images will see this view. OK so that’s all fun, what about the serious stuff? That’s in next blog when these building blocks are put to good use. But it’s fun to play with for now.

Running on Azure for now here: busker.cloudapp.net:8080

Available on github here: https://github.com/petermreid/buskerproxy