Do It Yourself Fiddler Service

I recently upgraded to Windows 8.1 which required a full install (upgraded from the 8.1 Preview which annoyingly didn’t support upgrades). A full install of my laptop is getting easier and easier as more of the things I use are delivered as services. The install list is getting smaller due to the combined effect of software as a service and a simpler working life.

I still had to install these:

• Microsoft Office 2013
• Microsoft Visio 2013
• Microsoft Project (yes yes I know but there really is no good alternative yet)
• LastPass
• Visual Studio 2013
• And…Fiddler!

The install list from bare metal to productive machine is getting smaller and smaller (arguably Office is available online but slightly under featured for my liking). The world of development is moving online too. There is a significant move away from the personal development environment backed by shared source control and build machine to a development environment that is team first, always integrated and continuously saved to repository and built by default. The editing and debugging experience is moving online too. Take a look at jsFiddle for a window into the online development experience. I think we’ll see a lot more in this space and much stronger interaction with continuously integrated source code repositiories.

Fiddler

Anyway I digress, what I really wanted to highlight is Fiddler. That invaluable tool that injects itself as a proxy between your browser and the internet and even performs a man in the middle attack on your secure traffic. I use it all the time for everything from solving performance problems to reverse engineering web sites. But what does the Fiddler of the future look like? How can it be relevant in service oriented world where the PC is no longer the centre of your universe but just a dumb terminal? In a cloud and service oriented world there are a great deal of conversations happening between machines hosted remotely that we are interesting in listen in on. When developing a modern cloud solution which is aggregating a number of RESTful services, sometimes the most interesting conversations are happening not between browser and server but between machines in a data centre far-far away.

What I need is a Fiddler as a Cloud Service

SignalR

Alert readers from my previous post may have noticed something; there was a technology mentioned in the forward that I didn’t really use in the implementation of the WebAPI Proxy, “SignalR”. SignalR is one of those cool new technologies built over the WebSockets protocol that is a technological leap forward in Internet technologies and a game changer for implementing complex interactive websites that significantly augments the page request based model of the Internet we have been stuck with since the 80’s. Broadly speaking the WebSockets API enables a server to open, and maintain a connection to the browser which can be used to push information to the client without user interaction. This is much more like the traditional old thick client application model where TCP connections are held open for creating an interactive real-time user experience. While this technology will (and has) opened the door for a whole new style of web based applications; it invites back invites back all those scalability problems of managing client state which was so artfully dodged by the disconnected stateless model the Internet is built on today.

So what’s all this got to do with Fiddler?

What if we could deploy to the cloud a WebAPI Proxy as described in the previous blog but when passing through traffic, use Signal-R to feed that logging information back to a browser. Then we’d have something much like Fiddler but not installed on the local machine but rather as a service in the cloud; able to intercept and debug traffic of anyone choosing to use it as a proxy. Sounds good in theory but how to implement it?

Again as I demonstrated with the proxy, it’s relatively easy to plug in handlers into the pipeline to modify the behaviour of the proxy, and that includes putting in a logging handler to watch all traffic passing through. Something like this

protected
 override
 async System.Threading.Tasks.Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, System.Threading.CancellationToken cancellationToken)
 {
   //add a unique Id so we can match up request and response laterrequest.Properties["Id"] = System.Guid.NewGuid().ToString();
   //log the request
    await LogRequest(request);
   //start a timer
    Stopwatch stopWatch = Stopwatch.StartNew();
   //pass the request through the proxy 
    var response = await
    base.SendAsync(request, cancellationToken);
   //stop the timerstopWatch.Stop();
   //log the response and the time taken
    await LogResponse(request, response, stopWatch);
    return response;
 }

Now comes the interesting part, adding Signal-R from nuget adds a folder into your WebAPI project called Hubs. Now a Signal-R “Hub” is your opportunity to set up the relationship between an incoming browser request and an outgoing signal (or push) to the browser. It is in this hub area that you can add handlers for javascript functions on the client and use the Context.ConnectionId to identify the incoming request that can later be used to push out data to the same client.

 $("#start").click(function ()
 {
 loggingHub.server.start();
 });

 public
 void
 Start()
 {
  Trace.WriteLine("connection", Context.ConnectionId);
 }

So that’s pretty cool, we can call functions on the server from the client javascript (once you understand the conventions of properly naming to deal with language case differences), but then you could always call functions on the server from the client, that’s what HTTP is. The difference here is in the server side function “Start” there is a Context.ConnectionId, this is a GUID that represents the caller and can be used to later to asynchronously call the user back.

Fiddler as a Service

A service isn’t a service unless it’s multi-tenant. So what we’ll do is build the ability for a client to register interest in just some of the logs passing through the proxy. For simplicity we’ll just pick up the user’s ipaddress and push logs at them for anything we see running through the proxy from that ipaddress. That way we’ll get something similar to the functionality of the local Fiddler running on the local machine where we set the internet proxy for browsing and see the logs for anything running through the proxy on that machine.

Remember (from the previous post), the code written for this proxy is designed to be hosting platform agnostic; that is it can be run in IIS but can also be run in the new simple hosting platforms such as OWIN. But this throws up some interesting challenges when trying to sniff out HTTP constants like ipaddress which are represented through different APIs. So I use some code like this to stay generic.

public static string GetClientIp(this HttpRequestMessage request)
{
   if (request.Properties.ContainsKey("MS_HttpContext"))
   {
     return ((dynamic)request.Properties["MS_HttpContext"]).Request.UserHostAddress as string;
   }
   else if (request.Properties.ContainsKey("MS_OwinContext"))
   {
     return ((dynamic)request.Properties["MS_OwinContext"]).Request.RemoteIpAddress as string;
   }
   else if (request.Properties.ContainsKey(RemoteEndpointMessageProperty.Name))
   {


     RemoteEndpointMessageProperty prop;
     prop = (RemoteEndpointMessageProperty) request.Properties[RemoteEndpointMessageProperty.Name];
     return prop.Address;
   }
   else
   {
     throw new Exception("Could not get client IP");
   }
}

Next we need a page on the proxy that the client can request to register their interest in logs, this is called logging.html and apart from including the necessary signal-R javascript glue, just has a couple of buttons to Start and Stop logging and a list to accept the logs coming back from the browser. (Serving that static page was a bit of a challenge at the time of writing since there was no official Static File Handler for OWIN so I wrote my own.)

Now here’s where the interesting part of SignalR shows itself. There is some javascript on the page which defines a function called “logUrl“ and some code on the server which calls that function from the WebAPI handler. So whenever a request or response comes through the proxy we look up the ipaddress, determine which registered hub connection it maps to and call the appropriate client’s logUrl function which logs the data in the browser window.

 private static async System.Threading.Tasks.Task LogRequest(HttpRequestMessage request)
 {
 string connection;
 if (connections.TryGetValue(request.GetClientIp(), out connection))
 {
   var loggingHubContext = GlobalHost.ConnectionManager.GetHubContext<LoggingHub>();
   await loggingHubContext.Clients.Client(connection).LogUrl(request.RequestUri.ToString());
 }
 }

 loggingHub.client.logUrl = function (url)
 {
  $('#logs').append('<li>' + url + '</li>');
 };

Anyway, what we should end up with is something that looks like this:

Busker

What better name for Fiddler as a Service but “Busker”? Busker Proxy is running as a WorkerRole in Azure. (I can’t use the free Azure WebSites because the proxy effectively serves traffic for any url but Azure WebSites uses host headers to determine which site to send traffic to.) While I’m still hosting the service for a limited time, try this:

Set your internet proxy to http://busker.cloudapp.net port:8080

Browse to http://busker.cloudapp.net/logging.html

Hit Start and then open a new browser window and browse to a (non secure) website like http://www.google.com. You should see something like this in your first browser window.

For now it’s an interesting debugging tool and could do with quite a bit more work. But more importantly it demonstrates how the technologies available to us as developers are now helping to bridge the gap between web client and server to provide better, more interactive software as a service from a remote, deployed, multi-tenant environment replacing the need to install applications locally other than a browser.

On the todo list for Busker:

• There are some very interesting certificate tricks that Fiddler plays on your local machine to implement the man in the middle attack which will be quite hard to implement securely in a Fiddler as a Service
  (I might give that a go in a later blog).
• The logging screen should be able to nominate an ipaddress rather than just using the one you come from so you can sniff traffic that doesn’t just originate from your machine
• The logging view needs to properly match up requests and responses so you can see things like response time and outstanding requests

Check out the code here https://github.com/petermreid/buskerproxy

Let me know if there are any cool features I should consider adding.

Happy Busking