Extensible Monitoring Solution

Introduction

The solution is comprised of a number of components. A windows service (F#, C#) that performs the monitoring. This loads libraries of 'monitoring' functions via reflection, that are used to instantiate concrete instances of 'monitor's. The monitors can be composed into 'MonitorTree' types, using Boolean composition operators - for example, if one wanted to monitor that a process was running, but only at certain times, you could compose an 'And' MonitorTree consisting of a monitor checking that the process was running, and a monitor checking that the current datetime was between those specified. The service publishes the monitoring data via JSON over websockets to a server (Azure worker role, F#). The clients (WPF, C#, F#) authenticate, and subscribe to the websocket server, and display the monitoring data in charts. The client can also configure the service, to edit existing monitors, create new ones etc. Communication is brokered through the Azure service.

Background

I chose this for my entry into the App Innovation Contest, chiefly because most of the service/server side code was already written.

The code

The service

The Monitor is declared as an abstract class, with the only abstract member being 'SampleFunc', a function taking a string[] as parameter, returning an int.

[<AbstractClass>]
[<DataContract>]
type Monitor (s:MonitorSettings) =
 
let mutable _val = 0
    let threshExceededEvt = new Event<_>()
    abstract member SampleFunc : string[] -> int
    [<DataMember>]
    member x.Name = s.name
    [<DataMember>]
    member x.Threshold = s.threshold
    [<DataMember>]
    member x.Frequency = s.freq
    [<DataMember>]
    member x.Value with get() = _val
    [<DataMember>]
    member x.IsOutOfThreshold = _val > x.Threshold
    member x.Sample() = async {
                               _val <-x.SampleFunc(s.parms)
                               if x.IsOutOfThreshold then
                               threshExceededEvt.Trigger(x) 
                              }
    member x.ThresholdExceeded = threshExceededEvt.Publish

This allows for instances of the class to be instantiated with different SampleFuncs, that know how to monitor different things. For example to monitor the size of a file, you could provide a SampleFunc thus:

let FileSizeFunc(parms:string[])   = let file = FileInfo(parms.[0])
                                     let size = file.Length / 1024L //in KB
                                     int size

And to monitor if the current time is between two specified (as strings in the parms string[]):

let DateTimeFunc(parms:string[])   = let strtDte = match DateTime.TryParse(parms.[0]) with
                                                   |true,start -> start
                                                   |false,_    -> new DateTime()
                                     let endDte  = match DateTime.TryParse(parms.[1]) with
                                                   |true,nd    -> nd
                                                   |false,_    -> new DateTime()
                                     if ((strtDte < DateTime.Now) && (DateTime.Now < endDte)) then 1
                                     else 0

Windows sensor APIs

As this competition is intended to highlight special capabilities of ultrabook, I am also going to implement some SampleFuncs using the sensor APIs. This requires a bit of fettling, as us F# devotees seem to have been overlooked in the world of WinRT, so far - it is not possible to referrence WinRT libarary from an F# project (as far as I can tell). So, the project needs to be C#. But, this gives us a further problem, becuase F# functions are not the same as C# Func delegates. F# uses FSharpFunc<>, so we need to convert the C# Func delegate to an FSharpFunc type. This can be done using extensions in the FSharp powerpack. First, I create a C# library project. Then, I need to referrence the WinRT library, as detailed here http://www.hanselman.com/blog/HowToCallWinRTAPIsInWindows8FromCDesktopApplicationsWinRTDiagram.aspx[^] so I can use the sensor APIs. Basically, you unload the project, and edit the .csproj file manually, adding:

<PropertyGroup>
  <TargetPlatformVersion>8.0</TargetPlatformVersion>
<PropertyGroup>

Then, for the Func -> FSharpFunc conversion, I need to referrence FSharp.Core, FSharp.PowerPack, and FSharp.PowerPack.Linq dlls, and open the namespaces.

using Microsoft.FSharp;
using Microsoft.FSharp.Control;
using Microsoft.FSharp.Core;
using Windows.Devices.Sensors;

Now, I can implement the converter as an extension

static class FuncToFSFuncConverter
{
    public static FSharpFunc<A, B> ToFastFunc<A, B>(this Func<A, B> f)
    {
        return FuncConvertExtensions.ToFSharpFunc(f);
    }
}

static class SensorSampleFuncs
{
    public static FSharpFunc<string[], int> LightMonitorSampleFunc()
    {
        LightSensor lightsens = LightSensor.GetDefault();
        var reading = (int)lightsens.GetCurrentReading().IlluminanceInLux;
        return (new Func<string[], int>(p => reading)).ToFastFunc();
    }
}

...which is going to give me a SampleFunc that returns the current reading (in Lux), from the Light sensor.

The Monitor Tree DU

The MonitorTree type is delcared as a recursive Discriminated Union type:

type MonitorTree =
    |And of String * MonitorTree * MonitorTree 
    |Or  of String * MonitorTree * MonitorTree 
    |Not of String * MonitorTree
    |Tip of Monitor
    member self.isAlarming = match self with
                             |And(_,l,r) -> l.isAlarming && r.isAlarming
                             |Or (_,l,r) -> l.isAlarming || r.isAlarming
                             |Not(_,l)   -> not (l.isAlarming)
                             |Tip(mon)   -> mon.IsOutOfThreshold
    member self.Name = match self with
                             |And(n,_,_)
                             |Or (n,_,_)
                             |Not(n,_)   -> n
                             |Tip(mon)   -> mon.Name

So, a MonitorTree can either be a 'Tip' (a leaf), of a single Monitor, or a tuple consisting of a String (for the Name) and two MontorTree branches, for And/Or types, or a tuple of String * MonitorTree for the Not type. The two other members use pattern matching to decompose the type, and return appropriate values based on this - for example, if the MonitorTree is an And type, the isAlarming member will be 'true' if and only if the isAlarming member of both left and right trees are 'true'.

The service reads the config from an XML file, starting up the list of monitors in parallel, each in an Async task.

When the SampleFunc function in the monitor returns the int result, the 'root' MonitorTree parent is serialilsed to JSON, and sent to the Azure service:

let serialiser = new JsonSerializer()
do serialiser.Converters.Add(new UnionTypeConverter())
do serialiser.Formatting <- Formatting.Indented
let toJSON obj = let wrtr = new StringWriter()
                 serialiser.Serialize(wrtr,obj)
                 wrtr.ToString()

The JSON.Net serialiser is extended with a bespoke Converter, that implements the serialisation of the discriminated union type - borrowed from Robert Pickering:

https://github.com/robertpi/FsRavenDbTools/blob/master/src/FsRavenDbTools/Converters.fs

The websocket client authenticates with the service using Basic Auth, and the username from the header is used in the server to label the data, such that the correct data can be published to the correct clients, and the confiugration data from the client can be sent to the correct service.

let sckt = new ClientWebSocket()
    let authheader = Convert.ToBase64String(Encoding.UTF8.GetBytes(un + ":" + pw))
    do sckt.Options.SetRequestHeader("Authorization","Basic " + authheader)
    let openSckt() =
        do sckt.ConnectAsync(new Uri("wss://server:1234/update"), CancellationToken.None) |> ignore
    do openSckt()
    let resp (json:string) =
        let buf = Encoding.UTF8.GetBytes(json)
        printfn "%s" (sckt.State.ToString())
        match sckt.State with
        |WebSocketState.Open -> do sckt.SendAsync(ArraySegment<byte>(buf),WebSocketMessageType.Text,false,CancellationToken.None) |> ignore
        |WebSocketState.Aborted
        |WebSocketState.CloseReceived
        |WebSocketState.Closed -> try 
                                    do sckt.ConnectAsync(new Uri("wss://server:1234/update"), CancellationToken.None) |> ignore
                                  with
                                  |_ -> printfn "Server sckt closed"
        |_ -> ()

The server

The server listens for incoming http requests, authenticates using the Authorization header, and if the request is a websocket request, passes the username and the websocket over to one of two functions implemented in a MailBoxProcessor, based on the URL (/update or /subscr).

let server = async { 
    use listener = new HttpListener()
    listener.Prefixes.Add(url)
    listener.AuthenticationSchemes <- AuthenticationSchemes.Basic
    listener.Start()
    while true do 
      let! context = listener.AsyncGetContext()
      let isWS = context.Request.IsWebSocketRequest
      let path   = context.Request.Url.LocalPath
      match isWS with
      |false -> {serve static files etc.} ..
      |true -> let! wsctxt = context.AsyncAcceptWebSocket()
               let ws = wsctxt.WebSocket
               let auth = context.Request.Headers.["Authorization"].Split([|' '|]).[1]
               let usrpwd =  Encoding.UTF8.GetString(Convert.FromBase64String(auth)).Split([|':'|])
               let usr,pwd = usrpwd.[0],usrpwd.[1]
               match authenticate(usr,pwd) with
               |true ->match path with
                        |"/update" -> agent.Post(Publish(usr,ws))
                        |"/subscr" -> agent.Post(Subscribe(usr,ws))
                        |_ -> failwith "unknow URL"
               |false->failwith "Did not authenticate!"
      }

 The MailBoxProcessor agent keeps 2 * Dictionary<string,webscoket>'s internally (one for services, one for clients), so that it can match an incoming update from the service, to a connected 'client', and publish the data to the correct connected client. And vice versa, matching up incoming config data from clients to the correct connected service.

The Client

The client is implemented as a WPF app, in a combination of F# and C#. The code that communicates with the websocket server, and deserialises the data, is implemented in F#. When a message arrives, and is deserialised, an event is raised that the C# XPF app subscirbes to, such that the UI can be updated with the new data. 

The interface of the client will implement some touch interaction, to Windows-8 it up a bit ...