Event-Based Asynchronous WebRequest

• Download source files - 24.2 Kb
• Download demo project - 42.1 Kb

Introduction

This code sample is intended to be used as a reference for handling the WebRequest and WebResponse objects in the System.Net namespace as well as how to implement the Event-Based Asynchronous Pattern as defined on the MSDN website.

Working the WebRequest and WebResponse

Using the WebRequest object couldn't be simpler. The following code snippet will create a WebRequest for www.google.com and get a response.

Dim request As WebRequest = WebRequest.Create("http://www.google.com")
Dim response As WebResponse = request.GetResponse()

Processing the WebResponse is a little trickier. First, you have to read the response from a stream, and then, based on the WebResponse.ContentType, you need to figure out how to turn that into the desired output. The following function will take a WebResponse and read the result into a String, Image, or Byte array.

Public Shared Function ProcessResponseStream( _
    ByVal response As WebResponse, _
    Optional ByVal defaultCharset As String = "utf-8") _
    As Object

    Dim st As IO.Stream
    st = response.GetResponseStream()

    Dim mem As New IO.MemoryStream()
    Dim buffer(1024) As Byte
    Dim read As Integer = 0

    Try
        ' Read the response stream into memory
        Do
            read = st.Read(buffer, 0, buffer.Length)
            mem.Write(buffer, 0, read)
        Loop While read > 0
        st.Close()
        response.Close()

        ' Reset the memory position so we can read
        ' from the stream.
        mem.Position = 0

        ' Parse the content type (ContentType is an
        ' internal class).
        Dim contentType As ContentType
        contentType = _
            New ContentType(response.ContentType)

        Select Case contentType.Type
            Case "text"
                ' we should be able to read any text
                ' content into a string (assuming we
                ' have the correct encoding type).
                Dim result(CInt(mem.Length)) As Byte

                mem.Read(result, 0, CInt(mem.Length))

                ' We need to get the appropriate
                ' charset in order to decode the
                ' byte array as a string.
                ' This information can be sent
                ' in the Content-Type. If it isn't
                ' we need to use the default
                ' charset.
                Dim charset As String
                charset = contentType.GetValue( _
                                "charset", _
                                defaultCharset)

                ' We have the charset, now get the
                ' Encoding object and decode the
                ' content into a string.
                Dim enc As Encoding
                enc = Encoding.GetEncoding(charset)
                Return enc.GetString(result)
            Case "image"
                ' We should be able to read most image
                ' types directly into an image.
                Return Image.FromStream(mem)
            Case Else
                ' Let the caller figure out how to
                ' handle this content.
                Dim result(CInt(mem.Length)) As Byte
                mem.Read(result, 0, CInt(mem.Length))
                Return result
        End Select
    Finally
        mem.Close()
    End Try
End Function

If a web resource requires authentication, you can provide it using the WebRequest.Credentials property. Simply set it to a System.Net.NetworkCredential object. This allows you to send the user's name, password, and domain to the server to provide the user access to the web content.

If you want to store the user's credentials between sessions, you need to make sure the data is secure. If you want to learn more about this, look into the DataProtection API in .NET. You can read more about it in my blog post on Protecting Data in .NET.

Event-Based Asynchronous Pattern

This is the same pattern that the BackgroundWorker uses to provide asynchronous processing with events being raised on the calling thread. This is a very useful pattern, and the .NET threading libraries make it fairly easy to implement. The MSDN documentation is better than anything I can come up with, so I encourage you to read it (I won't be discussing any of the fine print here). 

The key to this pattern is the System.ComponentModel.AsyncOperation class. This is the class that allows you to make cross thread calls. Basically, you instantiate the class on the main thread, pass it to a new thread, and then call AsyncOperation.Post(AddressOf MyMethod, MyArg). AsyncOperation will then use Windows messaging to call your method on the calling thread (you will want to review the documentation for the fine print on this :). When you are done with the thread, call AsyncOperation.PostOperationCompleted(AddressOf MyMethod, MyArg).

If you're interested in how I discovered this pattern, you can read my blog post on The Holy Grail of .NET Threading.

BackgroundWebRequest Component

The NetLib project (included for download in this article), uses the Event-Based Asynchronous Pattern to provide a multi-threaded solution to downloading web content. It can be dropped onto a WinForm to be used like the BackgroundWorker component. Simply set a couple of properties, handle a couple of events, and call the BackgroundWebRequest.GetResponseAsync method. This method will return an object that acts as the key to the request. All of the events and methods specific to a particular request use this object to identify the request (it's just a System.Object).

The following code sample shows how the sample project uses the request key to process the BackgroundWebRequest.GetRequestCompleted event (note, the SetStatus method simply sets a label to the status of the request).

Private mRequestKey As Object
Private Sub txtAddress_Validated( _
    ByVal sender As System.Object, _
    ByVal e As System.EventArgs) _
    Handles txtAddress.Validated

    ' make sure we cancel the last request
    wrcBrowser.CancelRequest(mRequestKey)

    If txtAddress.Text = "" Then
        mRequestKey = Nothing
        browser.DocumentText = ""
    Else
        ' GetResponseAsyc returns an object that is
        ' used to identify the specific web request.
        ' The value must be stored in order to
        ' perform operations on it (such as cancelling)
        mRequestKey = wrcBrowser.GetResponseAsync( _
                                    txtAddress.Text)
    End If
End Sub
Private Sub wrcBrowser_GetRequestCompleted( _
    ByVal sender As System.Object, _
    ByVal e As GetRequestCompletedEventArgs) _
    Handles wrcBrowser.GetRequestCompleted

    ' we only care about the last request that we made
    If e.RequestKey IsNot mRequestKey Then Return
    ' This method simply displays the status in the
    ' status bar.
    SetStatus(e.Status, e.Error)

    If e.Status = WebRequestStatus.Complete Then
        ' The request completed successfully, dump the
        ' html into the web browser.
        browser.DocumentText = CStr(e.ResponseResults)
        lblAddress.Text = e.Uri
    End If
End Sub

A couple of notes about the sample code. The code has not gone through any sort of rigorous testing. At this point, it is simply a sample application. There are many different error conditions that I am not handling at this time that should be for production code (some of the omissions are intentional, others are not, you'll have to guess which ones are which :). Use the code at your own risk and if you find any major issues, please let me know.