Click here to Skip to main content
15,867,453 members
Articles / Web Development / XHTML

HTTP GET with .NET WebClient

Rate me:
Please Sign up or sign in to vote.
4.17/5 (13 votes)
2 Mar 2009CPOL2 min read 251.4K   8.2K   37   17
Methods for performing HTTP GETs in C# using WebClient and StreamReader.

Introduction

One of the joys of developing with .NET is, a significant amount of the ground work which we previously had to code ourselves is now part of the framework. In this article, I show methods for performing HTTP GETs in C# using the WebClient and the StreamReader. I'll use these methods in future articles.

First, let's introduce Stream and StreamReader, which can both be found in the System.IO namespace. The StreamReader implements a TextReader to read UTF-8 characters by default from a stream (the source), which makes it ideal for reading from a URI. Take note that StreamReader is different from Stream, which reads bytes.

For sending data to and from a URI, .NET provides the WebClient class which can be found in the System.Net namespace. Several methods are available to enable us to send and receive files and data, both synchronously and asynchronously. The method we are interested in here is OpenRead(URI), which returns data from the URI as a Stream.

Let's code

The basic code to read from our URI can be achieved in three lines. We create our WebClient instance, create a Stream from the WebClient, and then read into a StreamReader until the end of the file, like so:

C#
using System.IO;
using System.Net;

String URI = "http://somesite.com/somepage.html";

WebClient webClient = new WebClient();
Stream stream = webClient.OpenRead(URI);
String request = reader.ReadToEnd();

After we have run over this code, the contents of somepage.html will be in the request string variable. This is all great, but we are presuming that the request here is faultless.. i.e., no exceptions are thrown. With exception handling being so easy in .NET, there's no excuse not to make benefit of it... although from experience, it seems not everyone is of the same opinion...

Let's wrap our Stream requests into a try-catch loop. We can catch a WebException to clearly identify what has gone wrong, and deal with it nicely.

C#
try
{
    WebClient webClient = new WebClient();
    Stream stream = webClient.OpenRead(URI);
    String request = reader.ReadToEnd();
}
catch (WebException ex)
{
    if (ex.Response is HttpWebResponse)
    {
        switch (((HttpWebResponse)ex.Response).StatusCode)
        {
            case HttpStatusCode.NotFound:
                response = null;
                break;

            default:
                throw ex;
        }
    }
}

We can further optimize the code by the wrapping using(...) around the WebClient/Stream, but that's beyond the scope of this article.

Authentication

If you have a URI which requires authentication, you can add a NetworkCredential to the WebClient reference before you call the OpenRead method, like so:

C#
WebClient webClient = new WebClient();
webClient.Credentials = new NetworkCredential(username, password);
Stream stream = webClient.OpenRead(URI);

A real world example of using the above would be retrieving a list of your latest Tweets from Twitter. You need to pass your username and password to be able to get to the feed. The example download uses this as a demonstration, so you will need to add your own Twitter username and password.

History

No changes.

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)


Written By
Software Developer (Senior)
United Kingdom United Kingdom
I'm a ASP.NET developer based in Brighton, UK.

Comments and Discussions

 
QuestionI am getting login page data I think URL is redirecting to login page. Pin
Shirish@120-Mar-17 4:13
professionalShirish@120-Mar-17 4:13 
QuestionServerProtocolViolation Pin
kPlusPlusA15-Aug-14 21:33
kPlusPlusA15-Aug-14 21:33 
Questionyou have not defined reader in your code. Pin
random424-Jan-13 23:39
random424-Jan-13 23:39 
AnswerRe: you have not defined reader in your code. Pin
Gluups30-Sep-22 14:23
Gluups30-Sep-22 14:23 
QuestionHttp get Pin
Abirami-8721-Jan-13 19:46
Abirami-8721-Jan-13 19:46 
AnswerRe: Http get Pin
random424-Jan-13 23:52
random424-Jan-13 23:52 
QuestionMy working code 2 Pin
brentminder28-Nov-12 3:56
brentminder28-Nov-12 3:56 
AnswerRe: My working code 2 Pin
random424-Jan-13 23:51
random424-Jan-13 23:51 
AnswerRe: My working code 2 Pin
Gluups30-Sep-22 14:44
Gluups30-Sep-22 14:44 
QuestionEndofStream blocking Pin
Eldon Zacek14-Aug-12 10:41
Eldon Zacek14-Aug-12 10:41 
AnswerRe: EndofStream blocking Pin
Gluups30-Sep-22 14:47
Gluups30-Sep-22 14:47 
QuestionMy working code Pin
Herbert Yu6-Feb-12 10:15
Herbert Yu6-Feb-12 10:15 
AnswerRe: My working code Pin
Gluups30-Sep-22 14:37
Gluups30-Sep-22 14:37 
QuestionCompile error in code above Pin
Mark J2-Jan-12 20:08
Mark J2-Jan-12 20:08 
GeneralWebclient Not working ... Pin
bhalaniabhishek5-Sep-10 22:24
bhalaniabhishek5-Sep-10 22:24 
Generalhhtps does not work Pin
Member 75098514-Mar-10 21:26
Member 75098514-Mar-10 21:26 
GeneralHTTP Get.. Pin
sbarclay18-Jan-10 6:36
sbarclay18-Jan-10 6:36 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.