Re: Need to capture data of currently displayed webpage - ASP.NET Discussion Boards

Antone Eason8-Jul-09 4:07

8-Jul-09 4:07

Not clear why it is rude??? I am asking different forums and those that do not necessary see my question from other forums. So, if my question maybe answered in Perl and not in C# then the Perl guys will never see my question.

If this is not correct let me know. I have not been on here in many years.

Thanks,

Antone

Re: Ignore report

N a v a n e e t h8-Jul-09 4:45

N a v a n e e t h

8-Jul-09 4:45

Antone Eason wrote:
Not clear why it is rude??? I am asking different forums and those that do not necessary see my question from other forums

Well, in such cases, you can post the question in most appropriate forum and link to it while posting in the other forums. Putting the same text in all places is considered to be rude.

To answer your question,

You have said the page you want to download requires authentication and some word art. Do you mean a CAPTCHA[^] ? If yes, getting the authentication pass is tough as you can't predict the value of CAPTCHA.

Smile | :)

Navaneeth

How to use google | Ask smart questions

Re: Ignore report

Antone Eason8-Jul-09 4:52

Antone Eason

8-Jul-09 4:52

OK thanks I will look into linking for future posts. As far as my question I do not need to authenticate or worry about login credentials. I am already in fine, the page just changes every few minutes.

Think of it as simply being able to click the [view source] button on any web page that happens to be on the browser and then the source e.g. raw HTML is written to a text file.

That is what I need a script or program to be the view source button that I can invoke inside a batch file that is called by the scheduler every 5 minutes or so to check for the latest update.

Thanks,

Antone

Re: Ignore report

N a v a n e e t h8-Jul-09 17:21

N a v a n e e t h

8-Jul-09 17:21

Look at the WebClient class and work with DownloadString[^] method. It does exactly what you are looking for. Try to follow the below steps

1 - Create a console application which quits automatically once the job is done.
2 - Inside this, use WebClient class and request to the web page. This will give you raw HTML.
3 - Schedule this exe to run every 5 minutes for getting the latest updates.

Hope that helps Smile | :)

Navaneeth

How to use google | Ask smart questions

Re: Ignore report

Antone Eason8-Jul-09 18:46

Antone Eason

8-Jul-09 18:46

Hi - yes that sounds like an excellent solution. I will try just a little short on time.

Thanks again!

Antone

Re: Ignore report

Antone Eason8-Jul-09 9:44

Antone Eason

8-Jul-09 9:44

Hi - Yes that is security used. Thanks for your input and quick response.

Thanks,

Antone

Re: Need to capture data of currently displayed webpage

PJWindsor8-Jul-09 4:17

PJWindsor

8-Jul-09 4:17

Hi, have you tried Fiddler its a HTTP Debugging tool and logs whats going on as your are browsing, you can see html, js files downloaded and alot more, might help.

If that doesnt do the trick then look at the WebRequest object this can be used to get the HTTP response, as you said there is a log screen it maybe harder, but you can do your own POST by changing the username, password, and wordart answer into a Byte array and then sending this with your request to whatever page the form points too.

try here http://msdn.microsoft.com/en-us/library/system.net.webrequest.aspx[^]

Phil

Re: Need to capture data of currently displayed webpage

Antone Eason8-Jul-09 4:27

Antone Eason

8-Jul-09 4:27

Hi Phil,

Thanks for the quick reply! I will take a look at fiddler. I am an assembly language programmer mainly, so I do not know so much about script languages. If fiddler can write the webpage to a file then it should work!

Please understand the login is not an issue anymore I can login fine and leave the opera browser on with refresh set to 30 seconds. Then as the page refreshes with new data I want to write it to a file. Next my program runs and check the raw data to see if what I am looking for is there, if it is then the bell will ring, if not then it ends and waits for the next html file to check.

Thanks,

Antone

Re: Need to capture data of currently displayed webpage

Antone Eason8-Jul-09 4:32

Antone Eason

8-Jul-09 4:32

Hi Phil,

Sorry Fiddler is a no go. It does not write data to a file.

Thanks,

Antone

Re: Need to capture data of currently displayed webpage

PJWindsor8-Jul-09 5:03

PJWindsor

8-Jul-09 5:03

How about if you use the WebBrowser Control on a windows form? Im not sure but it think you should beable to browse to the page you want and then write some code to get the HTML of the current page on the WebBrowser control.

If that doesnt work then you can look at using Automating IE directly i forget the dll name its something like SHDocVW and this contains the Interfaces for automating IE Directly.

From here you can get the current document.

Re: Need to capture data of currently displayed webpage

Antone Eason8-Jul-09 5:10

Antone Eason

8-Jul-09 5:10

Thanks for the input, but I do not believe I have the knowledge to do that. I was hoping that this has been done before. The IE automation sounds interesting.

Thanks,

Antone

Re: Need to capture data of currently displayed webpage

PJWindsor8-Jul-09 5:20

PJWindsor

8-Jul-09 5:20

If you want it learn its Called Platform Invoke in .NET its all about using the existing Windows dll's from .NET

here is a good artical to get started its shows you how to use some of the functions in the User32.dll

[PInoke on CodeProject]

this website list alot of the windows functions with code to automate them

http://www.pinvoke.net/[www.pinvoke.net]

Phil

Re: Need to capture data of currently displayed webpage

Antone Eason8-Jul-09 9:39

Antone Eason

8-Jul-09 9:39

Hi Phil,

Thanks for all your assistnce. I really though this would have been done in the past. I will take a look at all the info you sent to me.

Still waiting to hear back from the Perl guys, I really thought you can do this in Perl.

Thanks,

Antone

Re: Need to capture data of currently displayed webpage

PJWindsor8-Jul-09 5:47

PJWindsor

8-Jul-09 5:47

It is possible to get the HTML From a WebBrowser Control see below code sorry if you dont know C#

I just put a textbox on the page to enter the url, a button to navigate to the url(btnLoadWebsite_Click handles this button) and a button to get the HTML from the currently displayed page (btnGetHTML_Click handles this).

private void btnLoadWebsite_Click(object sender, EventArgs e)
        {           
            try
            {
                Uri oURL = new Uri(txtURL.Text); //Get the Url from the text box must be format http://www.google.com
                wbMain.Navigate(oURL); //Navigate to the Url
            }

            catch (Exception err)
            {
                MessageBox.Show(err.ToString());
            }
        }


        private void btnGetHtml_Click(object sender, EventArgs e)
        {
            MessageBox.Show(wbMain.DocumentText); //Display the HTML
        }

The important bit is wbMain.DocumentText. wbMain is the WebBrowser Control and DocumentText is the property that holds the current html page text.

Phil

Calling a window from an ASP Web Page

ahmedmahiuddin8-Jul-09 2:51

ahmedmahiuddin

8-Jul-09 2:51

I want to call an window from an ASP Web Page by clicking an ASP:Button
and the code will be run at both server and client side.

Can anybody guide me please?

Re: Calling a window from an ASP Web Page

Manas Bhardwaj8-Jul-09 2:59

Manas Bhardwaj

8-Jul-09 2:59

Member 4056066 wrote:
I want to call an window from an ASP Web Page by clicking an ASP:Button

What kind of window? Confused | :confused:

Manas Bhardwaj
Please remember to rate helpful or unhelpful answers, it lets us and people reading the forums know if our answers are any good.

Re: Calling a window from an ASP Web Page [modified]

ahmedmahiuddin8-Jul-09 3:05

ahmedmahiuddin

8-Jul-09 3:05

just another asp page in the window

modified on Wednesday, July 8, 2009 9:30 AM

Re: Calling a window from an ASP Web Page

Manas Bhardwaj8-Jul-09 3:14

Manas Bhardwaj

8-Jul-09 3:14

What you need is this[^] Rose | [Rose]

Manas Bhardwaj
Please remember to rate helpful or unhelpful answers, it lets us and people reading the forums know if our answers are any good.

Re: Calling a window from an ASP Web Page

ahmedmahiuddin8-Jul-09 3:42

ahmedmahiuddin

8-Jul-09 3:42

Yees !
I found some very good way.

and thanks for the search link

Very small problem - but i had stucked here

Satish - Developer8-Jul-09 1:47

Satish - Developer

8-Jul-09 1:47

Hi friends,

Its very silly problem. But still it scares me.

int _Totalcnt = 62;
int _Namecnt = 31;

decimal Pctmatch = Convert.ToDecimal((_Namecnt / _Totalcnt) * 100);

I am getting value for Pctmatch as '0'. why it showing wrong. How can i get exact decimal value there?

G. Satish

Re: Very small problem - but i had stucked here

DoctorMick8-Jul-09 2:04

DoctorMick

8-Jul-09 2:04

Convert _Namecnt and _Totalcnt to decimals rather than the result of the calculation, i.e.

decimal Pctmatch = (Convert.ToDecimal(_Namecnt) / Convert.ToDecimal(_Totalcnt)) * 100;

Re: Very small problem - but i had stucked here

Spirits Soft Tech Nology , Dharmapuri8-Jul-09 2:06

Spirits Soft Tech Nology , Dharmapuri

8-Jul-09 2:06

www.spiritssoft.com

Re: Very small problem - but i had stucked here

Manas Bhardwaj8-Jul-09 2:19

Manas Bhardwaj

8-Jul-09 2:19

Do not spam Mad | :mad:

Manas Bhardwaj
Please remember to rate helpful or unhelpful answers, it lets us and people reading the forums know if our answers are any good.

Re: Very small problem - but i had stucked here

Abhijit Jana8-Jul-09 2:25

Abhijit Jana

8-Jul-09 2:25

Shivass wrote:
www.spiritssoft.com

Please Don't SPAM. This is second time you are doing the same thing. Dead | X|

cheers,
Abhijit
CodeProject MVP
Web Site:abhijitjana.net
View My Recent Article

Re: Very small problem - but i had stucked here

Christian Graus8-Jul-09 2:27

Christian Graus

8-Jul-09 2:27

you are a retard

Christian Graus

Driven to the arms of OSX by Vista.

Read my blog to find out how I've worked around bugs in Microsoft tools and frameworks.

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.