Retrieval of a webpage content...

Question

1.00/5 (1 vote)

See more:

HI friends
Im doing my final year project on Web page summarization.
I need an from u ppl... :( My project is all about summarizing the google page links and provide a short abstract of those links in the place of snippets below each link when we go for google search.
To start this ive to retrieve the contents of all those 10 links... For this i tried using web crawlers , html2txt s/w's but all ended in failure..
Please someone guide me to retrieve the contents in the web page links given by google search. Whe i used crawlers it retrieved all the contents.. like all the hyper links from that search result page.
Im talking about those 10 links alone which are returned by the google engine in return to our query... Please help me.. Still ive oly 3months to complete my project :(

[edit]Urgency deleted - OriginalGriff[/edit]

Posted 2-Jan-13 4:09am

Ponscedric

Updated 2-Jan-13 4:11am

OriginalGriff

v3

Add a Solution

Comments

OriginalGriff 2-Jan-13 10:12am

Urgency deleted: It may be urgent to you, but it isn't to us. All that your stressing the urgency does is to make us think you have left it too late, and want us to do it for you. This annoys some people, and can slow a response.

BTW: Multiple exclamation marks are a sign of a diseased mind...

Ponscedric 2-Jan-13 10:14am

thanks

1 solution

Add a Solution

Add your solution here

Treat my content as plain text, not as HTML

Preview 0

…

Existing Members

Sign in to your account

...or Join us

Download, Vote, Comment, Publish.

Your Email
Password
Forgot your password?

Your Email
This email is in use. Do you need your password?
Optional Password

I have read and agree to the Terms of Service and Privacy Policy
Please subscribe me to the CodeProject newsletters

When answering a question please:

Read the question carefully.
Understand that English isn't everyone's first language so be lenient of bad spelling and grammar.
If a question is poorly phrased then either ask for clarification, ignore it, or edit the question and fix the problem. Insults are not welcome.
Don't tell someone to read the manual. Chances are they have and don't get it. Provide an answer or move on to the next question.

Let's work to help developers, not make them feel stupid.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Adam R Harris · Answer 1 · 2013-01-02T04:22:00

Solution 1

Read over this: http://www.codersource.net/MicrosoftNet/CAdvanced/HTMLScreenScrapinginC.aspx[^]

It's a windows application and he is just getting the links on a page but it should get your headed in the right direction.

Posted 2-Jan-13 4:22am

Adam R Harris

Comments

Ponscedric 2-Jan-13 10:35am

but sir i want to retrieve just the contents of a link. Say if we use "data mining" as our search query in google then it ll return top 10 links related to our query . I just want to retrieve the contents of those 10 links alone. not unnecesary links :(

Adam R Harris 2-Jan-13 12:02pm

You are going to have to use Regular Expressions or something similar to parse the links out of the response. Then you are going to have to get that page content the same way you got it from Google.