Don't break the screen scraping after changing the layout of the website

Question

0.00/5 (No votes)

See more:

Hi,

I am using the C#.net for screen scrapping. I am able to write the screen scrap program for scrapping information on website. But if the layout of the website will be changed, the screen scrap program will not longer be work. (the scraping will be break'd after changing the layout of the website).

So, can anyone tell me what is the solution of this problem?

Thank You
Harish Kumar

Posted 8-Apr-11 3:09am

Harish Kumar Bansal

Add a Solution

2 solutions

Solution 2

@John: okay! but if the layout of the website will be changed, can it is possible to receive an automatically alert shows that the website will be changed.

Posted 8-Apr-11 3:29am

Harish Kumar Bansal

Comments

#realJSOP 8-Apr-11 9:58am

Use the Add Comment link under an answer. I moved this for you, so refer to my original answer for a response.

Add a Solution

Add your solution here

Treat my content as plain text, not as HTML

Preview 0

…

Existing Members

Sign in to your account

...or Join us

Download, Vote, Comment, Publish.

Your Email
Password
Forgot your password?

Your Email
This email is in use. Do you need your password?
Optional Password

I have read and agree to the Terms of Service and Privacy Policy
Please subscribe me to the CodeProject newsletters

When answering a question please:

Read the question carefully.
Understand that English isn't everyone's first language so be lenient of bad spelling and grammar.
If a question is poorly phrased then either ask for clarification, ignore it, or edit the question and fix the problem. Insults are not welcome.
Don't tell someone to read the manual. Chances are they have and don't get it. Provide an answer or move on to the next question.

Let's work to help developers, not make them feel stupid.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

#realJSOP · Accepted Answer · 2011-04-08T03:22:00

Solution 1

There is no way to prevent this from happening. You can mitigate it by using IDs (if they're assigned), but even that could change at the whim of the web site's coder. In essence, you're screwed and there's nothing you can do about it excpet try to keep up with the changes and hope they don't happen too often.

Posted 8-Apr-11 3:22am

#realJSOP

Updated 8-Apr-11 3:23am

v2

Comments

#realJSOP 8-Apr-11 9:51am

From OP: okay! but if the layout of the website will be changed, can it is possible to receive an automatically alert shows that the website will be changed.

#realJSOP 8-Apr-11 9:58am

Sure - just wait fro something unexpected to happen and handle it appropriately in your code. Unfortunately, the only way you can really do this is by actually scrapinhg the page and trying to parse it with your existing code. No matter what you do, you're going to have to change your code to make it work after a change to the site. Your best strategy is to put your scraping code into a DLL so that all you have to do is change/update that.