Re: Accessing the HTML in WebBrowser difficulty - C# Discussion Boards

Accessing the HTML in WebBrowser difficulty

Michael Potter22-Jul-09 3:41

22-Jul-09 3:41

I am trying to grab the HTML source of a web page using the WebBrowser control. The page in question allows the user to query out a specific record (or set of records) from a database. Once these records have been listed, the user clicks on the desired title and javascript (or AJAX) overlays the current query page with the desired result display.

Problem:
If I try to programmatically grab the source, I get the original query page, not the overlayed desired result object. I can right click on the result and view source correctly but I can't seem to get it via code.

Anyone out there solved this issue in the past?

Re: Accessing the HTML in WebBrowser difficulty

Muhammad Mazhar22-Jul-09 4:25

Muhammad Mazhar

22-Jul-09 4:25

I have tried some tricks of querying a webpage via code and outputting the result. Check following topic on my blog may be it helps you.
http://shareyour-experience.blogspot.com/2009/06/find-geo-location-through-ip-address.html[^]

Share your experience with others
Check my Blog...

Re: Accessing the HTML in WebBrowser difficulty

led mike22-Jul-09 4:31

led mike

22-Jul-09 4:31

Michael Potter wrote:
I am trying to grab the HTML source of a web page using the WebBrowser control.

I don't know what your requirements are but making an HTTP Request will get you the HTML code. It's far simpler than using a WebBrowser Control. You can use many different Base Class items to do this, one is the HttpWebRequest Class[^]

Re: Accessing the HTML in WebBrowser difficulty

Michael Potter22-Jul-09 5:08

Michael Potter

22-Jul-09 5:08

Thanks for the response.

I can't hide the functionality of the website I wish to scrape. I need its query interface to function as designed. I just can't get to the result source HTML. I am guessing it is inserted somewhere in the DOM but, I failed to locate it.

Essentially, a small square 'frame' appears (via java script) in the center if the page. If I right click on the small square 'frame' and choose [view source], I get what I want. If I right click OFF the small square 'frame' and choose [view source], I get the intial query HTML. I can't find the small square 'frame's HTML programically.

Re: Accessing the HTML in WebBrowser difficulty

led mike22-Jul-09 5:55

led mike

22-Jul-09 5:55

Michael Potter wrote:
I can't hide the functionality of the website I wish to scrape.

Not sure what that means but if you must use a WebBrowser Control you could still use the URL from the control to make separate HTTP Requests to obtain the HTML. If you are trying to capture the dynamic changes to the DOM from any client side script then of course that will not help you.

Michael Potter wrote:
I am guessing it is inserted somewhere in the DOM

Yes the DOM is the in memory version of the HTML. Again if you want the original stream from the server then just make a HTTP Request. If you need the dynamic HTML you will have to use the DOM. You will have to dig through the DOM documentation to find the parts you need. The basic concept is that each Frame has a Body and a Body element might give you access to the Inner HTML as Text.

Re: Accessing the HTML in WebBrowser difficulty

Adam R Harris22-Jul-09 6:14

Adam R Harris

22-Jul-09 6:14

Is the "frame" an iFrame? If it is that would explain your problem. An iFrame hold it's contents in it's own innerHTMl property so it wouldn't come back from the webbrowsers.Document.InnerHTML.

If at first you don't succeed ... post it on The Code Project and Pray.

Re: Accessing the HTML in WebBrowser difficulty

Michael Potter22-Jul-09 10:01

Michael Potter

22-Jul-09 10:01

After some javascript research - yes it is an IFrame.

I was able to capture the navigated URL and use HttpWebRequest (thanks led mike) to re-grab the IFrame when it is unsecured. I am unable to do so when it is secured data. I can't seem to hitch onto the rights the WebBrowser object has negotiated and I don't know how to negotiate a new set (I am not privy to the sites inner workings).

So the problem remains but, is better defined. How do I read an IFrame's source from the WebBrowser control?

Re: Accessing the HTML in WebBrowser difficulty

Adam R Harris22-Jul-09 10:12

Adam R Harris

22-Jul-09 10:12

What I would do, I'm sure there is a better way, is just append a JavaScript function and a hidden textbox to the innerHTML of the loaded document.
then call InvokeScript on the webbrowser to run your JavaScript (which should set the hidden textboxs text to the inner HTML of the iframe) then get the text from the textbox by getting the innerhtml and parsing out the textbox value.

Like I said I'm sure there is a better way.

If at first you don't succeed ... post it on The Code Project and Pray.

Re: Accessing the HTML in WebBrowser difficulty

Michael Potter23-Jul-09 3:45

Michael Potter

23-Jul-09 3:45

Any idea on what the script would look like? I have not done a lot of web programming.

Re: Accessing the HTML in WebBrowser difficulty - SOLVED [modified]

Michael Potter23-Jul-09 9:23

Michael Potter

23-Jul-09 9:23

Found this on the net that allowed me to use HttpWebRequest (as suggested earlier).

http://mmarinov.blogspot.com/2007/10/using-exsiting-ie-cookies-with.html[^]

Thanks for all those that helped - refining the definition of the problem was very helpful.

Special Note: The WPF WebBrowser control doesn't even fire the events (IFrame navigation) necessary for the above solution. I have to use the Windows Forms version.

modified on Friday, July 24, 2009 2:07 PM

Will C# support primitive data type in Generics

Gopal_Kanchana22-Jul-09 3:28

Gopal_Kanchana

22-Jul-09 3:28

Public class MyClass<t> where T:int
{

}
why it's giving error. why .net framework not supporting

Re: Will C# support primitive data type in Generics

DaveyM6922-Jul-09 3:31

DaveyM69

22-Jul-09 3:31

You can use struct

public class MyClass<T> where T : struct
{

}

or wrap an int in a (immutable?) class of your own and use that for the constraint.

Dave
BTW, in software, hope and pray is not a viable strategy. (Luc Pattyn)
Visual Basic is not used by normal people so we're not covering it here. (Uncyclopedia)
Why are you using VB6? Do you hate yourself? (Christian Graus)

Re: Will C# support primitive data type in Generics

Gopal_Kanchana22-Jul-09 3:34

Gopal_Kanchana

22-Jul-09 3:34

why this restriction.
Allowing struct. what problem allowing int?

Re: Will C# support primitive data type in Generics

DaveyM6922-Jul-09 3:44

DaveyM69

22-Jul-09 3:44

You'd have to ask Anders Hejlsberg[^]. It's the way generics were built into C#. Search for generic type constraints and you might find some reasoning.
MSDN[^]

Re: Will C# support primitive data type in Generics

stancrm22-Jul-09 3:43

stancrm

22-Jul-09 3:43

public class MyClass<T>
{
  ...
}

MyClass<int> myClass = new MyClass<int>();

is this what you need ?

Re: Will C# support primitive data type in Generics

Gopal_Kanchana22-Jul-09 19:25

Gopal_Kanchana

22-Jul-09 19:25

public class CplusEnumExtension<t, f=""> : whehre T:int
{

public CplusEnumExtension(int t)
{
enumT = t;
}
public bool Has(F findvalue)
{
return ((int)(object)enumT & (int)(object)findvalue)>0;
}

private int enumT=0;
}

In above code Has method i comparing int type that's why i expecting int from client side.
Apartment from this
I have implemented code below in C++ which client can use any type.(int, long etc) Is it possible in C#

//EnumT->int,long,Ulong...0x0001;
//Findvalue->ePDVE_DllError
//usage : CEnumerationExtensions<ULONGLONG,ePileplandesign_verificationError> obj(errorcode);
template <typename enumt,typename="" findvalue="">;
class CEnumerationExtensions
{
public:
CEnumerationExtensions(EnumT e)
: enum_(e)
{}

public:
bool Has(Findvalue value)
{
return ((enum_ & static_cast<enumt>(value))>0);
}
private:
EnumT enum_;
};

Re: Will C# support primitive data type in Generics

Daniel Grunwald22-Jul-09 4:12

Daniel Grunwald

22-Jul-09 4:12

What types do you expect you should be able to use with "int" as a constraint?

You would only be able to use a single type - "int" itself (no uint, long etc. - those don't derive from int).
But if there's only one possible type argument, it doesn't make sense to use generics at all!

Re: Will C# support primitive data type in Generics

Luc Pattyn22-Jul-09 4:22

Luc Pattyn

22-Jul-09 4:22

Hi,

you might want to enumerate the acceptable types and use common features:
public class MyClass<T> where T : int, long, float, double and now perform addition.

Smile | :)

Luc Pattyn [Forum Guidelines] [My Articles]

The quality and detail of your question reflects on the effectiveness of the help you are likely to get.
Show formatted code inside PRE tags, and give clear symptoms when describing a problem.

Re: Will C# support primitive data type in Generics

Daniel Grunwald22-Jul-09 4:41

Daniel Grunwald

22-Jul-09 4:41

But multiple constraints mean the type T must implement all of those.
There is no way to say "int OR float" (which would have to automatically generate an interface with all methods common between int and float).
But unfortunately, .NET doesn't support 'dynamic interfaces' (you declare an interface, and all classes having the appropriate methods will automatically implement it). Actually, I think VB10 will have something dynamic interfaces, but I guess it'll be implemented on top of Reflection/the DLR.

Re: Will C# support primitive data type in Generics

Luc Pattyn22-Jul-09 4:51

Luc Pattyn

22-Jul-09 4:51

Thanks.
Syntax proposal: public class MyClass<T> where T in { int, long, float, double }

Smile | :)

Luc Pattyn [Forum Guidelines] [My Articles]

The quality and detail of your question reflects on the effectiveness of the help you are likely to get.
Show formatted code inside PRE tags, and give clear symptoms when describing a problem.

Re: Will C# support primitive data type in Generics

PIEBALDconsult22-Jul-09 5:11

PIEBALDconsult

22-Jul-09 5:11

Right, that would be good. Or the numeric types should implement some sort of IDoMath interface and we could use that.

I also want an enum constriant.

Re: Will C# support primitive data type in Generics

DaveyM6922-Jul-09 5:28

DaveyM69

22-Jul-09 5:28

If they made it so we had a Numeric constraint, or they implemented some common interface (INumeric), that would fix it for all the integral data types and enums too in one go.

Re: Will C# support primitive data type in Generics

PIEBALDconsult22-Jul-09 6:19

PIEBALDconsult

22-Jul-09 6:19

I want enums separate from the other types. I have generic classes that should accept enums only. As it stands, I have to check at runtime and throw and Exception. Dead | X|

SplitContainer problem

lune1222-Jul-09 3:21

lune12

22-Jul-09 3:21

Hi,

I'm using the control SplitContainer on run time.
uppon request I'm generating a tab that contains SplitContainers.
I set all the properties as needed including the SplitterDistance,
BUT when the application is running the spliter is not positioned as expected.
it seems that it use some default parameter that ovveride the explicit setting.

Thanks for your help,
lune

Re: SplitContainer problem

Muhammad Mazhar22-Jul-09 3:55

Muhammad Mazhar

22-Jul-09 3:55

Its hard to comment about it but a wild guess could be to check your dock and anchor properties.

Share your experience with others
Check my Blog...

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.