|
I am writing an application that accesses a web page (the first page) with an httpget to retrieve the first page, issues an httppost with selection data and cookie from the first page httpget to retrieve a second page, then retrieves subsequent third pages with httpgets based on links in the second page. These third retrieves fail in the C# application.
If I use IE or Mozilla to make these third page retrieves, a proxy sniffer shows that on the third retrieve some additional data has been added to the cookie. The proxy sniffer also shows that some javascript routines have been downloaded on the first page retrieve and looking at the source from the second page retrieve it seems that it is one of these javascript routines that is adding the data to the httpget for the third page.
So in my C# application, what happens to the downloaded javascript routines and how do I access and run them in the same way my browsers do this?
Jerome K Jerome
|
|
|
|
|
The javascript files get downloaded the same was as everything else: data. What happens with the data is completely up to the client (your code).
Internet Explorer actually hosts the scripting engines (JScript, VBScript, and any other third-party scripts like PerlScript from ActiveState[^]). This doesn't happen just by downloading data (actually, nothing happens just by downloading data no matter what the client is - it's just a stream of bytes).
Hosting the scripting engines in such a way that you can simulate what the browser does is more complicated than you could probably imagine. Not only do you have to implement the interfaces that the browser does (practically all the interfaces defined by both mshtml.dll and shdocvw.dll, but many, many other COM interfaces (i.e., you can't just implement these interfaces: they don't exist in .NET), not to mention the Windows Script Host interfaces. Believe me, as part of the team that develops and sustains Windows Script I can assure you that's it's a complicated and very long job that you may not want to emark on! If you do, you must completely understand COM interop, P/Invoke, and data marshaling between unmanaged and managed code, and understand COM and OLE and how COM clients and servers interact through sites, threading models, and more.
Your best bet is to use inflection to determine what the script file would do by detecting certain patterns.
This posting is provided "AS IS" with no warranties, and confers no rights.
Software Design Engineer
Developer Division Sustained Engineering
Microsoft
[My Articles] [My Blog]
|
|
|
|
|
Well, you are right that I don't want to reinvent IE in my code nor am I proficient enough to do that. I don't think the problem is all that complicated and I don't really need the application to react as if it were a browswer. The proxy sniffer reports that the cookie has appended to it a string very much like this: "ForeseeLoyalty_MID_dh4QBcM8h8=55". On subsequent retreives the last two digits change.
A Google search on "FORESEELOYALTY" brings up very little but the javascript comments indicates it is being used for survey purposes and controlls a pop up. But using IE, I have never seen such a pop up.
It looks to me as though the string between "Foreseeloyalty_MID" and "=" is being passed in with the javascript and the last two digits are being computed. But I don't really know how to capture the javascript or how to append the new string to the cookie. The webRequest and webResponse only pass back to me the html, not the javascript and in the case of redirects, not even the pre-redirected html
As I see it, I at least have to capture the javascript so I can extract the token for the foreseeloyalty key.
|
|
|
|
|
If the javascript is being sent to you in your HttpWebResponse (your original description isn't very clear between steps 2 and 3), then you already have it. You don't need to parse it as javascript - just parse it as text. With the HttpWebResponse.GetResponseStream<?code> method, use the <code>HttpWebResponse.CharacterSet passed to Encoding.GetEncoding to get the proper Encoding class (I recommend using the UTF8Encoding as a fallback since it'll handle both ANSI and most Unicode, and is becoming more and more common on the web anyway).
Use Encoding.GetString after reading in all your bytes (so long as your return data isn't too big) and parse the resulting string.
If this isn't applicable, please explain steps 2 and 3 in a little more detail.
This posting is provided "AS IS" with no warranties, and confers no rights.
Software Design Engineer
Developer Division Sustained Engineering
Microsoft
[My Articles] [My Blog]
|
|
|
|
|
Below find the code for my web retrieves, adapted from Johnvey's Gmail Agent (Code Project).
What is returned after calling this the first time is simply the html of the first page and with it a cookie. On the second retrieve, I simply make a retrieve identical to the first one, but this time there is a cookie to send that I got from the first response. The result of the second call is also simply an html page. No javascript either time. (Perhaps this double retrieve is unneccesary, it works and I've never tried retrieving it only once. Have to put that on the todo list...)
I should also mention that this first/second page retrieve involves one redirect. I think the redirect is used to pass the cookie and possibly the javascript as well, so maybe the javascript is coming in as a response to the page I am not seeing because it is redirected.
private string MakeNewWebRequest(Uri location, string method, string referrer, string postData, bool allowAutoRedirect)
{
// prepare HTTP request
HttpWebRequest webRequest = (HttpWebRequest)WebRequest.Create(location);
webRequest.CookieContainer = new CookieContainer();
if (cookieJar!=null)
{
webRequest.CookieContainer.Add(webRequest.RequestUri, this.cookieJar.GetCookies(webRequest.RequestUri));
}
else
{
System.Windows.Forms.MessageBox.Show("No CookieJar!", "Debug");
}
// if POSTing, add request page and modify the headers
byte[] encodedPostData = new byte[0];
if(method == "POST")
{
ASCIIEncoding encoding = new ASCIIEncoding();
encodedPostData = encoding.GetBytes(postData);
webRequest.Method = "POST";
webRequest.ContentType="application/x-www-form-urlencoded";
webRequest.ContentLength = encodedPostData.Length;
}
else
{
webRequest.Method = "GET";
webRequest.ContentType = "text/html";
}
webRequest.Timeout=295000; // 295 seconds (4 min 55 sec) supports long downloads
webRequest.Headers["Accept-Language"] = "en-us";
webRequest.Headers["Accept-Charset"] = "ISO-8859-1,utf-8;q=0.7,*;q=0.7";
webRequest.ProtocolVersion = HttpVersion.Version11;
webRequest.AllowAutoRedirect = allowAutoRedirect;
webRequest.KeepAlive = true;
webRequest.Referer = referrer;
webRequest.UserAgent = "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7) Gecko/20040626 Firefox/0.8";
webRequest.Accept = "image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/vnd.ms-excel, application/vnd.ms-powerpoint, application/msword, */* ";
//sample cookie "Cookie:WEBTRENDS_ID=68.255.78.35-1098489905.786792::588AC2E5087CF3F2AD7B775FB2116AB2; ForeseeLoyalty_MID_dh4QBcM8h8=55"
//note: cookie may also be JSESSIONID
//Attempt to send request stream to server if POSTing
if(method == "POST")
{
Stream requestStream = null;
try
{
requestStream = webRequest.GetRequestStream();
requestStream.Write(encodedPostData, 0, encodedPostData.Length);
}
catch(Exception ex)
{
Debug.WriteLine(ex.Message);
}
finally
{
if(requestStream != null)
{
requestStream.Close();
}
}
}
// Attempt to get response from server
HttpWebResponse webResponse = null;
string output = "";
try
{
string
headers = "";
foreach(string key in webRequest.Headers.Keys)
{
headers += key + "=" + webRequest.Headers[key] + " | ";
}
Debug.WriteLine(" Request Headers: " + headers);
// get response
webResponse = (HttpWebResponse)webRequest.GetResponse();
webResponse.Cookies = webRequest.CookieContainer.GetCookies(webRequest.RequestUri);
this.cookieJar.Add(webResponse.Cookies);
string cooks="";
// Print the properties of each cookie.
foreach (Cookie cook in webResponse.Cookies)
{
cooks += "\n Name = " +cook.Name + "\n Value = " +cook.Value+"\n Domain = "+cook.Domain+"\n Path = "+cook.Path+"\n Port = "+cook.Port+"\n Secure = "+cook.Secure+"\n | ";
Debug.WriteLine(" Cookies: " + cooks);
//
foreach(Cookie ck in webResponse.Cookies)
{
Debug.WriteLine(" Adding cookie: " + ck.ToString());
}
// read response stream and dump to string
Stream streamResponse = webResponse.GetResponseStream();
StreamReader streamRead = new StreamReader(streamResponse);
output = streamRead.ReadToEnd();
streamRead.Close();
streamResponse.Close();
Debug.WriteLine("Received response (" + output.Length + " char(s))");
}
catch(Exception ex)
{
Debug.WriteLine(ex.Message);
System.Windows.Forms.MessageBox.Show(ex.Message, "Exception Caught");
}
finally
{
if(webResponse != null)
{
webResponse.Close();
}
}
// return the response document
return output;
}//end string MakeNewWebRequest
|
|
|
|
|
In my last post I mistakenly said I made this first retrieve twice to get the cookie. That is not true, I only make it once to get the cookie for the next retrive which is a post.
|
|
|
|
|
After some further testing, I see I can manually add the loyalty factor cookie but for some reason it never gets sent out on the get, only posts seem to send out the cookie.
|
|
|
|
|
Have you watched your traffic through a packet monitor? I find it odd - as I know you do - that your cookie isn't being sent with the GET request. Perhaps it is but the server doesn't handle the GET request and, thus, doesn't process the cookie.
If you think the cookie is coming with the HTTP redirect (again, a packet monitor really helps in these situations; you can get them all over on the web, but most will require a packet driver that, for some reason, the name seems to elude me right now - sorry) you should set HttpWebRequest.AllowAutoRedirect to false and handle it yourself. If the HTTP status code is 302, 303, or 305 (see the HttpStatusCode or on the web for the HTTP RFC or something) an exception will be thrown (which one, they don't say so give it a test and handle only that exception (other exceptions could be thrown for other reasons), which you'd handle and get the Location: header to send a request for the redirection location.
BTW, for cleaner code, you might consider using the using block statement instead of your own try-finally. It really has more to do with readability, but the effect is the same (compiles to a try-finally just as you're doing it). That's just my opinion, though. What you're doing is correct coding practice.
This posting is provided "AS IS" with no warranties, and confers no rights.
Software Design Engineer
Developer Division Sustained Engineering
Microsoft
[My Articles] [My Blog]
|
|
|
|
|
I think I finally got it to work. Thanks so much for your help. Now I have a new problem. I posted it as a new thread, but it seems to be so much in your ball park, I want to repeat it here.
The application follows several links in a web site, finally arriving at a link to a pdf file. I issue an httpget for this file and receive a response like the following:
HTTP Response 200 (OK) "APPLICATION/PDF" (72'407 bytes)
But what is returned from the streamRead.ReadToEnd is just an html page that says the pdf is downloading and that this page will be updated every seven seconds until the download is complete. How do I navigate my way around this so that I end up with the pdf document? Is this status page interleaved with the pdf document stream or are they separate?
|
|
|
|
|
I'll answer on the new thread.
This posting is provided "AS IS" with no warranties, and confers no rights.
Software Design Engineer
Developer Division Sustained Engineering
Microsoft
[My Articles] [My Blog]
|
|
|
|
|
Hi!
Now that I have asked a few single questions about my project, and I couldn't find a proper solution let me please explain the whole problem:
I would like to program an add-in that allows the user of vs to use comments similar to the word-comments.
I thougt it would be a good idea to show the comment in a tool-tip but this is not as simple as it seems. The problem is, that the textselection (that should be commented) should become an persistent object in the active document providing a mouse-hover-event that shows the tooltip if fired.
By now I think, that it is impossible (with justified effort) to carry out this plan. Perhaps anyone of you has an idea.
As this is a scientific project for my study of information sciences it is very important to me finding a proper solution. Please help.
Thanks in advance
Christian
|
|
|
|
|
For all the automation objects provided to you by the Visual Studio Design-Time Extensibility object model (DTE), you are limited by what it can do (it's the code that implements the functionality you're trying to use).
So, persisting information is up to you. You have control - using the DTE object model - to select text, display objects (you can display tooltips; just search this site or http://google.com[^] for keywords like "DTE" and "tooltip", but it works much differently than with the .NET assemblies; they're entirely different classes implemented by entirely different platforms), and persist data.
For example, you could persist the line numbers when lines are changed (there are events for when lines change) and store a diff (or just a comment similar to what you'd see in Word's tracking feature) to a persist file. When the file changes, update your persisted data accordingly. As the user scrolls you must track the position of where this data is to be displayed.
It's all possible, but probably not in the way you're expecting. You're going to have to provide the implementation to persist and display these changes, but the code to catch changes, displays UI adornments, interact with the text editor, and more are provided to you through the DTE object model.
This posting is provided "AS IS" with no warranties, and confers no rights.
Software Design Engineer
Developer Division Sustained Engineering
Microsoft
[My Articles] [My Blog]
|
|
|
|
|
 Hi everybody!
I have to run a c# application that have to interact with Outlook. But the problem is that i have to adapt my application to the Outlook version...
Here are the running environment:
- Windows XP and Outlook 2003,
- Windows 2000 and Outlook 2000.
Moreover i got problem when i run my application and i think it's a problem of assembly.
Here are the error messages:
- "The reference is not set to an instance of an object."
- "The specified cast is not valid".
I don't think the problem comes from my application code but i can think everything...
Is there someone that can indicates me which assembly are required exactly? Should i put them in the Debug\Bin repertory of the project or somewhere else?
Here are the assembly i have referenced in my project:
- Interop.Outlook.dll (local copy),
- stdole.dll (local copy),
- Microsoft Outlook 10.0 Library
The application run perfectly on my computer, but when i try to install it on an other computer with FrameWork.Net1.1 i got the previous error messages.
Does someone can help me please!!!!
Doudou81
|
|
|
|
|
Always target the earliest possible typelib for Outlook. So, if you want to support Outlook 10 and 11, then interop the Outlook 10 typelib. For the most part (and for all Office products and properly written COM servers), COM servers are backward compatible (published interfaces should never changed, class IDs should never change but should implement new versions of interfaces, etc.).
You should not use your own, though. Office 2003 ships with the Office Primary Interop Assemblies (PIAs) (the ".NET Framework Programming Support" feature, or something like that) and you can download the PIAs for Office 2002 from http://www.microsoft.com/downloads/details.aspx?FamilyId=C41BD61E-3060-4F71-A6B4-01FEBA508E52&displaylang=en[^]. It's good to install and use the PIAs (once you install them correctly (read the README file) they will be referenced automatically instead of your own being created) since they are "official" RCWs (Runtime Callable Wrappers), typically from the company that created the COM server you're trying to interop with (and in these case they do come from Microsoft).
You must deploy these with your application, but should not - without a license - deploy the actual products (interop assemblies merely allow the Runtime to marshal calls to the COM servers that the interop assemblies/typelibs describe).
As far as the other exceptions, you must validate your code. The first is what happens when you don't check return values for null and the second is when you cast objects to types when the objects are that type or aren't derivatives of that type (or don't implement the interface if the interface was imported from a COM typelib).
It's not enough to just install the interop assemblies as I said above. The actual products must be installed on the machine. The interop assemblies (and typelibs, for that matter) do not contain the implementation, but merely describe it and assist clients with how to call the implementation it describes.
This posting is provided "AS IS" with no warranties, and confers no rights.
Software Design Engineer
Developer Division Sustained Engineering
Microsoft
[My Articles] [My Blog]
|
|
|
|
|
hi all,
I want to create a application that include the function of a web server(including Https protocol), so can the code of my application invoke
the IIS component object on the PC ? (Assume that the PC don't have IIS installed and also I dont want to install IIS for the application)
Thanks
|
|
|
|
|
There are a few things that you can do... Just out of interest, how are you going to be using this in your application?
This posting is provided "AS IS" with no warranties, and confers no rights.
Alex Korchemniy
|
|
|
|
|
There is no "IIS component" to handle SSL. Either IIS handles the SSL handshaking and passes off the unencrypted data to ASP.NET, or you handle the SSL handshaking.
You can host ASP.NET yourself without IIS using the System.Web.Hosting.ApplicationHost class, but it doesn't handle SSL (IIS does that for ASP.NET).
If you use the Socket class and program your own HTTP daemon you will still need to handle SSL handshaking.
Bottom line, firewall your machine, keep up to date with patches, and just use IIS. ASP.NET isn't itself an HTTP daemon (unless you host it outside IIS, but even then it's not optimized for so many connections and performance); it "inherits" that functionality from IIS, which handles the requests and passes those requests off to ASP.NET for file types, etc., that ASP.NET handles.
This posting is provided "AS IS" with no warranties, and confers no rights.
Software Design Engineer
Developer Division Sustained Engineering
Microsoft
[My Articles] [My Blog]
|
|
|
|
|
Hi there!
I need to convert an AVi file to a Windows Media Video file and I want to convert it using C#. I don´t know where to start and I don´t know if it is possible.
Can you give me some tips?!!!
Thanks for your help, Sérgio
|
|
|
|
|
Download the Windows Media SDK from http://msdn.microsoft.com/library/en-us/dnanchor/html/audiovideo.asp[^], read it well, then create an interop assembly (using tlbimp.exe or VS.NET itself by adding a reference to your project from the COM tab) and follow the Visual Basic 6 documentation, which uses the very same automation interfaces and supporting types that gets thrown into the interop assembly (RCW; Runtime Callable Wrapper).
There should be some articles that may help you here on this site, so use the search box at the top of every page right below the logo to search for them.
This uses the automation interfaces to make use of the WMV/WMA codecs (encoders and decoders), as well as other codecs on the system (you'll need the codecs for the AVI file, since AVI is only a file format - not a file encoding - and can use any codec the author wants to use).
Beyonds this, any questions you have about converting formats should be addressed in a different forum. If you have questions specific to C# (calling methods, constructing objects, P/Invoking native APIs not exposed in the automation interfaces, etc.), this forum is the place to ask.
This posting is provided "AS IS" with no warranties, and confers no rights.
Software Design Engineer
Developer Division Sustained Engineering
Microsoft
[My Articles] [My Blog]
|
|
|
|
|
Hi,
I have been developing a test application in Windows Forms (C#) to integerate with a Web Service to return a formatted XML document and then apply an XSLT template to convert the XML into a useful format.
Within the Windows Form enviorment I have had no problems, however when ported over to a class file i keep getting a protection error with the XslTransform object. (code snippet of the method that applies the transform below):
<br />
<font color="Green"><br />
private void WriteFFWImportFile(string strFileToLoad, string strFileToSave)<br />
{<br />
<font color="Green">
XslTransform myXsl = new XslTransform();<br />
myXsl.Load(ConfigurationSettings.AppSettings["XsltImportTransform"]);<br />
<br />
<font color="Green">
XsltArgumentList myArgs = new XsltArgumentList();<br />
myArgs.AddParam("ParsolImport","",strFileToLoad);<br />
<br />
<font color="Green">
Stream fs = new FileStream(strFileToSave,FileMode.Create);<br />
XmlWriter writer = new XmlTextWriter(fs,System.Text.Encoding.Unicode);<br />
<br />
<font color="Green">
XmlDocument myXml = new XmlDocument();<br />
myXml.Load(strFileToLoad);<br />
myXsl.Transform(myXml,myArgs,writer,null);<br />
<br />
<font color="Green">
writer.Close();<br />
fs.Close();<br />
}
Basically the code that I ported into a Class is exactly the same apart from where I am calling ConfigurationSettings.AppSettings["XsltImportTransform"] in WinForms - I have exposed via a property of the Class.
When I attempt to build the solution I get:
"Integration.cs(122): 'System.Xml.Xsl.XslTransform.XslTransform(object)' is inaccessible due to its protection level". I don't understand why this is happening ... is it a bug 
Paul Griffiths
.Net Developer
|
|
|
|
|
There several things wrong with this line:
myXsl.Load(ConfigurationSettings.AppSettings["XsltImportTransform"]); What if the .config file isn't present or that setting isn't found? Either do validation checking or catch the inevitable exception that will occur from the null return value. Validation is far less expensive than throwing exceptions, though you should do that anyway (the XML document that contains the XSLT might be invalid, so Load will throw an exception).
What is that property called? You said you encapsulated the preceeding line in a property? Depending on the namespaces you're importing for that source file and the name of the property, you may have a naming conflict. Showing us line 122 and surrounding lines of Integration.cs would be most helpful.
For example, if you're importing System.Xml.Xsl and your property type and name are both XslTransform then you'll have a name clash and the compiler will complain (how it complains depends on the circumstances). If this is the case, you should either fully-qualify your property type or use an alias like so:
using XSLT = System.Xml.Xsl.XslTransform;
public XSLT XslTransform
{
get { return ConfigurationSettings.AppSettings["XsltImportTransform"]; }
}Don't forget to dermine if that property returns null.
This posting is provided "AS IS" with no warranties, and confers no rights.
Software Design Engineer
Developer Division Sustained Engineering
Microsoft
[My Articles] [My Blog]
|
|
|
|
|
Thanks for the reply.
With regards to the ConfigurationSettings line of code, this was purely for a test application that I had direct control over so I hadn't put any validation in the code at the time.
Within my Class file I defined two string properties that will give a path to an import and export XSLT file. When these properties are set I will check for a valid path etc ... I've just illustrated the null case below.
<br />
public string XslImportTransform<br />
{<br />
get { return m_XslImportTransform; }<br />
set { m_XslImportTransform = value;<br />
RuleBroken("** Error: Invalid XSLT file.",m_XslExportTransform == null); }<br />
}<br />
When I tried to create an XslTransform object in a private method of the class, using the string property to Load() the XslTransform, I was encountering the protection level error (NB: It was not the Load() method that was causing the problem but the instantiation call XmlTransform myXsl = new Transform(); ). I couldn't understand why this was happening because I was creating the XslTransform as a local variable to the method (the name did not clash with any reserved words, public or private members).
<br />
public void WriteFFWImportFile(string strFileToLoad, string strFileToSave)<br />
{<br />
try<br />
{<br />
XslTransform myXsl = new XslTransform(this.m_XslImportTransform);<br />
myXsl.Load(this.m_XslImportTransform);<br />
<br />
XsltArgumentList myArgs = new XsltArgumentList();<br />
myArgs.AddParam("ParsolImport","",strFileToLoad);<br />
<br />
Stream fs = new FileStream(strFileToSave,FileMode.Create);<br />
XmlWriter writer = new XmlTextWriter(fs,System.Text.Encoding.Unicode);<br />
<br />
XmlDocument myXml = new XmlDocument();<br />
myXml.Load(strFileToLoad);<br />
myXsl.Transform(myXml,myArgs,writer,null);<br />
<br />
writer.Close();<br />
fs.Close();<br />
}<br />
catch(Exception ex)<br />
{<br />
ExceptionManager.Publish(ex);<br />
}<br />
}<br />
Is it simply a case that all XslTransform objects have to be exposed as public when used in a Class? If so why is that the case?
I have changed the class so I can set a transform object as a public property and everything works fine, although I only wanted to load the transform when a specific method is called.
Once again thanks for all your help. 
Paul Griffiths
.NET Developer
|
|
|
|
|
Whoopsie bit of a school boy error. 
Realised that I accidently put the string property into the constructor of the XslTransform object!
Tiredness is my worst enemy! 
Again thanks for the input and apologies for wasting your time. 
Paul Griffiths
.NET Developer
|
|
|
|
|
Hi
I have an Image class object.
How I can get a new Image - rectangular piece of first Image.
Is there a function like
Image piece = SourceImage.GetSubImage(int x1, int y1, int x2, int y2);
?
Thanks.
|
|
|
|
|
DrawImage has overloaded methods that allow you to draw part of an image ( via a graphics object ) onto another image ( via a Graphics object ).
Christian
I have several lifelong friends that are New Yorkers but I have always gravitated toward the weirdo's. - Richard Stringer
|
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
