Re: How to read/extract data from a web page? - Visual Basic Discussion Boards

To put it simply you don't display an Excel file in a RichTextBox, this control is really only designed to display RTF and plain text files.

I think your best option would be to invest in a 3rd party control designed to display Excel files such as those from Spreadsheet Gear[^].

I am trying to build a VB.NET windows app which gets cars info from a webpage(webapplication) that has a username and password.
I was able to programmatically login to this webpage(by automatically populating the input boxes) And after I logged in, I could view the cars data in browser and I did "View Source" but the cars data (such as car model, brand, color etc..) were not viewable in the page source code. So how can I read these data with my application?
I hope my question was clear and literally need some help.
Thanks

This is an example of calling a webpage and getting the ip address from the source - i use it as part of my dynamic dns client that i made.

Dim Req As HttpWebRequest = WebRequest.Create("http://www.whatsmyip.us/")

*****************************************************************
** if you need to pass credentials then do this line **
Req.Credentials = New NetworkCredential("USERNAME", "PASSWORD")
*****************************************************************

Dim rs As New StreamReader(Req.GetResponse.GetResponseStream(), System.Text.Encoding.GetEncoding("utf-8"))
Dim s As String = rs.ReadToEnd 'you know now have the entire source in variable 's'

rs.Close()
rs.Dispose()
rs = Nothing

Req = Nothing

Dim IP As String = Regex.Match(s, "\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}").Value

'Never argue with an idiot; they'll drag you down to their level and beat you with experience.' ~ anonymous

'Life's real failure is when you do not realize how close you were to success when you gave up.' ~ anonymous

Message Removed

That code will only work for a web page that forces the browser to show a login dialog box. For all others (most) websites, you'll have to get the login page, find the textboxes in the returned HTML, fill them in and submit the page back. No, I don't have an example.

A guide to posting questions on CodeProject[^]

Dave Kreskowiak
Microsoft MVP
Visual Developer - Visual Basic
2006, 2007, 2008
But no longer in 2009...

Message Removed

jadothebest wrote:
I done this part already, I can finally login(i used webbrowser control).

You already did it. You use the same type of code to find the elements in the page and extract the data from it.

A guide to posting questions on CodeProject[^]

Dave Kreskowiak
Microsoft MVP
Visual Developer - Visual Basic
2006, 2007, 2008
But no longer in 2009...

The data did not appear in source code at all so I had to install a firefox addon called "Web developer"
and I clicked "View generated source code" and I could see the data there.
In my case the cars data are being shown in a dynamic table, 1 row adds up as soon as a new car comes.
Here is 1 row of data:

<td class="x-grid3-col x-grid3-cell x-grid3-td-dealer x-grid3-cell-first " style="width: 78px;" tabindex="0"><div class="x-grid3-cell-inner x-grid3-col-dealer" unselectable="on">Privat</div></td><td class="x-grid3-col x-grid3-cell x-grid3-td-manufacturer " style="width: 98px;" tabindex="0"><div class="x-grid3-cell-inner x-grid3-col-manufacturer" unselectable="on">AUDI</div></td><td class="x-grid3-col x-grid3-cell x-grid3-td-modelDescription " style="width: 157px;" tabindex="0"><div class="x-grid3-cell-inner x-grid3-col-modelDescription" unselectable="on">A2 1.4 </div></td><td class="x-grid3-col x-grid3-cell x-grid3-td-price " style="width: 88px; text-align: right;" tabindex="0"><div class="x-grid3-cell-inner x-grid3-col-price" unselectable="on"><span class="format-right">7.000</span></div></td><td class="x-grid3-col x-grid3-cell x-grid3-td-firstRegistration " style="width: 58px;" tabindex="0"><div class="x-grid3-cell-inner x-grid3-col-firstRegistration" unselectable="on">6/2000</div></td><td class="x-grid3-col x-grid3-cell x-grid3-td-mileage " style="width: 73px;" tabindex="0"><div class="x-grid3-cell-inner x-grid3-col-mileage" unselectable="on"><span class="format-right">122.000</span></div></td><td class="x-grid3-col x-grid3-cell x-grid3-td-powerInKw " style="width: 48px;" tabindex="0"><div class="x-grid3-cell-inner x-grid3-col-powerInKw" unselectable="on"><span class="format-right">55</span></div></td><td class="x-grid3-col x-grid3-cell x-grid3-td-modificationDate " style="width: 98px;" tabindex="0"><div class="x-grid3-cell-inner x-grid3-col-modificationDate" unselectable="on"><span class="format-right">Heute - 21:29</span></div></td><td class="x-grid3-col x-grid3-cell x-grid3-td-location " style="width: 98px;" tabindex="0"><div class="x-grid3-cell-inner x-grid3-col-location" unselectable="on">85256 Vierkirchen</div></td>

few of values which i want to grap are 'AUDI', 7.000, "85256 Vierkirchen"
So far I coudn't grab these values using my code... So I thought of a new solution, I installed another ffox addon called "firebug" and checked how the data is coming and found out it's REVERSE Ajax DWR requests and responses.
I monitored these requests, and I noticed that every 1 minute, a request/response comes via POST and all these responses have the size of 90 bytes. But then I saw a response with a bigger size(about 648 bytes) I quickly checked the table in website, and YES a new car was there.

Since I had no idea about what reverse ajax dwr is, i googled a bit and found out that this website is using the polling method for pushing data every 1 minute from server to browser.
So to make this short, the data is coming from website to my browser every 1 minute, here is the response Body which I saw in firebug:

//#DWR-START#
adQueueListener.addNewAd({adId:124725772,category:'Limousine',changeInfo:{adId:124725772,adVersion:24,changeEventId:457859860,changeSubtype:['DESCRIPTION'],changeType:'Update',comment:"DESCRIPTION",creationTime:1262659849452,modificationTime:new Date(1262659849000),objectType:"AD"},city:"hanau",climatisation:'AUTOMATIC_CLIMATISATION',color:'BEIGE',commercial:false,consumerGrossPrice:4200.00,creationDate:new Date(1260208837000),dealer:false,dealerGrossPrice:null,doorCount:'FOUR_OR_FIVE',firstRegistration:new Date(788914800000),fuel:'PETROL',machtedFilterNames:["Mercedes"],makeName:"MERCEDES-BENZ",manufacturerColorName:"",metallic:false,mileage:149000,modelDescription:"C 180 Elegance",modificationDate:new Date(1262659849000),numSeats:5,powerInKw:90,powerInPs:122,priceDecrease:false,sellerId:2467966,transmission:'AUTOMATIC_GEAR',vehicleCategory:null,version:24,zipcode:"63450"});
//#DWR-END#
//#DWR-START#
dwr.engine.remote.handleCallback("47","0",0);
//#DWR-END#

The response location(ajax script location) is http://gb-ticker.mobile.de/ticker/dwr/call/plainpoll/ReverseAjax.dwr The data I want are in the adQueueListener.addNewAd method. So how to get these data from this ajax function?

Please let me know whether it can be done by parsing the html code or the ajax requests.
I will be pleased if you can shed some lights.
Many thanks

I have no idea. I've never done it and never will.

A guide to posting questions on CodeProject[^]

Dave Kreskowiak
Microsoft MVP
Visual Developer - Visual Basic
2006, 2007, 2008
But no longer in 2009...

Hi all,

I use the following code to import a XML file into an Access database:

Dim ds As New DataSet
Dim streamRead As New System.IO.FileStream(strFileName, System.IO.FileMode.Open)
ds.ReadXml(streamRead)
streamRead.Close()
If ds.Tables.Count > 0 Then
strConnection = "Provider=Microsoft.ACE.OLEDB.12.0;Data Source=" & dbname
objConnection = New OleDbConnection(strConnection)
objConnection.Open()
objAdapter = New OleDbDataAdapter()
objAdapter.ContinueUpdateOnError = True
Dim cb As New OleDbCommandBuilder(objAdapter)
For intI = 0 To ds.Tables.Count - 1
If ds.Tables(intI).Rows.Count > 0 Then
cb.RefreshSchema()
objAdapter.SelectCommand = New OleDbCommand("SELECT * FROM [" + ds.Tables(intI).TableName + "] ", objConnection)
objAdapter.Fill(ds, ds.Tables(intI).TableName.ToString)
objAdapter.UpdateCommand = cb.GetUpdateCommand()
objAdapter.Update(ds, ds.Tables(intI).TableName.ToString)
End If
Next
objConnection.Close()
End If

The import works fine, but when a next XML file is imported with less records, the database don't get updated and the 'old' records remain in the database. Simply delete the 'old' records is fine by me.

Can somebody point me to the right direction.

With kind regards,
Jurgen

explicitly set the update command

I have develop one window application in C#.Net. I want to create a trial version for 15 Days. How can I do.

Well done on creating your c# application.......so why post your question in the vb forum?

You will need to have a mechanism that records when the application was first used and then check the time stamp on each successive startup or at a random time during running the application in a background worker thread. Obviously you would have to have the saved date/time obscured in some way so the user can't edit it to circumvent the mechanism.

Applications such has winzip (i think it was this app) used to use a strange named registry key, buried deep in the registry, but using tools such as regmon, you could monitor the application startup, discover the key location and change the value.

Dave
Who am I?:
Web|Facebook|Twitter|LinkedIn|Bebo

I create controls that
Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load
Dim l As Integer
l = 20
'MsgBox(MDIParent1.k)
For i = 1 To MDIParent1.k
l = l + 30
Dim lab As New Label
Dim t As New TextBox
Dim chk As New CheckBox
lab.Location = New Point(10, 30 + l)
t.Location = New Point(100, 30 + l)
chk.Location = New Point(300, 30 + l)
Me.Panel1.Controls.Add(chk)
Me.Panel1.Controls.Add(lab)
Me.Panel1.Controls.Add(t)
lab.BringToFront()
t.BringToFront()
chk.BringToFront()
t.Size = New Size(170, 20)
t.Name = "t" & i
lab.Name = "lab" & i
lab.Text = "Отговор" & i
chk.Name = "chk" & i
If i = 2 Then
t.Text = "bravo be"
End If
Next
End Sub

but I don't know how to use these controls, for example
Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs)
t1.text="Hi"
End Sub

this give me error because controls t1 missing. This TexBox control is creating dinamycally

Because you created the controls dynamically, you cannot access them in the same way. You added the controls to the panel1 so only through panel1 can you acccess them. Panel1.controls("t1").Left

'Never argue with an idiot; they'll drag you down to their level and beat you with experience.' ~ anonymous

'Life's real failure is when you do not realize how close you were to success when you gave up.' ~ anonymous

Thank You very much. This work

The below code will work...
check it.

Public Class Form1
'Dim cmb As New ComboBox
Private WithEvents cmb As System.Windows.Forms.ComboBox

Private Sub cmb_SelectedValueChanged(ByVal sender As Object, ByVal e As System.EventArgs) Handles cmb.SelectedValueChanged
MsgBox("The data for the item you selected is : " & CType(cmb.SelectedItem, DataItem).Getdata())
'cmb.Focus()
End Sub

Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click
Dim Objects(20) As DataItem

Dim lbl As New Label
Dim pnt As Drawing.Point
cmb = New ComboBox

cmb.DropDownStyle = System.Windows.Forms.ComboBoxStyle.DropDown
cmb.Dock = DockStyle.Fill

pnt.X = 10
pnt.Y = 20
lbl.Location = pnt

lbl.Text = "Select Items"
Me.Controls.Add(lbl)
'Me.Controls.Add(cmb)
Panel1.Controls.Add(cmb)
'Form1.Controls.Add(cmb)
cmb.BringToFront()
cmb.BeginUpdate()
Dim intloopindex As Integer
For intloopindex = 0 To 20
Objects(intloopindex) = New DataItem("Item " & intloopindex, CSng(intloopindex))
Next
cmb.Items.AddRange(Objects)
'cmb.Items.Clear()
'For intloopindex = 0 To 20
' cmb.Items.Add("item - " & intloopindex)
'Next
cmb.EndUpdate()

End Sub

End Class

Hi every one could someone give a direction how do I create a Arabic text editor without using Unicode
I want to make a text editor font based
set the font in text propriety to the modified font
To use a font modifier to replace all 256 ASCII character with Arabic character
Can I achieve same Arabic writing system as Unicode dose by creating a font contain only arabic character

when you install a font and you can type in Arabic the way Unicode in your textbox

anybody

Your impatience is just scaring away those who might help you.

"I love deadlines. I like the whooshing sound they make as they fly by." (DNA)

Genius Real men

can anyone give me the source code for simple multi-wavelet image compression in visual basic i really need your help??

Hello experts.

I have a .txt file which all works fine and i can read it fine. I have a program which i where i need to split this 1 big text box up. I roughly think i know how to do it but am stuck.

nFileNum = FreeFile

' Open a text file for input. inputbox returns the path to read the file
Open App.Path & "\stock.txt" For Input As nFileNum
lLineCount = 1



' Read the contents of the file
Do While Not EOF(nFileNum)
   Line Input #nFileNum, sNextline
   'do something with it
   'add line numbers to it, in this case!
   sNextline = sNextline & vbCrLf
   stext = stext & sNextline & vbNewLine    'take out sNumbers if dont work



Loop


txtRead.Text = stext

what i intially want to do is have txtRead1.text for 1 piece of information, then nextline txtRead2.text and so on.

I thought i had to use a for next loop but they really don't like me!!

If any body can help that would be really helpful Smile | :)

Cheers Dan

How about create an array of textboxs then put in loop with a variable that increments.

For example

......

textbox(x).text = stext

x = x+1

loop

Dear all,

I use VB 2005 with shell comman for zip file.
my task for zip is with space of folder name and file.

So when i zip them. it was not successfull.
It causes of space.

that my source code:
shell("C:\Program Files\WinZip\WZZIP.EXE -u D:\Ready Sent\File1.zip D:\Source File\File Send.PDF")

Do you have any solve of this issue ?

Thanks for your help...

VB.Net

modified on Friday, January 1, 2010 6:25 AM