Re: Programmatically controlling mouse. - Visual Basic Discussion Boards

Hi,

I would like to write a vb.net program that will programmatically position the mouse anywhere on the screen (not just the programs form) and then control the buttons and wheel. I.e. click on other applications.

Is this possible and can some please point me in the right direction.

Thanks,

Steve

Well, this isn't as easy as it would sound, but it is possible. The biggest problem is that control positions change along with the position of the windows you're clicking in. Also, directly controlling the mouse and simulating a click is not the way to go. You have to send the appropriate window messages to the control you're "clicking" on, without doing anything with the mouse. You have to get the handles of control windows (every control is a seperate window) to send these messages to. This means knowing the controls class type and name and how to find the correct control you want to click on.

You'd have to do research on FindWindow, SendMessage, various window messages, Windows Hooks, the CBT hook, message pumps, ...

Seriously, this is NOT a beginner level project. If you don't know this stuff, you're going to find it extremely difficult to do.

A guide to posting questions on CodeProject[^]

Dave Kreskowiak
Microsoft MVP
Visual Developer - Visual Basic
2006, 2007, 2008

Thanks for you reply.

Whilst I am a beginer in this topic I have actually been programming since the days of the ZX81 (~1981), all be it as a hobby and mostly self taught, I have come across the API's you mention before. Whilst it will be hard, if feel that with a little help pointing me in the right direction I ought to be able to manage it (if it's possible from within VB.net).

I do not want to acces any partucular control on any prticular form.

Rather I want to control mouse behaviour using an input device other than a mouse, if I can do it with a keyboard' I can do it with the controller I have in mind.

I have become disabled and am looking for a more effective way of interfacing with the computer than the conventional mouse and KB.

Thanks again

Steve.

the fossil wrote:
I do not want to acces any partucular control on any prticular form.

You don't really have a choice there. If not, you're just going to end up randomly clicking around the screen, not knowing what you're clicking on, if anything.

the fossil wrote:
Rather I want to control mouse behaviour using an input device other than a mouse, if I can do it with a keyboard' I can do it with the controller I have in mind.

That changes the question considerably. Normally, this stuff is done by writing a device driver. You're driver would have to take the hardware input and generate the appropriate mouse-equivilent messages. That is outside the relm of VB.NET or C#, but is squarely in the sights of Visual C++.

A guide to posting questions on CodeProject[^]

Dave Kreskowiak
Microsoft MVP
Visual Developer - Visual Basic
2006, 2007, 2008

Thanks,

That answers my question I had a feeling that would be the case but you can do so much more with vb.net now that I Thought it was worth asking the question.

Have you any idea about how I could go about getting such a device driver written?

Thanks again,

Steve.

Device Drivers are more of a black art. For every 1000 people you find that can write a normal application, about 1 or 2 can do device driver work.

A guide to posting questions on CodeProject[^]

Dave Kreskowiak
Microsoft MVP
Visual Developer - Visual Basic
2006, 2007, 2008

It seems it ca be done, check this out-

http://forums.devx.com/showthread.php?p=516633#post516633[^]

Steve

Hi Steve,
You are right , you can use mouse_even API to control the mouse movements , you can use the same API to control the click and scroll of the mouse ,following is an example to show you the use of this API

You don't require to write a device driver for it Smile | :)

I was try to answer this thread long , but got some error while posting , then it skipped of my mind Smile | :)

Sorry for that

This is the example to move the mouse

Public Class Form1
    Declare Auto Function mouse_event Lib "user32.dll" (ByVal dwflags As Integer, ByVal dx As Integer, ByVal dy As Integer, ByVal dwData As Integer, ByVal dwExtraInfo As Integer) As Integer
    Const MOUSEEVENTF_MOVE As Int32 = &H1 '  mouse move
    Const MOUSEEVENTF_LEFTDOWN As Int32 = &H2 '  left button down
    Const MOUSEEVENTF_LEFTUP As Int32 = &H4 '  left button up
    Const MOUSEEVENTF_RIGHTDOWN As Int32 = &H8 '  right button down
    Const MOUSEEVENTF_RIGHTUP As Int32 = &H10 '  right button up
    Const MOUSEEVENTF_MIDDLEDOWN As Int32 = &H20 '  middle button down
    Const MOUSEEVENTF_MIDDLEUP As Int32 = &H40 '  middle button up
    Const MOUSEEVENTF_ABSOLUTE As Int32 = &H8000 '  absolute move
    Const MOUSEEVENTF_WHEEL As Int32 = &H800 ' wheel button rolled
    Dim x As Int32 = 50
    Dim y As Int32 = 50
    Dim multiPlyfactorX = 65535 / Screen.PrimaryScreen.WorkingArea.Width
    Dim multiPlyfactory = 65535 / Screen.PrimaryScreen.WorkingArea.Height

    Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click
        mouse_event(MOUSEEVENTF_ABSOLUTE + MOUSEEVENTF_MOVE, x * multiPlyfactorX, y * multiPlyfactory, 0, 0)
        x = x + 1
        y = y + 1
    End Sub
End Class

Here you would have noticed that i have used two variable called multiPlyfactorX , multiPlyfactory. This is because the API takes 0,0 as left-top corner and 65535,65535 as right-bottom corner

also to send a click you can use the following code

mouse_event(MOUSEEVENTF_LEFTDOWN + MOUSEEVENTF_LEFTUP, x * multiPlyfactorX, y * multiPlyfactory, 0, 0)

Here x,y are the co-ordinates in pixcels

I hope this helps !!
I have to move now , let me know if you need any other help / information

-Regards
Bharat Jain
bharat.jain.nagpur@gmail.com

Presmably this will transfer the focus to the clicked window, this is fine in some instances but in others where I want to position the cursor brfore sending text this will be a problem. Can you think of a way of returning the focus to my app. after positioning the cursor.

Thanks,

The control will be still with your application , just after sending the click command , write
Me.Activate()(i don't recommend its use)

this will activate your form , but if you have to enter any text in a different application the focus must be on the other application , not on your .

To make it more clear let us consider an example, if i have application "A" running it has a text box where we need to enter user name (without using a keyboard & mouse) , i will first position my mouse location on the text box (using the code i sent earlier) and we would send a mouse click (using the same code) just below the Mouse click code(now the focus is on other form (application "A")) we will write the code to send keystrokes (focus is on another form but your code will be executed , till your function ends), this keystroke would be sent to the form in focus (that is the form of application "A").

Also if you want to send keystrokes, you must use the Win32 API Called "keybd_event" defined in user32.dll , the same way we used mouse_event in previous reply.

does this make sense for you Smile | :)

-Regards
Bharat Jain
bharat.jain.nagpur@gmail.com

Thanks Bharat,

That's very clear.

I tend to use sendkeys.send rather than "keybd_event" it that OK?

Steve

I've come across another problem now.

I often use the forms topmost property, setting it to true, It has allways worked fine but now on this form it does not remain topmost. This happpens before any of the API's are called. Even just loading the form and moving away from that form causes it to be hidden. Any Ideas.

Thanks,

Steve

Don't wory, I just closed VS ran it again, reloaded the project and it works fine.

Wierd!

Steve

How can I create a DataTable from a System.Collections.Generic.List please?
(I am a VB newbie Blush | :O

)

One way you can do this is using Reflection[^], but why do you want to do this?

The need to optimize rises from a bad design.My articles[^]

My existing application (company application) returns data as a Generic.List. But, I need to filter the data displayed in a DataGridView. So, I need to use a DataView (which I can filter on) which gets it's data from the intended DataTable.The DataGridView then has its DataSource as the DataView.

if that makes any sense at all (hopefuly I'm on the right track)

Anyway, many thanks for the Reflection pointer. I'll start researching that right now. Thumbs Up | :thumbsup:

Yes, it makes sense, but I think it would be much easier if you use System.ComponentModel.BindingList<t>[^] since that collection can used as a datasource in binding. Put the data from the List<t> to BindingList<t>.

Then what comes to filtering, you could most likely use LINQ to Objects[^]. This way you would save all the trouble of using reflection.

The need to optimize rises from a bad design.My articles[^]

As I am new to VB I'll have to hammer the VS help and Google get this straight in noodle.

Many thanks for your pointers.

You're welcome Smile | :)

The need to optimize rises from a bad design.My articles[^]

I am using MS Access 2007 on a Vista system.

I want to programatically manipulate Vista file properties details of .jpg files (Tags, Comments, etc) with a VBA script in Access 2007.

DSOFile.dll shows Summary fields but not the new Detail fields Vista uses. Any idea how I might be able to do this?

Hi !
I have two comboboxes in VB.NET. One is to display State from state table and another is used to display districts from district table. When i select a partiular state districts corresponding to that state only needs to be displayed.The problem is at the time of databinding combobox control what query i need to place in the Query builder of TableAdapter Configuration Wizard so that if i select a particular state districts corresponding to that state only needs to be displayed. I tried with creating SQL Dataadapters and datasets manually but it doesn't work out. Also i tried various types of SQL Server joins in Query Builder of TableAdapter Configuration Wizard but it doesn't work out. So pls help me regarding this problem.

modified on Saturday, January 31, 2009 2:34 AM

This is so trivial, it's hard to know where to start. You grab the id of the item selected in the first column and use it to select the data you need to show in the second column. I don't see how you can mess that up, you're going to need to post some code, etc, so we can try to work out what you're screwing up.

Christian Graus

Driven to the arms of OSX by Vista.

I have placed my Code below. Everything works fine except that the Excel application is running at the backgroung even after quitting the Excel using code also setting the Excel object to Nothing. But the Excel application terminates on closing the application.

When I click the button for the first time, the excel application appears in the task manager, but is not closing. But when I click the button for the second time a second Excel application instance appears in the task manager, but it closes immediately when I set the Excel object to "Nothing".

Anybody, pls suggest me how to close the excel instance which is created for the first time.

Private Sub ExportExcel_Click()
Dim objExcel As Excel.Application
Dim objWorkbook As Excel.Workbook
Dim objwksSheet As Excel.Worksheet

Dim Msg, style, Title, Response
style = vbYesNoCancel + vbQuestion + vbDefaultButton2 ' Define buttons.
Msg = "Before Exporting the Tags to Excel, all the unsaved Excel Files must be closed. Are you sure you want to Continue?"
Title = "PV Trend" ' Define title.
Response = MsgBox(Msg, style, Title)
If Response = vbNo Or Response = vbCancel Then ' User chose No.
Exit Sub
End If

Dim i As Long
Dim n As Long
Dim gridChart As ChartObject
Dim tagdataChart As Chart
Dim curentChart As Object

If SelTagsCnt <= 0 Then
MsgBox "No Tags are there to export"
Exit Sub
End If
'***************************************************************
' create new hidden instance of Excel and open its workbooks collection
If DebugMode = True Then
xlsfilename = "c:\testExcel.xls"
Else
xlsfilename = glbSysPath & "esim\Model\" & ModelName & _
"\Instructor\PVTREND" & ModelName & ".xls"
End If
' Set objExcel = New Excel.Application
' objExcel.Workbooks.Open xlsfilename
' objExcel.Worksheets("Sheet1").Activate
' objExcel.DisplayAlerts = False

' With objExcel
' .ActiveWorkbook.Save FileName:= _
' xlsfilename, _
' FileFormat:=xlNormal, Password:="", _
' WriteResPassword:="", ReadOnlyRecommended:=False, _
' CreateBackup:=False
'
' .ActiveWindow.Close
' End With
'*******************************************************************

On Error Resume Next
'Call EmpMSFgrid_load
'Set objExcel = GetObject(, "Excel.Application")
Set objExcel = New Excel.Application
' If Err.Number Then
' Err.Clear
' Set objExcel = CreateObject("Excel.Application")
' If Err.Number Then
' MsgBox "Unable to open Excel Application."
' End If
' End If
objExcel.Visible = True
Set objWorkbook = objExcel.Workbooks.Add

objExcel.Worksheets("sheet1").Activate

xlAPP.Worksheets("Sheet1").Range (strRangeXVal)

' Merging the First 3 Rows
With objWorkbook.Sheets("Sheet1")
For i = 1 To 3
.Range(.Cells(i, 1), .Cells(i, 50)).Select
Selection.Merge
Next i
End With

' Add Heading "PROSIMULATOR HISTORICAL PV TREND PLOT" in the First Row of the Excel File
objWorkbook.ActiveSheet.Cells(1, 1).Font.Bold = True
objWorkbook.ActiveSheet.Cells(1, 1).Font.Size = 18
objWorkbook.ActiveSheet.Cells(1, 1).Font.Color = RGB(0, 0, 255)
objWorkbook.ActiveSheet.Cells(1, 1).Value = " PROSIMULATOR HISTORICAL PV TREND PLOT"
objWorkbook.ActiveSheet.Cells(1, 1).Interior.Color = RGB(255, 128, 0) 'RGB(185, 185, 185)

' Format 2nd Row
objWorkbook.ActiveSheet.Cells(2, 1).Font.Bold = True
objWorkbook.ActiveSheet.Cells(2, 1).Font.Size = 11
objWorkbook.ActiveSheet.Cells(2, 1).Font.Color = RGB(41, 11, 164)
objWorkbook.ActiveSheet.Cells(2, 1).Value = "Model Name:" & ModelName
'objWorkbook.ActiveSheet.Cells(2, 1).Interior.Color = RGB(137, 200, 237)

objWorkbook.SaveCopyAs xlsfilename
' .ActiveWorkbook.SaveAs FileName:= _
' xlsfilename
objWorkbook.Close savechanges:=False

objWorkbook.Close
'Set objWorkbook = Nothing
objExcel.Visible = False
objExcel.ActiveWindow.Close
objExcel.Quit
objExcel.Application.Quit

' End With

Dim sTitle As String
Dim iHwnd As Integer
Dim ihTask As Integer
Dim iReturn As Integer
sTitle = "EXCEL"
iHwnd = FindWindow(0&, sTitle)
iReturn = PostMessage(objExcel.hWnd, WM_QUIT, 0, 0)

Set objWorkbook = Nothing
Set objExcel = Nothing
Set curentChart = Nothing
Set tagdataChart = Nothing
Set gridChart = Nothing

End Sub

Excel won't quit because your code still has references to object you were using in Excel. You have to explicitly set EVERY object you create in the Excel object model to Nothing, THEN call Quit, otherwise, you'll get the behavior you're seeing.

A guide to posting questions on CodeProject[^]

Dave Kreskowiak
Microsoft MVP
Visual Developer - Visual Basic
2006, 2007, 2008

i have installed Visual Studio 2005.net and a writing a student management system,however, i need to create a help file,but i have got no clue....where do i begin to create one..