Re: VARIANT_BOOL return value in Automation class - COM Discussion Boards

9-Jan-07 5:07

I have a Automation class present in local Automation server. This class has following method
VARIANT_BOOL IsWorking();

which is part of a dispinterface.

I have a MFC Client which contains a wizard generated class that derives from COleDispatchServer. This class represents the above mentioned Automation Class. The wizard generated code for this method is as follows -

BOOL IsWorking()<br />
    {<br />
        BOOL result;<br />
        InvokeHelper(0x2, DISPATCH_METHOD, VT_BOOL, (void*)&result, NULL);<br />
        return result;<br />
    }

now every call to IsWorking() method always returns 1 ( IsWorking() returns VARIANT_TRUE and VARIANT_FALSE correctly. I have verified this by debuggin server code ) . VARIANT_TRUE and VARIANT_FALSE are being transalted somewhere during the call being completed from servers side.

Any guess what I am missing here?

Second, I tried making the return type as void and instead return the boolean as an OUT param as VARIANT_BOOL*
In this case, i get invalid pointer address inside my server's
void IsWorking(VARIANT_BOOL* pRetVal) method
hence resulting in access violation.
What could be the problem in this case.

S o h a i l K a d i w a l a

To Err Is Human; to Debug, Divine

modified 21-Apr-21 21:01pm.

Roger Stoltz9-Jan-07 5:45

Re: VARIANT_BOOL return value in Automation class [modified]

9-Jan-07 5:45

Do you explicitly use the VARIANT_BOOL type in the .IDL file as well?
Anyway, you should if you're not.
It could be a mixup between the use of boolean which is defined as an unsigned char and BOOL which is defined as a long for MIDL.
If you've checked the option to be MkTypLib-compatible you'll run into problems since for MkTypLib BOOL, boolean and VARIANT_BOOL are defined as short.

You need to delete the wizard generated files when you change the interface and run the wizard again.

--
Roger

"It's supposed to be hard, otherwise anybody could do it!" - selfquote

"No one remembers a coward!" - Jan Elfström 1998
"...but everyone remembers an idiot!" - my lawyer 2005 when heard of Jan's saying above

User 2155979-Jan-07 5:52

9-Jan-07 5:52

Thanks for your answer.

Roger Stoltz wrote:
Do you explicitly use the VARIANT_BOOL type in the .IDL file as well?
Anyway, you should if you're not.

Yes, I am explicitly using VARIANT_BOOL and not any other boolean type since I read about VARIANT_BOOL usage. FYI, the app i am working on is an old app and has an ODL file and not IDL. MIDL config uses "/mktyplib203" - MKTypLib compatible option.

Roger Stoltz wrote:
It could be a mixup between the use of boolean which is defined as an unsigned char and BOOL which is defined as a long for MIDL.

I doubt about the mixup because all method and interface and wrapper classes decalarations were generated using MFC wizard and has no modifications

Roger Stoltz wrote:
You need to delete the wizard generated files when you change the interface and run the wizard again.

Yes, I deleted and re-generated everytime.

-- modified at 13:02 Tuesday 9th January, 2007

Further information - the return value "pvarResult" changes to (True, VT_BOOL) after this call (MFC source, File: oledisp2.cpp, Line# 373) -

// make the call<br />
SCODE sc = m_lpDispatch->Invoke(dwDispID, IID_NULL, 0, wFlags,<br />
		&dispparams, pvarResult, &excepInfo, &nArgErr);

Though, now I have hard coded the return value in the function to return VARIANT_FALSE everytime!

S o h a i l K a d i w a l a

To Err Is Human; to Debug, Divine

modified 21-Apr-21 21:01pm.

Roger Stoltz9-Jan-07 21:18

9-Jan-07 21:18

Sohail Kadiwala wrote:
I doubt about the mixup because all method and interface and wrapper classes decalarations were generated using MFC wizard and has no modifications

Don't be too sure about this one. At least one of the wizards puts boolean as type when a method is added and you select BOOL as type, even if the interface has the attribute [oleautomation]. The result is that it won't even make it through the MIDL compilation: an error is generated and you'll be asked to use VARIANT_BOOL instead of boolean in automation interfaces.
I would really like to see the declarations in the .odl, .h and the wizard generated header file for the client.

Sohail Kadiwala wrote:
Further information - the return value "pvarResult" changes to (True, VT_BOOL) after this call (MFC source, File: oledisp2.cpp, Line# 373)

This rings a bell quite loudly... I've seen this problem before, but I cannot remember what the solution was.
I have a vague memory that it was really about the mixup of boolean types and marshalling.

Can you describe the relationship between your server and client a little bit further? EXE-server? Threads? The need for marshalling?
The name IsWorking suggests that it could be multithreaded...

--
Roger

User 21559710-Jan-07 0:26

10-Jan-07 0:26

Roger Stoltz wrote:
I would really like to see the declarations in the .odl, .h and the wizard generated header file for the client.

SERVER code is below -

dispinterface ISomeInterface<br />
	{<br />
		properties:<br />
		methods:<br />
		[id(1), helpstring("")] void Start(BSTR str);<br />
        [id(2), helpstring("")] VARIANT_BOOL IsWorking();<br />
    };

Start actually starts an asynchronous activity on the server. I use IsWorking method to determine whether the activity is over or not. In my client, when a request is recieved for another "start", I need to check whether the Server is already working, if yes, then I need to wait in my client. Server notifies the Client about activity completion through COM event source/sink mechanism. So in short, IsWorking determine whether the client should wait or not before issuing any requests

In Server's SomeInterface.H file, following is the declaration

void Start(LPCTSTR str);<br />
VARIANT_BOOL IsWorking();

Client code generated by Wizard -
void Start(LPCTSTR str)
{
static BYTE parms[] = VTS_BSTR ;
InvokeHelper(0x1, DISPATCH_METHOD, VT_EMPTY, NULL, parms, str);
}
BOOL IsWorking()
{
BOOL result;
InvokeHelper(0x2, DISPATCH_METHOD, VT_BOOL, (void*)&result, NULL);
return result;
}

Roger Stoltz wrote:
This rings a bell quite loudly... I've seen this problem before, but I cannot remember what the solution was.
I have a vague memory that it was really about the mixup of boolean types and marshalling.

While Googling for hours, I got this thread which is similar but the Class and file in talk though similar, is different. Check following thread -
http://groups.google.com/group/microsoft.public.vc.mfc/browse_thread/thread/fa6cffd879ac26da/1564231c27c4667a?lnk=st&q=V_BOOL(&rnum=10#1564231c27c4667a[^]

Roger Stoltz wrote:
Can you describe the relationship between your server and client a little bit further? EXE-server? Threads? The need for marshalling?
The name IsWorking suggests that it could be multithreaded...

The EXE server is created in MFC and it has exposed SomeInterface class which is CoCreatable. The Client could be any client, I have -
1. ActiveX component which is client to my server. And this control is placed inside C# application. ActiveX component is written in MFC
2. I have created a Simple MFC EXE with FormView, which directly calls the methods of the EXE server. This is did just to ensure there is not threading or marshalling issue because of Managed C# code. But the problem occurs in both the Managed client with ActiveX control or Unmanaged client

The coclass has no threads. The Exe server is multithreaded but the coclass Start method simply delegates the call there.
My clients are single threaded.
Regarding marshalling, I believe standard marshalling should take care of this because i am using the OleAutomation compatible type.
Everything works fine for Start method and I succesfully get the "BSTR" from my clients hence I see no issue about marshalling.

To me it seems like ~~a bug~~ some problem as discussed on the thread I have mentioned above. I think the problem is somewhere in IDispatch::Invoke code. I should avoid calling it a bug till I am sure about this Smile | :)

As of now, I have converted the return type to LONG and everything is working fine. I can proceed further on the original code. I have created a sample application which I can use to diagnose this issue with you side-by-side.

S o h a i l K a d i w a l a

To Err Is Human; to Debug, Divine

modified 21-Apr-21 21:01pm.

Roger Stoltz10-Jan-07 2:42

10-Jan-07 2:42

Regarding your declarations, I cannot find anything wrong: it looks as expected.
N.B. I'm not quite convinced yet. I think the problem described in the forum discussion you found could be a consequence of this mixup. The reason why I'm not sure is because it's an .odl file and you want to be compatible with MKTypLib and I'm not 100% on top of what happens then.

So, the server is an out-of-process server.
This means that every call has to be marshalled.

Sohail Kadiwala wrote:
The coclass has no threads. The Exe server is multithreaded but the coclass Start method simply delegates the call there.

I don't exactly know what you mean by these statements, but I guess you mean that you do not initialize more than one apartment in the server and that is in the application thread of the server. I hope this also means that you do not call any interface functions from anywhere else than from that thread, even if you're talking about an event sink.
Tell me if this is not the case.

Sohail Kadiwala wrote:
Regarding marshalling, I believe standard marshalling should take care of this because i am using the OleAutomation compatible type.

Well, if you haven't built and registered anything else explicitly, the universal marshaller will be used for automation interfaces. This is also known as type library marshalling. Standard marshalling requires you to build a second DLL with the proxy/stub.
I assume you're using typelib marshalling, which is exactly why I was talking about a mixup between boolean types since the generated typelib will be used for marshalling. I think you can figure out the rest if I present the problem as "you may declare a return type as 16-bit wide in your code but 32-bit wide in the typelib".

Sohail Kadiwala wrote:
Everything works fine for Start method and I succesfully get the "BSTR" from my clients hence I see no issue about marshalling.

This is not proof of anything other than the call to Start seems to work. The call to IsWorking may still be erroneous even though it looks OK as mentioned above.

I would like to follow up on the forum discussion you found. I think there might be something there to try out.
If you change the type back to VARIANT_BOOL and put a breakpoint after the call is made in oledisp2.cpp, line #373. Try forcing the server to return VARIANT_FALSE and VARIANT_TRUE every second time you call it. Take a look at the lower 16 bits of the return value. It should change between 0x0000 and 0xffff. The upper 16 bits could contain garbage which would make the return value evaluate to true even if VARIANT_FALSE is returned.
What happens? What type is returned (VT_BOOL) and what is the value of it?
Let's first find out what is returned and how it's wrong and then we can try and find the correct solution for it.

--
Rog

User 21559711-Jan-07 0:03

11-Jan-07 0:03

Let me work on this as you mentioned and update you. I'll create a sample which I can share with you if required.

BTW VARIANT_FALSE and FALSE and false are all "0" so when I return VARIANT_FALSE the return value should be 0 and not 1 Sigh | :sigh:

Any thoughts on this?

S o h a i l K a d i w a l a

To Err Is Human; to Debug, Divine

modified 21-Apr-21 21:01pm.

Roger Stoltz11-Jan-07 0:46

Re: VARIANT_BOOL return value in Automation class [modified]

11-Jan-07 0:46

Sohail Kadiwala wrote:
VARIANT_FALSE and FALSE and false are all "0" so when I return VARIANT_FALSE the return value should be 0 and not 1 Any thoughts on this?

Well, this is what I want you to verify immediately after the call has returned.
As read even in the forum discussion the thing is when you return a VARIANT_FALSE, which is defined as a 16-bit wide zero, it may be interpreted as 32-bit wide BOOL type. Infact those are not passed 'by value', they're passed as pointers. I think the following will demonstrate the problem:

short* pwValue = new short;
*pwValue = 0;
long* pdwMoreValue = (long*)pwValue; //This is what happens behind the scenes

// A statement like the one below is totally unreliable because
// the upper 16 bit have never been initialized
if( *pdwMoreValue )
{
    ....
}

My point is that your variable may contain zeros in the lower 16 bits, which would match VARIANT_FALSE, but the upper 16 bits may contain garbage. I want you to find out if this is the case as step #1.

--
Rog

User 21559711-Jan-07 5:03

Re: [SOLVED] VARIANT_BOOL return value in Automation class

11-Jan-07 5:03

The return value is returned in VARIANT. I checked the debug memory window and played with it for a while in Visual Studio. The value is being returned from IDispatch::Invoke method. The value of VARIANT succesfully changes from VT_EMPTY to VT_BOOL and VARIANT::boolVal memory location changes from 0000 to ffff when I return VARIANT_FALSE from my server.
It seems like something is going wrong in the IDispatch::Invoke. I suspect the problem you mentioned could be the one present there.

If you are willing to check this at your end and ofcourse spare sometime, I can create simple samples and send it across to you and tell you the locations breakpoints. Let me know

S o h a i l K a d i w a l a

To Err Is Human; to Debug, Divine

modified 21-Apr-21 21:01pm.

User 21559711-Jan-07 6:07

Re: [SOLVED] VARIANT_BOOL return value in Automation class

11-Jan-07 6:07

SOLVED!!!

The problem lies in OleDisp1.cpp

I debugged my server using MFC source files. My CoClass implements IDispatch which is called from my client through COleDispatchDriver derived wrapper class.
So when call goes to following method -

BOOL IsWorking()<br />
{<br />
BOOL result;<br />
InvokeHelper(0x2, DISPATCH_METHOD, VT_BOOL, (void*)&result, NULL);<br />
return result;<br />
}

InvokeHelper will get called, which resides in OleDisp2.cpp.

From here we will finally reach, OleDisp2.cpp Line#375 where the actual call to IDispatch::Invoke will occur taking us to Server's IDispatch implementation

On server side, we will reach
File#OleDisp1.cpp, Line# 1220, Function# COleDispatchImpl::Invoke
Which will call
File# OleDisp1.cpp, Line# 908, Function# CCmdTarget::CallMemberFunc

In this function, the problem occurs somewhere at following locations -
Call# PushStackArgs(...) Line# 991

And Line# 1033

default:<br />
	dwResult = pfnDispatch(pfn, pStack, nSizeArgs);<br />
	break;

after the above call, the value of dwResult changes which results in incorrect value of VARIANT_BOOL being assigned and returned at Line# 1100

case VT_BOOL:<br />
	V_BOOL(pvarResult) = (VARIANT_BOOL)(dwResult != 0 ? -1 : 0);<br />
	break;

I think this clarifies my problem. What are your thoughts?

S o h a i l K a d i w a l a

To Err Is Human; to Debug, Divine

modified 21-Apr-21 21:01pm.

Roger Stoltz11-Jan-07 22:45

Re: [SOLVED] VARIANT_BOOL return value in Automation class

11-Jan-07 22:45

Sohail Kadiwala wrote:
SOLVED!!!

Is it, really? Unsure | :~

So what do you think the solution is?

The information you've provided suggests that the problem only resides in the server. Is this correctly understood?

Sohail Kadiwala wrote:
default:
dwResult = pfnDispatch(pfn, pStack, nSizeArgs);
break;
after the above call, the value of dwResult changes which results in incorrect value of VARIANT_BOOL being assigned and returned at Line# 1100
case VT_BOOL:
V_BOOL(pvarResult) = (VARIANT_BOOL)(dwResult != 0 ? -1 : 0);
break;

What happens here? What is the incorrect value of dwResult (all 32 bits)?
After the assignment above, what does pvarResult->boolVal contain?
If you modify the boolVal value after the assignment, does your client get the same value? This test is to make sure that the problem is only at the server side.

--
Rog

"It's supposed to be hard, otherwise anybody could do it!" - selfquote

User 21559712-Jan-07 0:50

Re: [SOLVED] VARIANT_BOOL return value in Automation class

12-Jan-07 0:50

Roger Stoltz wrote:
Is it, really?

Sorry, I should say - "problem area detected"

Roger Stoltz wrote:
So what do you think the solution is?

I haven't figured out the exact solution to this. I will need to dig into the MFC functions - one that pushes content on stack and the other is an assembly code. I'll try to do that over weekend.

Roger Stoltz wrote:
What is the incorrect value of dwResult (all 32 bits)?

the value of dwResult in memory changes from 00 00 00 00 to 00 00 14 00 after
default:
dwResult = pfnDispatch(pfn, pStack, nSizeArgs);

Roger Stoltz wrote:
After the assignment above, what does pvarResult->boolVal contain?

Since dwResult is now 00 00 14 00 = 5120 Decimal, the value of pvarResult->boolVal becomes -1 i.e. 0xFF. This should not happen because I am returning 0 in my server.

Roger Stoltz wrote:
If you modify the boolVal value after the assignment, does your client get the same value? This test is to make sure that the problem is only at the server side.

Do you mean modifying the memory location in debugging? Or modifying the source - which will mean rebuilding the MFC source tree? I think you mean modifying in memory window - I changed it from 0xFF to 0x00 and it returns 0x00 to the client.

S o h a i l K a d i w a l a

To Err Is Human; to Debug, Divine

modified 21-Apr-21 21:01pm.

Roger Stoltz12-Jan-07 1:50

Re: [SOLVED] VARIANT_BOOL return value in Automation class

12-Jan-07 1:50

Sohail Kadiwala wrote:
Roger Stoltz wrote:

Is it, really?

Sorry, I should say - "problem area detected"

Ok, but in my opinion you've only found where it goes wrong, not why. The root cause could still be somewhere else.

Sohail Kadiwala wrote:
the value of dwResult in memory changes from 00 00 00 00 to 00 00 14 00

This is rather funny how we can misinterpret each other, because now I have to ask if this is big-endian or little-endian. If you've copied this directly from the memory debug window it would be little-endian and in that case the value would be 0x00140000 which is 1310720 in decimal. I assume this is the case since it would make more sense to me. My point is that the lower 16 bits that may represent the VARIANT_BOOL type is still equal to VARIANT_FALSE.

Never mind, try this instead:
I've created a simple ActiveX control and a client with two methods returning VARIANT_TRUE and VARIANT_FALSE respectively as I've described it in the interface file (.idl).
It worked like expected until I explicitly changed the return types in my server from BOOL to VARIANT_BOOL. Then I got exactly the behaviour as you have. I don't even have to get marshalling involved: everything is still in the same apartment.
Try and change your return type back to BOOL in your .cpp and .h files, not in the .idl file. Rebuild and see what happens.
I haven't dug into exactly why I get this behaviour, but I suspect there's a mixup (or mismatch rather) between pointers to 16-bit types and 32-bit types as I've mentioned earlier.

Let me know what the results are.

--
Rog

"It's supposed to be hard, otherwise anybody could do it!" - selfquote

User 21559712-Jan-07 4:11

Re: [SOLVED] VARIANT_BOOL return value in Automation class

12-Jan-07 4:11

Roger Stoltz wrote:
Try and change your return type back to BOOL in your .cpp and .h files, not in the .idl file. Rebuild and see what happens.

Overwhelmed Smile | :)

So called it SOLVED.

Roger Stoltz wrote:
This is rather funny how we can misinterpret each other, because now I have to ask if this is big-endian or little-endian. If you've copied this directly from the memory debug window it would be little-endian and in that case the value would be 0x00140000 which is 1310720 in decimal. I assume this is the case since it would make more sense to me. My point is that the lower 16 bits that may represent the VARIANT_BOOL type is still equal to VARIANT_FALSE

Yeah, I thought about it Smile | :)

That is the reason why I copied the values exactly from my memory window, but forgot to mention about it Smile | :)

So it still remained ambiguos

Roger Stoltz wrote:
Ok, but in my opinion you've only found where it goes wrong, not why

This change has solved the problem! I now think the problem is in Wizard generated code! What do you think?

BTW thank you for followup and your in depth reply. my 5s for you Smile | :)

S o h a i l K a d i w a l a

To Err Is Human; to Debug, Divine

modified 21-Apr-21 21:01pm.

Roger Stoltz12-Jan-07 4:40

Re: [SOLVED] VARIANT_BOOL return value in Automation class

12-Jan-07 4:40

Sohail Kadiwala wrote:
That is the reason why I copied the values exactly from my memory window, but forgot to mention about it

I suspected that, but I got a little confused when you wrote that it evaluated to 5120 in decimal. Don't worry about it!

Sohail Kadiwala wrote:
I now think the problem is in Wizard generated code! What do you think?

Well, partly true.
I haven't found a way to make the wizard to use VARIANT_BOOL as type anywhere. That could be a problem and if so you have to explicitly modify it and that could be considered a "problem" in the wizard generated code.
But, I think the return types in your source code files (.cpp and .h) have been manually modified and if that's the case we cannot blame the wizard. :->

Sohail Kadiwala wrote:
thank you for followup and your in depth reply.

You are most welcome! Rose | [Rose]

I'm glad if I was able to help you.

Sohail Kadiwala wrote:
my 5s for you

Let me express my most humble gratitude for that. Blush | :O

All in all it feels good to leave for the weekend after nailing this one.

Have a nice weekend Sohail
--
Roger

"It's supposed to be hard, otherwise anybody could do it!" - selfquote

User 21559712-Jan-07 5:24

Re: [SOLVED] VARIANT_BOOL return value in Automation class

12-Jan-07 5:24

Roger Stoltz wrote:
Let me express my most humble gratitude for that.

Someone who sincerely helps should deserve atleast a 5 on CP Smile | :)

Roger Stoltz wrote:

Sohail Kadiwala wrote:
I now think the problem is in Wizard generated code! What do you think?

Well, partly true.
I haven't found a way to make the wizard to use VARIANT_BOOL as type anywhere. That could be a problem and if so you have to explicitly modify it and that could be considered a "problem" in the wizard generated code.
But, I think the return types in your source code files (.cpp and .h) have been manually modified and if that's the case we cannot blame the wizard.

I don't think I modified the source files (.H and .cpp). To confirm this, in my sample application, I added another COM method to my interface selecting the return type to be VARIANT_BOOL. The result was VARIANT_BOOL data type as the return type in .H and .CPP files.

Now, this makes me curios - What version of Visual Studio are you using?
I am using MSDE 2003 version 7.1.3088

Roger Stoltz wrote:
All in all it feels good to leave for the weekend after nailing this one.

Have a nice weekend

Yes, I am also feeling happy to nail this one, although I could still have moved ahead with my work with some alternate declaration. But with you around and eager to help, I pushed myself a bit harder. Its been a long time since I got stuck with such a time consuming problem - so I am feeling a lot happy nailing it.

You have a great weekend.

I still have an hour before I head home, came late for work so need to wait some more time completing stuff.

Warm Regards,

S o h a i l K a d i w a l a

To Err Is Human; to Debug, Divine

modified 21-Apr-21 21:01pm.

Roger Stoltz12-Jan-07 5:37

Re: [SOLVED] VARIANT_BOOL return value in Automation class

12-Jan-07 5:37

Sohail Kadiwala wrote:
I am using MSDE 2003 version 7.1.3088

That explains it. I'm still using VC6.

Sohail Kadiwala wrote:
I still have an hour before I head home, came late for work so need to wait some more time completing stuff.

Either you don't arrive at work until the afternoon or you're not located in India as your profile claims... I hope it's the latter one.

In Sweden the time is now 17:35 and I'm about to leave for the weekend in half an hour.

"It's supposed to be hard, otherwise anybody could do it!" - selfquote

User 21559714-Jan-07 7:50