|
Hello everyone,
I need to know the wide character (unicode) and multibyte (UTF-8) values of a character string of czech. I personally know nothing about czech. Is the following approach correct?
1. I use L on the character string and watch memory to get the wide character representation of the character string in little endian form;
2. I change the computer region/language to czech, and use function WideCharToMultiByte, and use CP_ACP as input code page and use the L character string as input to get the output multibyte character string output from parameter lpMultiByteStr.
Is (1) and (2) correct? Any more efficient and smart ways?
thanks in advance,
George
|
|
|
|
|
try this
wcstombs function.
i think this will help you.
|
|
|
|
|
Hi Nitheesh,
Is this function relates to my question? I think I am using almost the same approach. Could you describe the steps to get wide character and multibyte value for a given czech character string please? 
regards,
George
|
|
|
|
|
Who gives you a czech string?
If the Lord God Almighty had consulted me before embarking upon the Creation, I would have recommended something simpler.
-- Alfonso the Wise, 13th Century King of Castile.
[my articles]
|
|
|
|
|
Hi CPallini,
I am parsing some information of multi-language. Could we come back to the original question please?
Any ideas?
regards,
George
|
|
|
|
|
Well, IMHO if you have the string then you either already have the Unicode or the Multibyte format of it, hence you need only one conversion.
BTW What's the difficult about examininig a character of a string?
If the Lord God Almighty had consulted me before embarking upon the Creation, I would have recommended something simpler.
-- Alfonso the Wise, 13th Century King of Castile.
[my articles]
|
|
|
|
|
Thanks CPallini,
Sorry that I may not make myself understood enough. I only have the literal format. The strings are something like MÍST, I want to get the wide character binary value and the multibyte binary value,
1. To get the wide character binary value, I use L"MÍST" and use debug mode to watch its internal buffer in Visual Studio.
2. To get the multibyte (UTF-8) binary value, I use WideCharacterToMultibyte API to convert L"MÍST" to multibyte value;
Is (1) and (2) correct solution?
regards,
George
|
|
|
|
|
George_George wrote: 1. To get the wide character binary value, I use L"MÍST" and use debug mode to watch its internal buffer in Visual Studio.
Fine. Since wide chars are unsigned shorts, I'm sure you're also able to figure out how programmatically find the encoded values.
George_George wrote: To get the multibyte (UTF-8) binary value, I use WideCharacterToMultibyte API to convert L"MÍST" to multibyte value;
When you use WideCharacterToMultibyte you must be aware that (of course) not all wide string characters can be mapped to the codepage you're specifying (I'm not an expert, but I think CP_UTF8 gives you have unmapped chars than Czech codepage (1250?))
If the Lord God Almighty had consulted me before embarking upon the Creation, I would have recommended something simpler.
-- Alfonso the Wise, 13th Century King of Castile.
[my articles]
|
|
|
|
|
Thanks CPallini,
1.
CPallini wrote: Fine. Since wide chars are unsigned shorts, I'm sure you're also able to figure out how programmatically find the encoded values.
I just use the unsigned short value itself (as binary hex value format). You mean we need additional conversion?
2.
CPallini wrote: but I think CP_UTF8 gives you have unmapped chars than Czech codepage (1250?))
Good point! I missed it. You mean using 1250 as code page value is better?
How do you find the magic value numebr 1250, in some Windows header (.h) file?
regards,
George
|
|
|
|
|
George_George wrote: How do you find the magic value numebr 1250, in some Windows header (.h) file?
here http://www.microsoft.com/globaldev/reference/WinCP.mspx[^], for instance.
If the Lord God Almighty had consulted me before embarking upon the Creation, I would have recommended something simpler.
-- Alfonso the Wise, 13th Century King of Castile.
[my articles]
|
|
|
|
|
Thanks CPallini,
Good link!
When using WideCharacterToMultiByte or using debugger to see the hex binary value of L string, there is no need to change the language setting of control panel, right?
regards,
George
|
|
|
|
|
As I already told you, I'm not an expert, anyway, I think you need to change the codepage on control panel only to render text (using multibyte encoding), hence no need for you.
If the Lord God Almighty had consulted me before embarking upon the Creation, I would have recommended something simpler.
-- Alfonso the Wise, 13th Century King of Castile.
[my articles]
|
|
|
|
|
Hi CPallini,
1.
CPallini wrote: hence no need for you.
Why no need for me? If I need to change language settings to get the correct value, I have to and should change.
2.
http://www.microsoft.com/globaldev/reference/WinCP.mspx
For the page you recommended before, there are two terms, SBCS and DBCS. When we map our term multibyte and wide character to them, is multibyte the same as SBCS and wide character the same as DBCS?
regards,
George
|
|
|
|
|
George_George wrote: Why no need for me? If I need to change language settings to get the correct value, I have to and should change.
I don't think so. I think WideCharToMultiByte function keep doing is honest work regardless of your control panel settings (provided you pass it the correct codepage as argument). Hence you can check WideCharToMultiByte result to find out character codes, however you cannot correctly render the latters without setting the proper codepage in your control panel.
George_George wrote: For the page you recommended before, there are two terms, SBCS and DBCS. When we map our term multibyte and wide character to them, is multibyte the same as SBCS and wide character the same as DBCS?
No. Wide character encoding corrensponds to UNICODE, while DBCS (and SBCS) correnspond to MultiByte.
(Maybe I'm wrong, though, I should tell you again, I'm not an expert about, my common sense is driving me on the argument).
If the Lord God Almighty had consulted me before embarking upon the Creation, I would have recommended something simpler.
-- Alfonso the Wise, 13th Century King of Castile.
[my articles]
|
|
|
|
|
Thanks CPallini!
My question is answered. Thanks for your patience again!
regards,
George
|
|
|
|
|
CPallini wrote: When you use WideCharacterToMultibyte you must be aware that (of course) not all wide string characters can be mapped to the codepage you're specifying (I'm not an expert, but I think CP_UTF8 gives you have unmapped chars than Czech codepage (1250?)
There is 1-1 mapping between UTF-16 (wide chars on Windows) and UTF-8 (both are simply different encoding forms for Unicode character set). On the other hand, there is no 1-1 mapping between Unicode and CP1250.
Conclusion? If you need UTF-8, convert directly from wide char to UTF-8, and don't play with legacy multibyte encodins such as CP1250
|
|
|
|
|
Thanks Nemanja,
Your reply is clear. For the same wide character string in Czech, I am not sure whether converting to Czech codepage (1250) or CP_UTF8 will making information lose ... i.e. resulting in ?? (0x3F) character. Any experiences?
regards,
George
|
|
|
|
|
George_George wrote: For the same wide character string in Czech, I am not sure whether converting to Czech codepage (1250) or CP_UTF8 will making information lose ... i.e. resulting in ?? (0x3F) character.
If you know for sure the wide string contains only Czech (or other Central Europian) characters, you can convert it to either CP1250 or UTF-8 without any loss. In any case, you don't need to mess with the system language settings - just use the correct codepage parameter in WideStringToMultiChar.
If you don't know for sure the wide string contains only Central Europian characters, you can still safely convert to UTF-8, but not to CP1250.
|
|
|
|
|
Great Nemanja!
But why not safe with CP1250?
Nemanja Trifunovic wrote: If you don't know for sure the wide string contains only Central Europian characters, you can still safely convert to UTF-8, but not to CP1250.
regards,
George
|
|
|
|
|
George_George wrote: But why not safe with CP1250?
If a wide string contains a non-Central Europian character, say - U+03A8 (Greek capital Psi); it has no representation in CP1250 and will appear as a replacement character (probably question mark) after the conversion.
|
|
|
|
|
Thanks for your clarification, Nemanja!
My question is answered.
regards,
George
|
|
|
|
|
Nemanja Trifunovic wrote: here is 1-1 mapping between UTF-16 (wide chars on Windows) and UTF-8 (both are simply different encoding forms for Unicode character set).
You're right indeed.
Nemanja Trifunovic wrote: On the other hand, there is no 1-1 mapping between Unicode and CP1250.
There is 1-1 mapping for the subset representing Czech characters, I suppose.
If the Lord God Almighty had consulted me before embarking upon the Creation, I would have recommended something simpler.
-- Alfonso the Wise, 13th Century King of Castile.
[my articles]
|
|
|
|
|
CPallini wrote: There is 1-1 mapping for the subset representing Czech characters, I suppose.
Correct. It is just not always easy to be sure that the wide string contains only the Czech subset.
|
|
|
|
|
George_George wrote: I need to know the wide character (unicode) and multibyte (UTF-8) values of a character string of czech.
Is this string provided as a user input, or you are dealing with a hard-coded string literal?
Anyway, to get a UTF-8 representation of a Unicode string, there is no need to change the system language. Either use WideCharToMultiByte with codepage set to CP_UTF8, or use a third-party library (like the one you see in my signature  )
|
|
|
|
|
Thanks Nemanja,
Nemanja Trifunovic wrote: Anyway, to get a UTF-8 representation of a Unicode string, there is no need to change the system language. Either use WideCharToMultiByte with codepage set to CP_UTF8, or use a third-party library (like the one you see in my signature )
1. Change system language settings before invoking WideCharToMultiByte, right?
2. Should I use UTF-8 code page or use some Czech code page?
regards,
George
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
