How to conveniently convert a stream of byte into UNICODE?

[Jack]

Hi,
Just Looking for an handy way to convert
a stream of BYTE into UNICODE....
Does anyone know any of those?
Thanks
Jack

[Ulrich Eckhardt]

Post by Jack
Just Looking for an handy way to convert
a stream of BYTE into UNICODE....
Does anyone know any of those?

MultiByteToWideChar

Uli

[David Wilkinson]

Post by Jack
Hi,
Just Looking for an handy way to convert
a stream of BYTE into UNICODE....
Does anyone know any of those?

It depends what the stream of bytes is. Any file is a stream of bytes. If it is
text, it might be 8-bit text using the local code page (or some other code
page), it might be 8-bit UTF-8 text, it might already be 16-bit Unicode text, it
might be ... anything.

[Jack]

Hello,
Thanks both of you, seems it is going to be a hard time...
to convert the whole thing (have been trying for 2 days)
If it is a cocktail of 8-bit and 16-bit characters, would that be more
difficult to achieve or just like ulrich said, pass the whole bunch of data
into MultiByteToWideChar?
Thanks
Jack

[David Wilkinson]

Post by Jack
Hello,
Thanks both of you, seems it is going to be a hard time...
to convert the whole thing (have been trying for 2 days)
If it is a cocktail of 8-bit and 16-bit characters, would that be more
difficult to achieve or just like ulrich said, pass the whole bunch of data
into MultiByteToWideChar?

You can only use MultiByteToWideChar() if you know what the format of the source
is. If the source is a mixture of 16-bit and 8-bit characters (unlikely), I do
not see how you could make sense of it, unless the format of the contents were
very carefully documented.

[Jack]

Hello David,
For simplicity, now pretend I make the assumption of all of them are made up
of 16-bit characters,
What is the general method of converting the stream of BYTEs into foreign
characters?
Thanks
Jack

[Scot T Brennecke]

Post by Jack
Hello David,
For simplicity, now pretend I make the assumption of all of them are made up
of 16-bit characters,
What is the general method of converting the stream of BYTEs into foreign
characters?
Thanks
Jack

If they are 16-bit characters, then they are either already in Unicode, or DBCS, or some other non-ANSI string format. There is not
a one-size-fits-all answer here. You MUST know something about the current format of the BYTE stream, or conversion is a crapshoot.

If they are ANSI or MBCS strings in the BYTE stream, then you have already been told the answer: MultiByteToWideChar

[Ulrich Eckhardt]

Post by Jack
For simplicity, now pretend I make the assumption of all of them are made
up of 16-bit characters, What is the general method of converting the
stream of BYTEs into foreign characters?

Question up front: what is a foreign character? Anything but Latin letters
and Arabian numbers? Hebrew is much older than Latin, so isn't Latin the
foreign one then?

Anyway, back from philosophy to programming. Without knowing the encoding of
the byte data, there is no way to convert it to anything. Further, and that
is something you might have missed, Unicode (not UNICODE) is not a file
format or anything like that. Rather, it is a standard that also defines
several file or transfer formats (UTF = Unicode Transfer Format). Now, MS
Windows internally uses WCHAR, which encodes Unicode codepoints using
UTF-16.

If your file happens to be UTF-16 encoded, too, you can just memcpy() the
file content into a WCHAR array and you're done. If it happens to be the
big-endian variant of UTF-16, you will further have to swap every two
bytes. Also, if you want your code to run on anything non-MS Windows or on
a big-endian machine (which MS-Windows doesn't run on anyways), you will
have to perform further conversions.

BTW: It would help if you showed a short snippet of a hexdump of the file in
question including the text it should represent. It should then be easy to
figure out if it is one of the common encodings.

Good luck!

Uli

[Jack]

That is the most constructive advice I ever had
Thanks Ulrich.....
Jack