CCnvCharacterSetConverter Class Reference

#include <charconv.h>

Link against: charconv.lib

class CCnvCharacterSetConverter : public CBase

Inherits from

  • CCnvCharacterSetConverter
Public Member Enumerations
enumanonymous { EInputConversionFlagAppend, EInputConversionFlagAllowTruncatedInputNotEvenPartlyConsumable, EInputConversionFlagStopAtFirstUnconvertibleCharacter, EInputConversionFlagMustEndInDefaultCharacterSet, EInputConversionFlagAssumeStartInDefaultCharacterSet }
enumanonymous { EOutputConversionFlagInputIsTruncated }
enumanonymous { KStateDefault }
enumanonymous { ELowestThreshold }
enumTAvailability { EAvailable, ENotAvailable }
enumTDowngradeForExoticLineTerminatingCharacters { EDowngradeExoticLineTerminatingCharactersToCarriageReturnLineFeed, EDowngradeExoticLineTerminatingCharactersToJustLineFeed }
enumTEndianness { ELittleEndian, EBigEndian }
enumTError { EErrorIllFormedInput }
Public Member Functions
virtual ~CCnvCharacterSetConverter()
IMPORT_C const SCnvConversionData &AsciiConversionData()
IMPORT_C voidAutoDetectCharSetL(TInt &, TUint &, const CArrayFix< SCharacterSet > &, const TDesC8 &)
IMPORT_C voidAutoDetectCharacterSetL(TInt &, TUint &, const CArrayFix< SCharacterSet > &, const TDesC8 &)
IMPORT_C TIntConvertCharacterSetIdentifierToMibEnumL(TUint, RFs &)
IMPORT_C HBufC8 *ConvertCharacterSetIdentifierToStandardNameL(TUint, RFs &)
IMPORT_C TIntConvertFromUnicode(TDes8 &, const TDesC16 &)
IMPORT_C TIntConvertFromUnicode(TDes8 &, const TDesC16 &, TInt &)
IMPORT_C TIntConvertFromUnicode(TDes8 &, const TDesC16 &, TInt &, TInt &)
IMPORT_C TIntConvertFromUnicode(TDes8 &, const TDesC16 &, TArrayOfAscendingIndices &)
IMPORT_C TUintConvertMibEnumOfCharacterSetToIdentifierL(TInt, RFs &)
IMPORT_C TUintConvertStandardNameOfCharacterSetToIdentifierL(const TDesC8 &, RFs &)
IMPORT_C TIntConvertToUnicode(TDes16 &, const TDesC8 &, TInt &)
IMPORT_C TIntConvertToUnicode(TDes16 &, const TDesC8 &, TInt &, TInt &)
IMPORT_C TIntConvertToUnicode(TDes16 &, const TDesC8 &, TInt &, TInt &, TInt &)
IMPORT_C voidConvertibleToCharSetL(TInt &, const TUint, const CArrayFix< SCharacterSet > &, const TDesC8 &)
IMPORT_C voidConvertibleToCharacterSetL(TInt &, const TUint, const CArrayFix< SCharacterSet > &, const TDesC8 &)
IMPORT_C CArrayFix< SCharacterSet > *CreateArrayOfCharacterSetsAvailableL(RFs &)
IMPORT_C CArrayFix< SCharacterSet > *CreateArrayOfCharacterSetsAvailableLC(RFs &)
IMPORT_C TIntDoConvertFromUnicode(const SCnvConversionData &, TEndianness, const TDesC8 &, TDes8 &, const TDesC16 &, TArrayOfAscendingIndices &)
IMPORT_C TIntDoConvertFromUnicode(const SCnvConversionData &, TEndianness, const TDesC8 &, TDes8 &, const TDesC16 &, TArrayOfAscendingIndices &, TUint &, TUint)
IMPORT_C TIntDoConvertToUnicode(const SCnvConversionData &, TEndianness, TDes16 &, const TDesC8 &, TInt &, TInt &)
IMPORT_C TIntDoConvertToUnicode(const SCnvConversionData &, TEndianness, TDes16 &, const TDesC8 &, TInt &, TInt &, TUint &, TUint)
TDowngradeForExoticLineTerminatingCharacters GetDowngradeForExoticLineTerminatingCharacters()
IMPORT_C CCnvCharacterSetConverter *NewL()
IMPORT_C CCnvCharacterSetConverter *NewLC()
IMPORT_C voidPrepareToConvertToOrFromL(TUint, const CArrayFix< SCharacterSet > &, RFs &)
IMPORT_C TAvailabilityPrepareToConvertToOrFromL(TUint, RFs &)
IMPORT_C voidSetDefaultEndiannessOfForeignCharacters(TEndianness)
IMPORT_C voidSetDowngradeForExoticLineTerminatingCharacters(TDowngradeForExoticLineTerminatingCharacters)
IMPORT_C voidSetMaxCacheSize(TInt)
IMPORT_C voidSetReplacementForUnconvertibleUnicodeCharactersL(const TDesC8 &)
Inherited Functions
CBase::CBase()
CBase::Delete(CBase *)
CBase::Extension_(TUint,TAny *&,TAny *)
CBase::operator new(TUint)
CBase::operator new(TUint,TAny *)
CBase::operator new(TUint,TLeave)
CBase::operator new(TUint,TLeave,TUint)
CBase::operator new(TUint,TUint)
CBase::~CBase()

Detailed Description

Converts text between Unicode and other character sets.

The first stage of the conversion is to specify the non-Unicode character set being converted to or from. This is done by calling one of the overloads of PrepareToConvertToOrFromL().

The second stage is to convert the text, using one of the overloads of ConvertFromUnicode() or ConvertToUnicode().

Where possible the first documented overload of PrepareToConvertToOrFromL() should be used because the second overload panics if the specified character set is not available: the first overload simply returns whether the character set is available or not available. However if the conversions are to be performed often, or if the user must select the character set for the conversion from a list, the second overload may be more appropriate.

The first overload is less efficient than the second, because it searches through the file system for the selected character set every time it is invoked. The second overload searches through an array of all available character sets. In this method, the file system need only be searched once - when CreateArrayOfCharacterSetsAvailableLC() or CreateArrayOfCharacterSetsAvailableL() is used to create the array.

The conversion functions allow users of this class to perform partial conversions on an input descriptor, handling the situation where the input descriptor is truncated mid way through a multi-byte character. This means that you do not have to guess how big to make the output descriptor for a given input descriptor, you can simply do the conversion in a loop using a small output descriptor. The ability to handle truncated descriptors also allows users of the class to convert information received in chunks from an external source.

The class also provides a number of utility functions.

Member Enumeration Documentation

Enum anonymous

Output flag used to indicate whether or not a character in the source descriptor is the first half of a surrogate pair, but is the last character in the descriptor to convert.

Note: This enumeration can be used in the DoConvertToUnicode() and DoConvertFromUnicode() functions. These are part of the Character Conversion Plug-in Provider API and are for use by plug-in conversion libraries only.
Since
6.0
EnumeratorValueDescription
EInputConversionFlagAppend0x00010000

Appends the converted text to the output descriptor.

EInputConversionFlagAllowTruncatedInputNotEvenPartlyConsumable0x00020000

By default, when the input descriptor passed to DoConvertFromUnicode() or DoConvertToUnicode() consists of nothing but a truncated sequence, the error-code EErrorIllFormedInput is returned. If this behaviour is undesirable, the input flag EInputConversionFlagAllowTruncatedInputNotEvenPartlyConsumable should be set.

EInputConversionFlagStopAtFirstUnconvertibleCharacter0x00040000

Stops converting when the first unconvertible character is reached.

EInputConversionFlagMustEndInDefaultCharacterSet0x00080000

Appends the default character set Escape sequence at end of converted text

EInputConversionFlagAssumeStartInDefaultCharacterSet0x00100000

Enum anonymous

EnumeratorValueDescription
EOutputConversionFlagInputIsTruncated0x01000000

Indicates whether or not the source descriptor ends in a truncated sequence, e.g. the first half only of a surrogate pair.

Enum anonymous

Initial value for the state argument in a set of related calls to ConvertToUnicode().

EnumeratorValueDescription
KStateDefault0

Enum anonymous

EnumeratorValueDescription
ELowestThreshold25

The lowest confidence value for a character set accepted by Autodetect

Enum TAvailability

Indicates whether a character set is available or unavailable for conversion. Used by the second overload of PrepareToConvertToOrFromL().

EnumeratorValueDescription
EAvailable

The requested character set can be converted.

ENotAvailable

The requested character set cannot be converted.

Enum TDowngradeForExoticLineTerminatingCharacters

Downgrade for line and paragraph separators

EnumeratorValueDescription
EDowngradeExoticLineTerminatingCharactersToCarriageReturnLineFeed

Paragraph/line separators should be downgraded (if necessary) into carriage return and line feed pairs.

EDowngradeExoticLineTerminatingCharactersToJustLineFeed

Paragraph/line separators should be downgraded (if necessary) into a line feed only.

Enum TEndianness

Specifies the default endian-ness of the current character set. Used by SetDefaultEndiannessOfForeignCharacters().

EnumeratorValueDescription
ELittleEndian

The character set is big-endian.

EBigEndian

The character set is little-endian.

Enum TError

Conversion error flags. At this stage there is only one error flag- others may be added in the future.

EnumeratorValueDescription
EErrorIllFormedInputKErrCorrupt

The input descriptor contains a single corrupt character. This might occur when the input descriptor only contains some of the bytes of a single multi-byte character.

Constructor & Destructor Documentation

~CCnvCharacterSetConverter ( )

IMPORT_C~CCnvCharacterSetConverter()[virtual]

The destructor frees all resources owned by the object, prior to its destruction.

Member Function Documentation

AsciiConversionData ( )

IMPORT_C const SCnvConversionData &AsciiConversionData()[static]

Returns a ready-made SCnvConversionData object for converting between Unicode and ASCII. This can be passed into the aConversionData parameter to DoConvertFromUnicode() or DoConvertToUnicode().

Note: This utility function should only be called by a plug-in conversion library.

Return Value
ASCII conversion data object.

AutoDetectCharSetL ( TInt &, TUint &, const CArrayFix< SCharacterSet > &, const TDesC8 & )

IMPORT_C voidAutoDetectCharSetL(TInt &aConfidenceLevel,
TUint &aCharacterSetIdentifier,
const CArrayFix< SCharacterSet > &aArrayOfCharacterSetsAvailable,
const TDesC8 &aSample
)

Attempts to determine the character set of the sample text from those supported on the phone.

For each of the available character sets, its implementation of IsInThisCharacterSetL() is called. The character set which returns the highest confidence level (i.e. which generates the fewest 0xFFFD Unicode replacement characters) is returned in aCharacterSetIdentifier.

This function merely determines if the sample text is convertible with this converter: it does no textual analysis on the result. Therefore, this function is not capable of differentiating between very similar encodings (for example the different ISO 8859 variants).

Any code making use of this function should provide a way for the user to override the selection that this function makes.

Please note that the operation of this function is slow.It takes no account of the usual ontext that would be used in guessing a character set (for example, the language that is expected to be encoded or the transport used). For situations where such context is known, a faster, more accurate solution is advisable.

To improve a performance of autodetection, a size (default is one) of interface proxy cache should be increased (see SetCharacterSetCacheSize()).However a boost of performance will not be visible within a first funtion call because during this first call character sets are loaded to a cache. Once created it will be preserved until CCnvCharacterSetConverter object is destroyed.

This is a static function which uses ECOM functionality. It cleans up ECOM by calling FinalClose()

Parameters
aConfidenceLevelSet by the function to a value between 0 and 100. 0 indicates the function has no idea what character set aSample is encoded in. In this case, aCharacterSetIdentifier is undefined. 100 indicates total confidence that aCharacterSetIdentifier is the character set of aSample.
aCharacterSetIdentifierOn return, the UID of the best available character set for the sample text aSample. Character set UIDs are defined in charconv.h.
aArrayOfCharacterSetsAvailableThe array of character sets available on the phone. If this is not already available, it can be created using CreateArrayOfCharacterSetsAvailableL() or CreateArrayOfCharacterSetsAvailableLC().
aSampleThe non-Unicode sample text string.

AutoDetectCharacterSetL ( TInt &, TUint &, const CArrayFix< SCharacterSet > &, const TDesC8 & )

IMPORT_C voidAutoDetectCharacterSetL(TInt &aConfidenceLevel,
TUint &aCharacterSetIdentifier,
const CArrayFix< SCharacterSet > &aArrayOfCharacterSetsAvailable,
const TDesC8 &aSample
)[static]

Deprecated

See also: AutoDetectCharSetL

Since
8.1
Pre-condition
Post-condition

ConvertCharacterSetIdentifierToMibEnumL ( TUint, RFs & )

IMPORT_C TIntConvertCharacterSetIdentifierToMibEnumL(TUintaCharacterSetIdentifier,
RFs &aFileServerSession
)

Converts the UID of a character set to its MIB enum value.

If the character set identified is not one for which Symbian OS provides built-in conversion, the function searches the file system for plug-ins which implement the conversion and which provide the UID-to-MIB enum mapping information.

Parameters
aCharacterSetIdentifierThe UID of the character set.
aFileServerSessionConnection to a file server session.
Return Value
The MIB enum value of the character set if the character set with Identifier has a known name and MIB enum.Otherwise zero is returned.

ConvertCharacterSetIdentifierToStandardNameL ( TUint, RFs & )

IMPORT_C HBufC8 *ConvertCharacterSetIdentifierToStandardNameL(TUintaCharacterSetIdentifier,
RFs &aFileServerSession
)

Returns the Internet-standard name of a character set identified in Symbian OS by a UID.

If the character set specified is not one for which Symbian OS provides built-in conversion, the file system is searched for plug-ins which implement the conversion, hence the need for a file server session.

Parameters
aCharacterSetIdentifierThe UID of the character set.
aFileServerSessionA file server session.
Return Value
The Internet-standard name for the character set if the character set with an identifier has a known name and MIB enum.Otherwise NULL is returned.

ConvertFromUnicode ( TDes8 &, const TDesC16 & )

IMPORT_C TIntConvertFromUnicode(TDes8 &aForeign,
const TDesC16 &aUnicode
)const

Converts text encoded in the Unicode character set (UCS-2) into other character sets.

The first overload of the function simply performs the conversion. The second overload converts the text and gets the number of characters that could not be converted. The third overload converts the text, gets the number of characters that could not be converted, and also gets the index of the first character that could not be converted. A fourth overload was introduced in v6.0 see below.All overloads cause a panic if no target character set has been selected to convert to (i.e. either overload of PrepareToConvertToOrFromL() must have been successfully called beforehand). You may also need to call SetDefaultEndiannessOfForeignCharacters() to define the endian-ness of the output descriptor.Notes:A sixteen-bit descriptor is used to hold the source Unicode encoded text, and an eight-bit descriptor is used to hold the converted non-Unicode text. Eight-bit descriptors are used because non-Unicode character sets may use a single byte per character (e.g. Code Page 1252) or more than one byte per character (e.g. GB 2312-80) or even a variable number of bytes per character (e.g. Shift-JIS).The function will fail to convert all the input descriptor if the output descriptor is not long enough to hold all the text.Unicode characters cannot be converted if there is no equivalent for them in the target character set. This does not stop the conversion, the missing character is simply replaced by the character in the target character set which represents unknown characters. This default unknown character can be changed using SetReplacementForUnconvertibleUnicodeCharactersL().

Parameters
aForeignOn return, contains the converted text in a non-Unicode character set.
aUnicodeThe source Unicode text to be converted.
Return Value
The number of unconverted characters left at the end of the input descriptor (e.g. because the output descriptor is not long enough to hold all the text), or one of the error values defined in TError.

ConvertFromUnicode ( TDes8 &, const TDesC16 &, TInt & )

IMPORT_C TIntConvertFromUnicode(TDes8 &aForeign,
const TDesC16 &aUnicode,
TInt &aNumberOfUnconvertibleCharacters
)const

ConvertFromUnicode ( TDes8 &, const TDesC16 &, TInt &, TInt & )

IMPORT_C TIntConvertFromUnicode(TDes8 &aForeign,
const TDesC16 &aUnicode,
TInt &aNumberOfUnconvertibleCharacters,
TInt &aIndexOfFirstUnconvertibleCharacter
)const

Converts text encoded in the Unicode character set (UCS-2) into other character sets.

The first overload of the function simply performs the conversion. The second overload converts the text and gets the number of characters that could not be converted. The third overload converts the text, gets the number of characters that could not be converted, and also gets the index of the first character that could not be converted. A fourth overload was introduced in v6, see below.

All overloads cause a panic if no target character set has been selected to convert to (i.e. either overload of PrepareToConvertToOrFromL() must have been successfully called beforehand). You may also need to call SetDefaultEndiannessOfForeignCharacters() to define the endian-ness of the output descriptor.

Notes:

A sixteen-bit descriptor is used to hold the source Unicode encoded text, and an eight-bit descriptor is used to hold the converted non-Unicode text. Eight-bit descriptors are used because non-Unicode character sets may use a single byte per character (e.g. Code Page 1252) or more than one byte per character (e.g. GB 2312-80) or even a variable number of bytes per character (e.g. Shift-JIS).

The function will fail to convert all the input descriptor if the output descriptor is not long enough to hold all the text.

Unicode characters cannot be converted if there is no equivalent for them in the target character set. This does not stop the conversion, the missing character is simply replaced by the character in the target character set which represents unknown characters. This default unknown character can be changed using SetReplacementForUnconvertibleUnicodeCharactersL().

Parameters
aForeignOn return, contains the converted text in a non-Unicode character set.
aUnicodeThe source Unicode text to be converted.
aNumberOfUnconvertibleCharactersOn return contains the number of characters which could not be converted.
aIndexOfFirstUnconvertibleCharacterOn return, contains the index of the first character in the input text that could not be converted. The value is negative if all characters were converted.
Return Value
The number of unconverted characters left at the end of the input descriptor (e.g. because the output descriptor is not long enough to hold all the text), or one of the error values defined in TError.

ConvertFromUnicode ( TDes8 &, const TDesC16 &, TArrayOfAscendingIndices & )

IMPORT_C TIntConvertFromUnicode(TDes8 &aForeign,
const TDesC16 &aUnicode,
TArrayOfAscendingIndices &aIndicesOfUnconvertibleCharacters
)const

Converts Unicode text into another character set.

Differs from the other overloads of this function by returning the indices of all of the characters in the source Unicode text which could not be converted.

Parameters
aForeignOn return, contains the converted text in a non-Unicode character set.
aUnicodeThe source Unicode text to be converted.
aIndicesOfUnconvertibleCharactersOn return, holds the indices of each Unicode character in the source text which could not be converted.
Return Value
The number of unconverted characters left at the end of the input descriptor (e.g. because the output descriptor is not long enough to hold all the text), or one of the error values defined in TError.

ConvertMibEnumOfCharacterSetToIdentifierL ( TInt, RFs & )

IMPORT_C TUintConvertMibEnumOfCharacterSetToIdentifierL(TIntaMibEnumOfCharacterSet,
RFs &aFileServerSession
)

Converts a MIB enum value to the UID value of the character set.

If the character set identified is not one for which Symbian OS provides built-in conversion, the function searches the file system for plug-ins which implement the conversion and which provide the MIB enum-to-UID mapping information.

Parameters
aMibEnumOfCharacterSetThe MIB enum value of the character set.
aFileServerSessionConnection to a file server session.
Return Value
TThe UID of the character set if the character set with a Mib enum has a known name and MIB enum.Otherwise zero is returned.

ConvertStandardNameOfCharacterSetToIdentifierL ( const TDesC8 &, RFs & )

IMPORT_C TUintConvertStandardNameOfCharacterSetToIdentifierL(const TDesC8 &aStandardNameOfCharacterSet,
RFs &aFileServerSession
)

Gets the UID of a character set identified by its Internet-standard name (the matching is case-insensitive).

If the character set specified is not one for which Symbian OS provides built-in conversion, the function searches the file system for plug-ins which implement the conversion and which provide the name-to-UID mapping information.

Parameters
aStandardNameOfCharacterSetInternet-standard name of a character set.
aFileServerSessionConnection to a file server session.
Return Value
The UID for the character set if the character set with a Internet-standard name has a known name and MIB enum.Otherwise zero is returned.

ConvertToUnicode ( TDes16 &, const TDesC8 &, TInt & )

IMPORT_C TIntConvertToUnicode(TDes16 &aUnicode,
const TDesC8 &aForeign,
TInt &aState
)const

Converts text encoded in a non-Unicode character set into the Unicode character set (UCS-2).

The first overload of the function simply performs the conversion. The second overload converts the text and gets the number of bytes in the input string that could not be converted. The third overload converts the text, gets the number of bytes that could not be converted, and also gets the index of the first byte that could not be converted.All overloads cause a panic if no source character set has been selected to convert from (i.e. either overload of PrepareToConvertToOrFromL() must have been successfully called beforehand). You may also need to call SetDefaultEndiannessOfForeignCharacters() to define the endian-ness of the input descriptor.Notes: Since Unicode is intended to be the superset of all character sets, the function should usually report zero unconverted characters. Unconvertible characters will exist if the input descriptor contains illegal characters, i.e. values not in the selected non-Unicode character set.The presence of illegal characters does not stop the conversion. The missing character is simply replaced by the Unicode character which represents unknown characters (0xFFFD).If the source text consists solely of a character that is not complete, the function returns EErrorIllFormedInput. The reason for this is to prevent the possibility of the calling code getting into a infinite loop.

Parameters
aUnicodeOn return, contains the converted text in the Unicode character aForeign The non-Unicode source text to be converted.
aStateThis is used to save state information across multiple calls to ConvertToUnicode(). You should initialise the value to KStateDefault, and then do not change it in a series of related calls.
Return Value
The number of unconverted bytes left at the end of the input descriptor (e.g. because the output descriptor is not long enough to hold all the text), or one of the error values defined in TError.

ConvertToUnicode ( TDes16 &, const TDesC8 &, TInt &, TInt & )

IMPORT_C TIntConvertToUnicode(TDes16 &aUnicode,
const TDesC8 &aForeign,
TInt &aState,
TInt &aNumberOfUnconvertibleCharacters
)const

ConvertToUnicode ( TDes16 &, const TDesC8 &, TInt &, TInt &, TInt & )

IMPORT_C TIntConvertToUnicode(TDes16 &aUnicode,
const TDesC8 &aForeign,
TInt &aState,
TInt &aNumberOfUnconvertibleCharacters,
TInt &aIndexOfFirstByteOfFirstUnconvertibleCharacter
)const

Converts text encoded in a non-Unicode character set into the Unicode character set (UCS-2).

The first overload of the function simply performs the conversion. The second overload converts the text and gets the number of bytes in the input string that could not be converted. The third overload converts the text, gets the number of bytes that could not be converted, and also gets the index of the first byte that could not be converted.

All overloads cause a panic if no source character set has been selected to convert from (i.e. either overload of PrepareToConvertToOrFromL() must have been successfully called beforehand). You may also need to call SetDefaultEndiannessOfForeignCharacters() to define the endian-ness of the input descriptor.

Notes:

Since Unicode is intended to be the superset of all character sets, the function should usually report zero unconverted characters. Unconvertible characters will exist if the input descriptor contains illegal characters, i.e. values not in the selected non-Unicode character set.

The presence of illegal characters does not stop the conversion. The missing character is simply replaced by the Unicode character which represents unknown characters (0xFFFD).

If the source text consists solely of a character that is not complete, the function returns EErrorIllFormedInput. The reason for this is to prevent the possibility of the calling code getting into a infinite loop.

Parameters
aUnicodeOn return, contains the converted text in the Unicode character set.
aForeignThe non-Unicode source text to be converted.
aStateThis is used to save state information across multiple calls to ConvertToUnicode(). You should initialise the value to KStateDefault, and then do not change it in a series of related calls.
aNumberOfUnconvertibleCharactersOn return, contains the number of bytes which were not converted.
aIndexOfFirstByteOfFirstUnconvertibleCharacterOn return, the index of the first byte of the first unconvertible character. For instance if the first character in the input descriptor (aForeign) could not be converted, then this parameter is set to the first byte of that character, i.e. zero. A negative value is returned if all the characters were converted.
Return Value
The number of unconverted bytes left at the end of the input descriptor (e.g. because the output descriptor is not long enough to hold all the text), or one of the error values defined in TError.

ConvertibleToCharSetL ( TInt &, const TUint, const CArrayFix< SCharacterSet > &, const TDesC8 & )

IMPORT_C voidConvertibleToCharSetL(TInt &aConfidenceLevel,
const TUintaCharacterSetIdentifier,
const CArrayFix< SCharacterSet > &aArrayOfCharacterSetsAvailable,
const TDesC8 &aSample
)

Given a character set UID aCharacterSetIdentifier, ConvertibleToCharacterSetL returns the likelihood that aSample is encoded in that character set. It goes through the array of character sets aArrayOfCharacterSetsAvailable and searches for the character set matching aCharacterSetIdentifier. The character sets IsInThisCharacterSetL function is called to determine the probability of it being encoded in that character set.

This is a static function which uses ECOM functionality. It cleans up ECOM by calling FinalClose()

Since
7.0
Parameters
aConfidenceLevelSet by the function to a value between 0 and 100. It indicates the likelihood that aSample is encoded in aCharacterSetIdentifier.
aCharacterSetIdentifierthe likelihood of aSample being encoded in that character set.
aArrayOfCharacterSetsAvailableThe array of character sets available on the device. If this is not already available, it can be created using CreateArrayOfCharacterSetsAvailableL() or CreateArrayOfCharacterSetsAvailableLC().
aSampleThe non-Unicode sample text string.

ConvertibleToCharacterSetL ( TInt &, const TUint, const CArrayFix< SCharacterSet > &, const TDesC8 & )

IMPORT_C voidConvertibleToCharacterSetL(TInt &aConfidenceLevel,
const TUintaCharacterSetIdentifier,
const CArrayFix< SCharacterSet > &aArrayOfCharacterSetsAvailable,
const TDesC8 &aSample
)[static]

Deprecated

See also: ConvertibleToCharSetL

Since
8.1
Pre-condition
Post-condition

CreateArrayOfCharacterSetsAvailableL ( RFs & )

IMPORT_C CArrayFix< SCharacterSet > *CreateArrayOfCharacterSetsAvailableL(RFs &aFileServerSession)[static]

Creates an array identifying all the character sets for which conversion is available. These can be character sets for which conversion is built into Symbian OS, or they may be character sets for which conversion is implemented by a plug-in DLL.

The array returned can be used by one of the PrepareToConvertToOrFromL() overloads to provide a list of all the character sets available for conversion. The caller of this function is responsible for deleting the array, and should not modify it.

Not all encoders returned will be suitable for conversion from Unicode. Such encoders have no name and no MIB enum and so will generally not be understood by a receiving process. The function ConvertCharacterSetIdentifierToMibEnumL can be used to determine whether this is the case or not.

Parameters
aFileServerSessionA file server session. This is no longer used, but is kept for Binary Compatibility.
Return Value
An array identifying all supported character sets.

CreateArrayOfCharacterSetsAvailableLC ( RFs & )

IMPORT_C CArrayFix< SCharacterSet > *CreateArrayOfCharacterSetsAvailableLC(RFs &aFileServerSession)[static]

Creates an array identifying all the character sets for which conversion is available and pushes a pointer to it onto the cleanup stack. These can be character sets for which conversion is built into Symbian OS, or they may be character sets for which conversion is implemented by a plug-in DLL.

The array returned can be used by one of the PrepareToConvertToOrFromL() overloads to provide a list of all the character sets available for conversion. The caller of this function is responsible for deleting the array, and should not modify it.

Not all encoders returned will be suitable for conversion from Unicode. Such encoders have no name and no MIB enum and so will generally not be understood by a receiving process. The function ConvertCharacterSetIdentifierToMibEnumL can be used to determine whether this is the case or not.

This is a static function which uses ECOM functionality. It cleans up ECOM by calling FinalClose()

Parameters
aFileServerSessionA file server session. This is no longer required, kept for Binary Compatibilty.
Return Value
An array of references to all supported character sets.

DoConvertFromUnicode ( const SCnvConversionData &, TEndianness, const TDesC8 &, TDes8 &, const TDesC16 &, TArrayOfAscendingIndices & )

IMPORT_C TIntDoConvertFromUnicode(const SCnvConversionData &aConversionData,
TEndiannessaDefaultEndiannessOfForeignCharacters,
const TDesC8 &aReplacementForUnconvertibleUnicodeCharacters,
TDes8 &aForeign,
const TDesC16 &aUnicode,
TArrayOfAscendingIndices &aIndicesOfUnconvertibleCharacters
)[static]

Converts Unicode text into another character set. The Unicode text specified in aUnicode is converted using the conversion data object (aConversionData) provided by the plug-in for the foreign character set, and the converted text is returned in aForeign.

Note

This is a utility function that should only be called from a plug-in conversion library's implementation of ConvertFromUnicode(). Users of the Character Conversion API should use one of the overloads of ConvertFromUnicode() instead.

Parameters
aConversionDataThe conversion data object. Typically, you should specify conversionData, as declared in convgeneratedcpp.h. This is the SCnvConversionData object which is created in the cnvtool-generated .cpp file (although for some complex character sets you may want to pass other SCnvConversionData objects into this parameter).
aDefaultEndiannessOfForeignCharactersThe default endian-ness to use when writing the characters in the foreign character set. If an endian-ness for foreign characters is specified in aConversionData (i.e. not SCnvConversionData::EUnspecified), then that value is used and the value of aDefaultEndiannessOfForeignCharacters is ignored.
aReplacementForUnconvertibleUnicodeCharactersThe single character which is to be used to replace unconvertible characters.
aForeignOn return, contains the converted text in a non-Unicode character set.
aUnicodeThe source Unicode text to be converted.
aIndicesOfUnconvertibleCharactersOn return holds the indices of each Unicode character in the source text which could not be converted (because the target character set does not have an equivalent character).
Return Value
The number of unconverted characters left at the end of the input descriptor (e.g. because aForeign was not long enough to hold all the text), or a negative error value, as defined in TError.

DoConvertFromUnicode ( const SCnvConversionData &, TEndianness, const TDesC8 &, TDes8 &, const TDesC16 &, TArrayOfAscendingIndices &, TUint &, TUint )

IMPORT_C TIntDoConvertFromUnicode(const SCnvConversionData &aConversionData,
TEndiannessaDefaultEndiannessOfForeignCharacters,
const TDesC8 &aReplacementForUnconvertibleUnicodeCharacters,
TDes8 &aForeign,
const TDesC16 &aUnicode,
TArrayOfAscendingIndices &aIndicesOfUnconvertibleCharacters,
TUint &aOutputConversionFlags,
TUintaInputConversionFlags
)[static]

Converts Unicode text into another character set. The Unicode text specified in aUnicode is converted using the conversion data object (aConversionData) provided by the plug-in for the foreign character set, and the converted text is returned in aForeign.

This overload differs from the previous one in that it allows the caller to specify flags which give more control over the conversion.

Note

This is a utility function that should only be called from a plug-in conversion library's implementation of ConvertFromUnicode(). Users of the Character Conversion API should use one of the overloads of ConvertFromUnicode() instead.

Parameters
aConversionDataThe conversion data object. Typically, you should specify conversionData, as declared in convgeneratedcpp.h. This is the SCnvConversionData object which is created in the cnvtool-generated .cpp file (although for some complex character sets you may want to pass other SCnvConversionData objects into this parameter).
aDefaultEndiannessOfForeignCharactersThe default endian-ness to use when writing the characters in the foreign character set. If an endian-ness for foreign characters is specified in aConversionData (i.e. not SCnvConversionData::EUnspecified), then that value is used and the value of aDefaultEndiannessOfForeignCharacters is ignored.
aReplacementForUnconvertibleUnicodeCharactersThe single character which is to be used to replace unconvertible characters. If aInputConversionFlags is set to EInputConversionFlagStopAtFirstUnconvertibleCharacter, this replacement character is used to replace the first unconvertible character, then the conversion will stop.
aForeignOn return, contains the converted text in a non-Unicode character set. This may already contain some text. If it does, and if aInputConversionFlags specifies EInputConversionFlagAppend, then the converted text is appended to this descriptor.
aUnicodeThe source Unicode text to be converted.
aIndicesOfUnconvertibleCharactersOn return holds the indices of each Unicode character in the source descriptor aUnicode which could not be converted. The maximum size of this array is KMaximumNumberOfIndices whose value is 25 now. so only the first 25 unconvertible characters will be recorded. (because the target character set does not have an equivalent character).
aOutputConversionFlagsIf the input descriptor ended in a truncated sequence, e.g. the first half of a Unicode surrogate pair, aOutputConversionFlags returns with the EOutputConversionFlagInputIsTruncated flag set.
aInputConversionFlagsSpecify EInputConversionFlagAppend to append the text in aUnicode to aForeign. Specify EInputConversionFlagStopAtFirstUnconvertibleCharacter to stop converting when the first unconvertible character is reached. Specify EInputConversionFlagAllowTruncatedInputNotEvenPartlyConsumable to prevent the function from returning the error-code EErrorIllFormedInput when the input descriptor consists of nothing but a truncated sequence.
Return Value
The number of unconverted characters left at the end of the input descriptor (e.g. because aForeign was not long enough to hold all the text), or a negative error value, as defined in TError.

DoConvertToUnicode ( const SCnvConversionData &, TEndianness, TDes16 &, const TDesC8 &, TInt &, TInt & )

IMPORT_C TIntDoConvertToUnicode(const SCnvConversionData &aConversionData,
TEndiannessaDefaultEndiannessOfForeignCharacters,
TDes16 &aUnicode,
const TDesC8 &aForeign,
TInt &aNumberOfUnconvertibleCharacters,
TInt &aIndexOfFirstByteOfFirstUnconvertibleCharacter
)[static]

Converts non-Unicode text into Unicode. The non-Unicode text specified in aForeign is converted using the conversion data object (aConversionData) provided by the plug-in for the foreign character set, and the converted text is returned in aUnicode.

Notes:

This is a utility function that should only be called from a plug-in conversion library's implementation of ConvertToUnicode(). Ordinary users of the Character Conversion API should use one of the overloads of ConvertToUnicode() instead.

The last two arguments return information about unconverted characters. Because Unicode is intended to cover all possible characters, these rarely report anything other than zero characters. However they report the existence of unconvertible characters if the input descriptor aForeign contains illegal characters, i.e. values not in the foreign character set.

Parameters
aConversionDataThe conversion data object. Typically, you should specify conversionData, as declared in convgeneratedcpp.h. This is the SCnvConversionData object which is created in the cnvtool-generated .cpp file (although for some complex character sets you may want to pass other SCnvConversionData objects into this parameter).
aDefaultEndiannessOfForeignCharactersThe default endian-ness of the foreign characters. If an endian-ness for foreign characters is specified in aConversionData, then that is used instead and the value of aDefaultEndiannessOfForeignCharacters is ignored.
aUnicodeOn return, contains the text converted into Unicode.
aForeignThe non-Unicode source text to be converted.
aNumberOfUnconvertibleCharactersOn return, contains the number of characters in aForeign which were not converted. Characters which cannot be converted are output as Unicode replacement characters (0xFFFD).
aIndexOfFirstByteOfFirstUnconvertibleCharacterOn return, the index of the first byte of the first unconvertible character. For instance if the first character in the input descriptor (aForeign) could not be converted, then this parameter is set to the first byte of that character, i.e. zero. A negative value is returned if all the characters were converted.
Return Value
The number of unconverted bytes left at the end of the input descriptor, or a negative error value, as defined in TError.

DoConvertToUnicode ( const SCnvConversionData &, TEndianness, TDes16 &, const TDesC8 &, TInt &, TInt &, TUint &, TUint )

IMPORT_C TIntDoConvertToUnicode(const SCnvConversionData &aConversionData,
TEndiannessaDefaultEndiannessOfForeignCharacters,
TDes16 &aUnicode,
const TDesC8 &aForeign,
TInt &aNumberOfUnconvertibleCharacters,
TInt &aIndexOfFirstByteOfFirstUnconvertibleCharacter,
TUint &aOutputConversionFlags,
TUintaInputConversionFlags
)[static]

Converts non-Unicode text into Unicode. The non-Unicode text specified in aForeign is converted using the conversion data object (aConversionData) provided by the plug-in for the foreign character set, and the converted text is returned in aUnicode.

This overload differs from the previous one in that it allows the caller to specify flags which give more control over the conversion.

Notes:

This is a utility function that should only be called from a plug-in conversion library's implementation of ConvertToUnicode(). Ordinary users of the Character Conversion API should use one of the overloads of ConvertToUnicode() instead.

The aNumberOfUnconvertibleCharacters and aIndexOfFirstByteOfFirstUnconvertibleCharacter arguments return information about unconverted characters. Because Unicode is intended to cover all possible characters, these rarely report anything other than zero characters. However they report the existence of unconvertible characters if the input descriptor aForeign contains illegal characters, i.e. values not in the foreign character set.

Parameters
aConversionDataThe conversion data object. Typically, you should specify conversionData, as declared in convgeneratedcpp.h. This is the SCnvConversionData object which is created in the cnvtool-generated .cpp file (although for some complex character sets you may want to pass other SCnvConversionData objects into this parameter).
aDefaultEndiannessOfForeignCharactersThe default endian-ness of the foreign characters. If an endian-ness for foreign characters is specified in aConversionData, then that is used instead and the value of aDefaultEndiannessOfForeignCharacters is ignored.
aUnicodeOn return, contains the text converted into Unicode.
aForeignThe non-Unicode source text to be converted.
aNumberOfUnconvertibleCharactersOn return, contains the number of characters in aForeign which were not converted. Characters which cannot be converted are output as Unicode replacement characters (0xFFFD).
aIndexOfFirstByteOfFirstUnconvertibleCharacterOn return, the index of the first byte of the first unconvertible character. For instance if the first character in the input descriptor (aForeign) could not be converted, then this parameter is set to the first byte of that character, i.e. zero. A negative value is returned if all the characters were converted.
aOutputConversionFlagsIf the input descriptor ended in a truncated sequence, e.g. a multi-byte character that is not complete, aOutputConversionFlags returns with the EOutputConversionFlagInputIsTruncated flag set.
aInputConversionFlagsSpecify EInputConversionFlagAppend to append the converted text to aUnicode, otherwise the contents of aUnicode are overwritten. Specify EInputConversionFlagStopAtFirstUnconvertibleCharacter to stop converting when the first unconvertible character is reached. Specify EInputConversionFlagAllowTruncatedInputNotEvenPartlyConsumable to prevent the function from returning the error-code EErrorIllFormedInput when the input descriptor consists of nothing but a truncated sequence.
Return Value
The number of unconverted bytes left at the end of the input descriptor, or a negative error value defined in TError.

GetDowngradeForExoticLineTerminatingCharacters ( )

TDowngradeForExoticLineTerminatingCharacters GetDowngradeForExoticLineTerminatingCharacters()[inline]

NewL ( )

IMPORT_C CCnvCharacterSetConverter *NewL()[static]

Allocates and constructs a CCnvCharacterSetConverter object. If there is insufficient memory to create the object, the function leaves.

Since the memory is allocated on the heap, objects of this type should be destroyed using the delete operator when the required conversions are complete.

Return Value
The newly created object.

NewLC ( )

IMPORT_C CCnvCharacterSetConverter *NewLC()[static]

Allocates and constructs a CCnvCharacterSetConverter object, and leaves the object on the cleanup stack. If there is insufficient memory to create the object, the function leaves.

Since the memory is allocated on the heap, objects of this type should be destroyed using either the CleanupStack::Pop() function and then the delete operator, or the CleanupStack::PopAndDestroy() function.

Return Value
The newly created object.

PrepareToConvertToOrFromL ( TUint, const CArrayFix< SCharacterSet > &, RFs & )

IMPORT_C voidPrepareToConvertToOrFromL(TUintaCharacterSetIdentifier,
const CArrayFix< SCharacterSet > &aArrayOfCharacterSetsAvailable,
RFs &aFileServerSession
)

Specifies the character set to convert to or from. aCharacterSetIdentifier is a UID which identifies a character set. It can be one of the character sets for which conversion is built into Symbian OS, or it may be a character set for which the conversion is implemented by a plug-in DLL.

The function searches the character set array specified (aArrayOfCharacterSetsAvailable). This is an array containing all of the character sets for which conversion is available. It is created by calling CreateArrayOfCharacterSetsAvailableL() or CreateArrayOfCharacterSetsAvailableLC(). You should be sure that conversion is available for aCharacterSetIdentifier, because if not, a panic occurs. Otherwise, use the other overload of this function.

Either this function or its overload, must be called before using the conversion functions ConvertFromUnicode() or ConvertToUnicode().

Unlike the other overload, this function does not search the file system for plug-in conversion DLLs, (unless aArrayOfCharacterSetsAvailable is NULL). This function should be used if conversions are to be performed often, or if the conversion character set is to be selected by the user. Generating the array of all the available character sets once and searching though it is more efficient than the method used by the other overload, in which the file system may be searched every time it is invoked.

Notes:

The file server session argument is used to open the required character set conversion data file.

The array passed to this function can also be used to provide a list from which a user can select the desired conversion character set.

Parameters
aCharacterSetIdentifierThe UID of the non-Unicode character set from or to which to convert. Must not be zero, or a panic occurs.
aArrayOfCharacterSetsAvailableArray of all character sets for which conversion is available, created by either CreateArrayOfCharacterSetsAvailableLC() or CreateArrayOfCharacterSetsAvailableL().
aFileServerSessionA file server session. No longer used, kept for Binary Compatibility

PrepareToConvertToOrFromL ( TUint, RFs & )

IMPORT_C TAvailabilityPrepareToConvertToOrFromL(TUintaCharacterSetIdentifier,
RFs &aFileServerSession
)

Specifies the character set to convert to or from. aCharacterSetIdentifier is a UID which identifies a character set. It can be one of the character sets for which conversion is built into Symbian OS, or it may be a character set for which conversion is implemented by a plug-in DLL. In the latter case, the function searches through the file system for the DLL which implements the character conversion.

Either this function or its overload must be called before using the conversion functions ConvertFromUnicode() or ConvertToUnicode().

This overload of the function is simpler to use than the other and does not panic if the character set with the specified UID is not available at run time, it simply returns ENotAvailable. It should be used when the conversion character set is specified within the text object being converted, e.g. an email message, or an HTML document. If the character set is not specified, the user must be presented with a list of all available sets, so it makes sense to use the other overload.

The function may need to search the file system each time it is called. If conversion takes place repeatedly over a short period, it may be more efficient to use the other overload.

Notes:

Although the other overload of this function is more efficient, if the character set is one for which conversion is built into Symbian OS, the difference in speed is negligible.

Parameters
aCharacterSetIdentifierThe UID of the non-Unicode character set from or to which to convert. Must not be zero, or a panic occurs.
aFileServerSessionA file server session. No longer used, kept for Binary Compatibility
Return Value
The availability of the specified character set. If EAvailable is returned, then the conversion functions ConvertToUnicode() and ConvertFromUnicode() will use aCharacterSetIdentifier as the foreign character set. If ENotAvailable is returned, then the foreign character set will either be undefined (and trying to use the conversion functions will cause a panic), or if it has previously been set, it will remain unchanged.

SetDefaultEndiannessOfForeignCharacters ( TEndianness )

IMPORT_C voidSetDefaultEndiannessOfForeignCharacters(TEndiannessaEndianness)

Sets the default endian-ness used by the ConvertFromUnicode() and ConvertToUnicode() functions to convert between Unicode and non-Unicode character sets.

The endian-ness of a multi-byte character set may be defined in the character set definition or, as in the case of UCS-2, be operating system dependent. If the endian-ness of the current character set is defined by the character set itself, then the default endian-ness specified by this function is ignored.

Notes:

The issue of endian-ness does not apply to single byte character sets as there is no byte order.

This function should be called (if at all) after calling PrepareToConvertToOrFromL() and before calling ConvertFromUnicode() and/or ConvertToUnicode().

Parameters
aEndiannessThe default endian-ness of the current character set.

SetDowngradeForExoticLineTerminatingCharacters ( TDowngradeForExoticLineTerminatingCharacters )

IMPORT_C voidSetDowngradeForExoticLineTerminatingCharacters(TDowngradeForExoticLineTerminatingCharactersaDowngradeForExoticLineTerminatingCharacters)

Sets whether the Unicode 'line separator' and 'paragraph separator' characters (0x2028 and 0x2029 respectively) should be converted into a carriage return / line feed pair, or into a line feed only when converting from Unicode into a foreign character set. This applies to all foreign character sets that do not contain a direct equivalent of these Unicode character codes.

By default, line and paragraph separators are converted into a CR/LF pair. This function should be called (if at all) after calling PrepareToConvertToOrFromL() and before calling ConvertFromUnicode() and/or ConvertToUnicode().

Parameters
aDowngradeForExoticLineTerminatingCharactersSpecify EDowngradeExoticLineTerminatingCharactersToCarriageReturnLineFeed if line/paragraph separators should be converted into a carriage return and line feed combination and EDowngradeExoticLineTerminatingCharactersToJustLineFeed if they should be converted into line feeds only. Any other value causes the function to panic.

SetMaxCacheSize ( TInt )

IMPORT_C voidSetMaxCacheSize(TIntaSize)
The method sets the max size of the internal character set converter cache. The cache is used mainly to improve the performance of AutoDetectCharSetL() calls. It caches loaded converter implementations. The next time when a specific implementation is needed, a search will be done in the cache if this implementation is already loaded and if it is there, the cached implementation will be used. SetMaxCacheSize() call is used to limit the max cache size, because the loaded implementatiions may consume a lot of the system resources (memory for example). By default (if SetMaxCacheSize() is never called) the max cache size is limited to 32 entries. Note: Setting very small cache size will impact the overall performance of CHARCONV functions. If the choosen cache size is less than the number of existing character set converter implementations, there will be no performance gain or it will be far beyond client's expectations. For best performance the choosen cache size should be bigger or equal to the number of the existing character set converter implementations.

See also: CCnvCharacterSetConverter::AutoDetectCharSetL

Parameters
aSizeThe new max cache size. It must be bigger or equal to 4.
Panic Codes
ifthe new max cache size is less than 4.

SetReplacementForUnconvertibleUnicodeCharactersL ( const TDesC8 & )

IMPORT_C voidSetReplacementForUnconvertibleUnicodeCharactersL(const TDesC8 &aReplacementForUnconvertibleUnicodeCharacters)

Sets the character used to replace unconvertible characters in the output descriptor, when converting from Unicode into another character set.

The default replacement for unconvertible Unicode characters is specified in the conversion data for the character set. The replacement text which is set using this function overrides the default value.

Notes:

If the replacement character is multi-byte, and its endian-ness is undefined in the character set, then its byte order is taken by default to be little-endian.

PrepareToConvertToOrFromL() undoes the effect of any previous calls to this function. So, to have any effect, this function should be called between the PrepareToConvertToOrFromL() call and the subsequent ConvertFromUnicode() call or calls.

The value only applies when converting from Unicode to another character set. In Unicode, the code for 'unknown character'is always 0xFFFD.

Parameters
aReplacementForUnconvertibleUnicodeCharactersThe single character which is to be used to replace unconvertible characters.