Programming Cairo Text Output Beyond the 'toy' text API (C# X11) - A Proof of Concept

• Download the complete projects for 32 bit and 64 bit - 599 KB

Introduction

I became more and more unhappy with the text drawing capabilities of Xlib/X11, I do heavily use for my Roma Widget Set (C# X11) project. Especially the laborious internationalization of strings and the lack of antialiasing became an increasing limitation to the project.

Fortunately, Cairo and Pango libraries provide a professional text output, that is based on UTF-8 encoded strings and provide a lot of cool display features - including antialiasing, gradients and outline. Both libraries have been integrated into most Linux distributions.

The GTK+ UI toolkit uses Cairo to render the majority of its controls, starting with version 2.8 from 2005, and it also uses Pango for its text rendering, starting with version 2.0 from 2002.

Background

Since the Mono.Cairo package already wraps the Cairo C API, it is obvious to try text drawing using this package. Unfortunately, only the 'toy' text API is provided by Mono.Cairo. Although Cairo.Context.ShowText() and Cairo.Context.TextExtents() provide a lot of cool text display features - including antialiasing - the functionality is insufficient because it processes the text to draw always at once.

Usage of Cairo.Context.ShowGlyphs() and Cairo.Context.GlyphsExtents() can overcome this limitation, but there is no Mono.Cairo method to convert UTF-8 strings into glyphs. And this type of convertion is a hard job to do:

• Not only that glyph indices differ from font family to font family (see the application screenshot and take a look at the lines "'Luxi sans' writing three '36' glyphs: AAA" and "'Utopia' writing three '33' glyphs: AAA" - the same output "AAA" but different glyph indices),
• the glyphs inside a font family are organized in a cmap and a cmap can have different formats (search the web for "character to glyph mapping" or "OpenFont glyph" to get more information).

Typically, this conversion is the job of the Pango library, but there is no Pango wrapper package available for Mono, that provides Pango separately. Instead, Pango is highly integrated into the gtk-sharp package.

Since my Roma Widget Set project should completely avoid any GTK+ stuff (not because it is bad, only to prevent a competition to Gtk#), I had to find a different way to convert UTF-8 strings into glyphs. Ironically, the C# sources for Cairo contained in the gtk-sharp package lead me to a feasible solution.

Using the Code

The sample application was written with Mono Develop 2.4.1 for Mono 2.8.1 on OPEN SUSE 11.3 Linux 32 bit EN and GNOME desktop. Neither the port to any older nor to any newer version should be a problem. The sample application's solution consists of one project containing all the necessary source code.

The sample application is also tested with Mono Develop 3.0.6 for Mono 3.0.4 on OPEN SUSE 12.3 Linux 64 bit DE and GNOME desktop, IceWM, TWM und Xfce.

The Xlib/X11 window handling is based on the X11Wrapper assembly version 0.5, that defines the function prototypes, structures and types for Xlib/X11 calls to the libX11.so. It has been developed for the Programming Xlib with Mono Develop - Part 1: Low-level (proof of concept) project and has been advanced during the Programming the Roma Widget Set (C# X11) - a zero dependency GUI application framework - Part 1, Basics project.

The sample application shows some text output using:

• Cairo's 'toy' text API
• a self-provided wrapper around basic Cairo 'toy' text API functions (like cairo_get_current_transformation_matrix(), cairo_set_current_transformation_matrix(), cairo_set_source_rgba(), cairo_move_to(), cairo_show_text() and cairo_set_scaled_font())
• and - finally - the self-provided string-to-glyphs converter using Cairo's cairo_scaled_font_text_to_glyphs()

The sample application is provided with full source code. The basic steps to use the self-provided string-to-glyphs converter looks like this:

// Load and remember the font. Cairo uses the last loaded font for 'toy' text API.
context.SelectFontFace ("Sans", FontSlant.Normal, FontWeight.Normal); // Georgia // Courier
Cairo.FontFace ffSans = context.ContextFontFace;

// Prepare the scaled font for glyph processing.
Cairo.Matrix     fm     = new Cairo.Matrix (/*font size*/ 20.0, 0.0, 0.0, /*font size*/ 20.0,
                                            /*translationX*/ 0.0, /*translationY*/ 0.0);
Cairo.Matrix     tm     = new Cairo.Matrix (1.0, 0.0, 0.0, 1.0, 0.0, 0.0);
FontOptions      fo     = new FontOptions();
Cairo.ScaledFont sfSans = new ScaledFont (ffSans, fm, tm, fo);
fo.Dispose ();

// Draw some text with the 'toy' text API.
Cairo.CairoWrapper.SetScaledFont (context, sfSans);
Cairo.CairoWrapper.MoveTo (context, 15, 270);
Cairo.CairoWrapper.ShowText (context, "'Sans' writing some glyphs converted with");
// Convert string into glyphs.
Cairo.Glyph[] glyphs;
Cairo.CairoWrapper.ScaledFontTextToGlyphs (sfSans,
    "the self-provided converter: μ-∑-√-‡-€-™", 15, 295, out glyphs);
// Draw the text from converted glyphs.
Cairo.CairoWrapper.ShowGlyphs (context, glyphs);

// Clean up.
sfSans.Dispose();
ffSans.Dispose();

The next step could be to move the additional Cairo wrapper functionality to Cairo method extensions.

To play with the executables, start either /bin/Debug/32/XrwCairo.exe on 32 bit systems or /bin/Debug/64/XrwCairo.exe on 64 bit systems.

To load the project, use either the XrwCairo32.sln on 32 bit systems or the XrwCairo64.sln on 64 bit systems.

Main Findings

Standard Encoding to UTF-8 Encoding Convertion

The C# sources for Cairo contained in the gtk-sharp package already include a private method TerminateUtf8() that does the job reasonably good - only one thing is annoying:

The UTF-8 encoding of a character can consume 1 up to 3 byte. The TerminateUtf8() implementation returns always a byte array, long enough to store the worst case (all characters require the maximum of bytes to be converted). Unused bytes are set to 0.

This is where the self-provided TerminateUtf8() comes into play.

// Tested: OK
/// <summary>Convert a standard string to a byte array, that ends with '\0'.</summary>
/// <param name="s">The standard string to convert.<see cref="System.String"/></param>
/// <param name="clean">Determine whether to clean tailing unused bytes.
/// <see cref="System.Boolean"/></param>
/// <returns>The guaranteed terminated UTF-8 byte array.<see cref="System.Byte[]"/></returns>
private static byte[] TerminateUtf8 (string s, bool clean)
{
    // Compute the byte count including the trailing \0.
    int       byteCount = System.Text.Encoding.UTF8.GetMaxByteCount(s.Length + 1);
    byte[]    bytes     = new byte[byteCount];
    
    // Compute the UTF-8 bytes.
    System.Text.Encoding.UTF8.GetBytes(s, 0, s.Length, bytes, 0);
           
    if (!clean)
        return bytes;
    
    // Count tailing unused bytes.
    int realLength = byteCount;
    for (int countByte = byteCount - 1; countByte >= 0 && bytes[countByte] == 0; countByte--)
        realLength--;
    
    // Clean tailing unused bytes.
    byte[]    result = new byte[realLength + 1];
    if (realLength > 0)
        Array.Copy (bytes, result, realLength);
    result[realLength] = 0;
    
    // Done.
    return result;
}

Managed Glyph to Unmanaged Memory Convertion (And Return)

The C# sources for Cairo contained in the gtk-sharp package also include an internal method FromGlyphToUnManagedMemory() that is needed to provide cairo_show_glyphs() with glyphs. Unfortunately, this implementation depends in the Context class implementation and can't be called separately. Here is my reengineered method:

// Copyright: Please see "Cairo.cs"!
// Tested: OK
/// <summary>Convert a glyph array to its equivalent unmanaged memory representation.</summary>
/// <param name="glyphs">The array of glyphs to convert.<see cref="Glyph[]"/></param>
/// <returns>The unmanaged memory representation of a glyph array.<see cref="IntPtr"/></returns>
internal static IntPtr FromGlyphToUnManagedMemory(Glyph [] glyphs)
{
    IntPtr    dest       = IntPtr.Zero;
    int       ptrSize    = Marshal.SizeOf (typeof (IntPtr));

    if (ptrSize != 4)
    {
        int native_glyph_size = Marshal.SizeOf (typeof (Glyph));
        dest = Marshal.AllocHGlobal (native_glyph_size * glyphs.Length);
        long pos = dest.ToInt64();
        
        foreach (Glyph g in glyphs)
        {
            Marshal.StructureToPtr (g, (IntPtr)pos, false);
            pos += native_glyph_size;
        }
    }
    else
    {
        int native_glyph_size = Marshal.SizeOf (typeof (NativeGlyph_4byte_longs));
        dest = Marshal.AllocHGlobal (native_glyph_size * glyphs.Length);
        long pos = dest.ToInt64();
        
        foreach (Glyph g in glyphs)
        {
            NativeGlyph_4byte_longs n = new NativeGlyph_4byte_longs (g);

            Marshal.StructureToPtr (n, (IntPtr)pos, false);
            pos += native_glyph_size;
        }
    }
    
    return dest;
}

Based on this, I also implemented the opposite direction FromUnManagedMemoryToGlyph(), that is required for ScaledFontTextToToGlyph().

// Tested: OK
/// <summary>Convert an unmanaged memory representation of glyphs to an array of glyphs.
/// </summary>
/// <param name="ptr">The unmanaged memory representation of glyphs to convert.
/// <see cref="IntPtr"/></param>
/// <param name="length">The number of glyphs to convert.<see cref="System.Int32"/></param>
/// <returns>The converted glyph array.<see cref="Glyph[]"/></returns>
internal static Glyph[] FromUnManagedMemoryToGlyph (IntPtr ptr, int length)
{
    Glyph[] glyphs = new Glyph[Math.Max (0, length)];
    
    if (length <= 0)
        return glyphs;
    
    int ptrSize    = Marshal.SizeOf (typeof (IntPtr));

    if (ptrSize != 4)
    {
        int native_glyph_size = Marshal.SizeOf (typeof (Glyph));
        long pos = ptr.ToInt64();
        
        for (int glyphCount = 0; glyphCount < length; glyphCount++)
        {
            glyphs[glyphCount] = (Glyph) Marshal.PtrToStructure ((IntPtr)pos, typeof(Glyph));
            pos += native_glyph_size;
        }
    }
    else
    {
        int native_glyph_size = Marshal.SizeOf (typeof (NativeGlyph_4byte_longs));
        long pos = ptr.ToInt64();
        
        NativeGlyph_4byte_longs buffer;
        
        for (int glyphCount = 0; glyphCount < length; glyphCount++)
        {
            buffer = (NativeGlyph_4byte_longs) Marshal.PtrToStructure ((IntPtr)pos,
                     typeof(NativeGlyph_4byte_longs));
            glyphs[glyphCount] = new Glyph (buffer.index, buffer.x, buffer.y);
            pos += native_glyph_size;
        }
    }
    
    return glyphs;
}

Both implementations work for 32 bit and 64 bit environments and distinguish the environment using the size of a void* pointer: Marshal.SizeOf (typeof (IntPtr)).

UTF-8 Text to Glyph Convertion

The self-provided wrapper around cairo_scaled_font_text_to_glyphs() looks like this:

// Tested: OK
// This method has been the final target to enable glyph drawing via Context.ShowGlyphs.
/// <summary>Convert an UTF-8 text to glyphs, using the indicated scaled font.</summary>
/// <param name="scaledFont">The scaled font, required to convert UTF-8 text to glyphs.
/// <see cref="ScaledFont"/></param>
/// <param name="utf8text">The UTF-8 text to convert.
/// <see cref="System.String"/></param>
/// <param name="startX">The X start position of the first glyph.
/// <see cref="System.Double"/></param>
/// <param name="startY">The Y start position of the first glyph.
/// <see cref="System.Double"/></param>
/// <param name="glyphs">The glyph array as result of the convertion.
/// <see cref="Glyph[]"/></param>
/// <returns>The Cairo status (success or error) of the convertion.
/// <see cref="Status"/></returns>
public static Status ScaledFontTextToGlyphs(ScaledFont scaledFont, string utf8text,
                                            double startX, double startY, out Glyph[] glyphs)
{
    byte[] terminatedUtf8 = TerminateUtf8(utf8text, true);
    IntPtr arrGlyph;
    int    numGlyph;
    
    Status status =
    NativeMethodsEx.cairo_scaled_font_text_to_glyphs (scaledFont.Handle, startX, startY,
                                                      terminatedUtf8,    
                                                      terminatedUtf8.Length - 1,
                                                      ref arrGlyph,      out numGlyph,
                                                      IntPtr.Zero,       
                                                      IntPtr.Zero, IntPtr.Zero);
    if (status != Status.Success)
    {
        glyphs = new Glyph[0];
        return status;
    }

    if (arrGlyph != IntPtr.Zero && numGlyph > 0)
        glyphs = FromUnManagedMemoryToGlyph (arrGlyph, numGlyph);
    else
        glyphs = new Glyph[0];
    
    //if (textExtends != null)
    //    NativeMethods.cairo_scaled_font_glyph_extents (scaledFont.Handle,
    //                                                   arrGlyph, numGlyph, out textExtends);
    
    NativeMethodsEx.cairo_glyph_free (arrGlyph);
    return status;
}

Where cairo_scaled_font_text_to_glyphs() and the other native Cairo methods are defined as:

internal static class NativeMethodsEx
{
    const string cairo = "libcairo-2.dll";
    
    // Tested: OK
    // If clusters are required.
    [DllImport (cairo, CallingConvention=CallingConvention.Cdecl)]
    internal static extern Status
    cairo_scaled_font_text_to_glyphs (IntPtr scaled_font,  double x, double y,
                                      byte[] utf8,         int utf8_len,
                                      ref IntPtr glyphs,   out int num_glyphs,
                                      ref IntPtr clusters, out int num_clusters,
                                      ref IntPtr cluster_flags);

    // Tested: OK
    // If clusters are NOT required.
    [DllImport (cairo, CallingConvention=CallingConvention.Cdecl)]
    internal static extern Status
    cairo_scaled_font_text_to_glyphs (IntPtr scaled_font,  double x, double y,
                                      byte[] utf8,         int utf8_len,
                                      ref IntPtr glyphs,   out int num_glyphs,
                                      IntPtr clusters,     IntPtr  num_clusters,
                                      IntPtr cluster_flags);

    // Tested: OK
    [DllImport (cairo, CallingConvention=CallingConvention.Cdecl)]
    internal static extern void
    cairo_show_text_glyphs (IntPtr scaled_font, byte[] utf8, int utf8_len,
                            IntPtr glyphs, int num_glyphs,
                            ref IntPtr clusters, ref int num_clusters,
                            ref /*ClusterFlags*/ IntPtr cluster_flags);
    
    // Tested: OK
    [DllImport (cairo, CallingConvention=CallingConvention.Cdecl)]
    internal static extern void cairo_glyph_free (IntPtr glyphs);
    
    // Tested: OK
    [DllImport (cairo, CallingConvention=CallingConvention.Cdecl)]
    internal static extern void cairo_text_cluster_free (IntPtr glyphs);
}

Points of Interest

The Cairo.CairoWrapper class contains all structures and helper methods necessary in addition to the Mono.Cairo package, to convert strings into glyph arrays, measure the glyphs (to realize auto line breaking) and draw them.

History

• 14^th July, 2014: Initial version
• 29^th July, 2014: Some orthography errors and minor bugs