1 Preface

Let's start with a soul torture: Why study OpenGL rendering text? Isn't it more fragrant to use Android canvas?!

It depends on the application scenario. A pure UI interface does not need to use OpenGL. However, for complex ones, such as bullet screen, the effect will be much better with OpenGL.

So what's the difference between Canvas and OpenGL?
Canvas is an API for 2D graphics. If hardware acceleration is not turned on, CPU is used for drawing (the bottom layer is through skia engine and pure software). If hardware acceleration is turned on, GPU is used for drawing (the work of canvas is handed over to GPU internally through OpenGLRender for hardware drawing).

OpenGL is the API of 3D graphics, and GPU drawing is adopted by default, that is, hardware implementation.

So the difference comes out: the mobile phone model or android version may affect whether it supports hardware acceleration. Therefore, Canvas can not guarantee that it can be drawn through GPU, so it can not guarantee a stable frame rate. Usually, it can only achieve 30fps.
OpenGL, which directly uses the hardware GPU, can improve the frame rate to the same as the VSYNC of the mobile phone, such as 60fps!

In addition, OpenGL supports 3D, which can achieve more cool effects.

Therefore, the reason discussed in this paper is not fooling around, but something happens for a reason. Now from the foundation, how can we render the text.

Let's think about what we need to do to render text?

OpenGL is basically inseparable from texture, so you can actually make text into texture, and then paste it on the screen like a picture.

For example, we have an English environment, which is very simple. The characters used are ASCII codes, with only 256 characters in total.

Take the above picture as a texture, and intercept the text from the coordinates of the texture. By enabling blending to keep the background transparent, you can eventually render a string to the screen.

That's right, but there will be some problems.
(1) How to maintain the text resolution? That is, how does a picture fit to different resolution screens?
(2) How to modify the text color?
(3) What if there are more characters, such as Chinese characters?

Let's think about the ideas of each problem.
(1) How to maintain the text resolution?
Instead of the above picture representing all the words, each word has a picture. Then, according to the font size set by the code, the texture corresponding to the resolution of each text is dynamically generated. Is there any third-party library that can do it? Yes, that's the FreeType library. The following text will focus on.

(2) How to modify the text color?
Load the text texture, only extract the gray value (that is, display or not display), and the color is dynamically configured during rendering.
Code description of the clip shader directly above.

out vec4 color; 
uniform sampler2D text;
uniform vec3 textColor;
void main()
{
    vec4 sampled = vec4(1.0, 1.0, 1.0, texture(text, TexCoords).r);
    color = vec4(textColor, 1.0) * sampled;
};

As above, text is the texture of the text.
sampled extracts only one gray value of the texture. (save gray value with r)
The final effect is the transparency of the text from the tcolor shader, that is, the color of the text from the texture.

(3) What if there are more characters, such as Chinese characters?
The commonly used text texture is generated in advance and can be used directly when drawing. Rarely used text, dynamically generated as needed, and then used.

2 Introduction to font file and parsing library FreeType

2.1 font file

There are several major categories of font format types: TrueType, OpenType, WOFF and SVG. The most famous of them are the first two. Here is a brief introduction.

2.1.1 TrueType

The file suffix is ttf. Microsoft and apple jointly launched, so it is also the most commonly used font format for Windows and Mac systems.
The outline of characters (or glyphs) in TrueType fonts consists of straight lines and fragments of quadratic Be zier curves. So it has nothing to do with the size of the text. Zooming in and out can ensure the clarity of the text, sawtooth? It doesn't exist.

2.1.2 OpenType

The file suffix is otf. Jointly launched by Microsoft and Adobe. It is also an outline font, which is more powerful than TrueType. Especially in cross platform.

In terms of OpenType file structure, it is exactly an extension of TrueType format. On the basis of inheriting TrueType format, it adds support for PostScript font data (mainly used in printers). Therefore, OpenType font data can adopt both TrueType font description and PostScript font description, This is entirely up to the font manufacturer to choose. From the perspective of file structure, OpenType may not be a really new font format, but the typesetting features added by the font format have opened up a new way for users to use words from the function.

2.2 font parsing library FreeType

FreeType is an open source font parsing library, which is very general. windows, ios, android and other operating systems use this library more or less. Support various font formats, including TTF or OTF mentioned above. If you want to see the official website or download the source code, click here.

FreeType loads a font library, which is very simple.

    if (FT_Init_FreeType(&ft)) {
        LOGCATE("ERROR::FREETYPE: Could not init FreeType Library");
        return false;
    }

    // find path to fonts
    std::string font_name = ASSETS_DIR + "/fonts/Antonio-Bold.ttf";
    // load fonts as face
    FT_Face face;
    if (FT_New_Face(ft, font_name.c_str(), 0, &face)) {
        LOGCATE("ERROR::FREETYPE: Failed to load fonts");
        return false;
    }

FT_New_Face gets FT_Face. This is an important structure. Each text is loaded through this face.

After the Face is loaded, we need to define the font size, which means how big the font we want to generate from the font Face:
For example:

FT_Set_Pixel_Sizes(face, 0, 96);

The second and third parameters represent width and height. If the width value is set to 0, it means that we need to dynamically calculate the width of the font from the given height of the font face.

3 text rendering

Main work of text rendering:
(1) Use FreeType to generate bitmap of common text in advance and upload it to GPU as texture pictures (1000 + small pictures, GPU can bear it, don't be afraid).
(2) When drawing a text, check whether there is texture data. If not, do the work of (1). (text within the common text range will be encountered)
(3) Calculate the vertex coordinates of a text
(4) Start drawing the text
(5) Repeat 2 ~ 3 ~ 4 until a paragraph of text is drawn

Next, we will discuss in detail.

First, a small goal is to generate a small bitmap from the text as a texture. Of course, the width of each text is different, which is also easy to understand, such as a small dot And a letter A should not occupy the same width space.

In addition, in addition to bitmap, freetype also gives some mathematical parameters of text.
Let's find out which parameters a text contains:

Above are some mathematical parameters of the text. Among them, the horizontal line Baseline is the most important (i.e. the horizontal arrow above). When rendering each text, it should be placed based on the Baseline to look good.

Here are the details of some parameters:

attribute	Acquisition method	describe
width	face->glyph->bitmap.width	Bitmap width (pixels)
height	face->glyph->bitmap.rows	Bitmap height (pixels)
bearingX	face->glyph->bitmap_left	Horizontal distance, i.e. the horizontal position of the bit map relative to the origin (pixels)
bearingY	face->glyph->bitmap_top	Vertical distance, i.e. the vertical position of the location map relative to the reference line (pixels)
advance	face->glyph->advance.x	Horizontal reserved value, i.e. the horizontal distance from the origin to the origin of the next font (unit: 1 / 64 pixel)

These parameters will be used when calculating the vertex coordinates of text.

Well, the exciting moment is coming. Let's see how to load fonts. Here is a complete function to load fonts.

int TextSample::makeTextAsGLTexture(const wchar_t *text, int size) {
    // find path to fonts
    std::string font_name = ASSETS_DIR + "/fonts/chinese_lvshu.ttf";

    // load fonts as face
    FT_Face face;
    if (FT_New_Face(ft, font_name.c_str(), 0, &face)) {
        LOGCATE("ERROR::FREETYPE: Failed to load fonts");
        return false;
    }

    // Set size to load glyphs as
    FT_Set_Pixel_Sizes(face, 0, 96);
    FT_Select_Charmap(face, ft_encoding_unicode);

    glPixelStorei(GL_UNPACK_ALIGNMENT, 1);

    for (int i = 0; i < size; ++i) {
        //int index =  FT_Get_Char_Index(face,unicodeArr[i]);
        if (FT_Load_Glyph(face, FT_Get_Char_Index(face, text[i]), FT_LOAD_DEFAULT)) {
            LOGCATE("Failed to load Glyph");
            continue;
        }

        FT_Glyph glyph;
        FT_Get_Glyph(face->glyph, &glyph);

        //Convert the glyph to a bitmap.
        FT_Glyph_To_Bitmap(&glyph, ft_render_mode_normal, 0, 1);
        FT_BitmapGlyph bitmap_glyph = (FT_BitmapGlyph) glyph;

        //This reference will make accessing the bitmap easier
        FT_Bitmap &bitmap = bitmap_glyph->bitmap;

        // Generate texture
        GLuint texture;
        glGenTextures(1, &texture);
        glBindTexture(GL_TEXTURE_2D, texture);
        glTexImage2D(
                GL_TEXTURE_2D,
                0,
                GL_LUMINANCE,
                bitmap.width,
                bitmap.rows,
                0,
                GL_LUMINANCE,
                GL_UNSIGNED_BYTE,
                bitmap.buffer
        );

        LOGCATE("initFreeType textureId %d, text[i]=%d [w,h,buffer]=[%d, %d, %p], advance.x=%ld",
                texture, text[i], bitmap.width, bitmap.rows, bitmap.buffer,
                glyph->advance.x / MAX_SHORT_VALUE);
        // Set texture options
        glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
        glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
        glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
        glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
        // Now store character for later use
        Character character = {
                texture,
                glm::ivec2(bitmap.width, bitmap.rows),
                glm::ivec2(face->glyph->bitmap_left, face->glyph->bitmap_top),
                static_cast<GLuint>((glyph->advance.x / MAX_SHORT_VALUE) << 6)
        };
        LOGCATE("initFreeType, add to slot[%d], size (%d,%d), bearing(%d, %d)", text[i],
                bitmap.width, bitmap.rows, face->glyph->bitmap_left, face->glyph->bitmap_top);
        mCharacters.insert(std::pair<GLint, Character>(text[i], character));
    }
    glBindTexture(GL_TEXTURE_2D, 0);
    FT_Done_Face(face);
    return 0;
}

Function parameter wchar_t *text is a text array. Note that wchar is not required if the font is pure English_ t. Just char. wchar_t takes up two bytes. Chinese characters are encoded in Unicode and need two bytes.

Let's take a look at what the makeTextAsGLTexture function does

(1) FT_Load_Glyph: load a text of the text array;
(2) FT_ Get_ Glyph (face - > glyph, & glyph): extract glyph from face structure.
(3) FT_Glyph_To_Bitmap: a bitmap that generates text
(4) Glgentextures & gltextimage2d: generate textures from bitmap and upload them to GPU
(5) mCharacters.insert: store the texture id and the mathematical parameters of the text together to facilitate query and use during rendering.

OK, let's see how to call the function:

static const wchar_t CHINESE_COMMON[] = L"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghigklmnopqrstuvwxyz12367890 First of all, I don't know that when he has this, when we come, the earth is the son. You say that the year of birth is the same";

makeTextAsGLTexture(CHINESE_COMMON, sizeof(CHINESE_COMMON) / sizeof(CHINESE_COMMON[0]) - 1);

CHINESE_ The common array defines more than 1000 commonly used Chinese characters. Because of the length, I didn't post them all. You can see the article. Finally, my source code. Of course, in fact, the Chinese font library also includes English letters and numbers. After all, there are only dozens. It's easy to support, so the English letters are also added. In this way, there is no need to load both Chinese and English fonts.

makeTextAsGLTexture can be called once during initialization.

Next, give a rendering function RenderTextChinese, which will be called every time onDraw.

void TextSample::RenderTextChinese(Shader *shader, const wchar_t *text, int textLen, GLfloat x,
                                   GLfloat y, GLfloat scale,
                                   glm::vec3 color, glm::vec2 viewport) {
    // Activate the appropriate render state
    shader->setVec3("textColor", color);
    glBindVertexArray(VAO);
    checkGLError("RenderTextChinese");
    x *= viewport.x;
    y *= viewport.y;
    for (int i = 0; i < textLen; ++i) {
        Character ch;
        getCharacter(text[i], ch);
        LOGCATD("RenderTextChinese, slot[%d], textureId %d", text[i], ch.TextureID);

        GLfloat xpos = x + ch.Bearing.x * scale;
        GLfloat ypos = y - (ch.Size.y - ch.Bearing.y) * scale;

        xpos /= viewport.x;
        ypos /= viewport.y;

        GLfloat w = ch.Size.x * scale;
        GLfloat h = ch.Size.y * scale;

        w /= viewport.x;
        h /= viewport.y;

        LOGCATD("RenderTextChinese [xpos,ypos,w,h]=[%f, %f, %f, %f]", xpos, ypos, w, h);

        // VBO of current character
        GLfloat vertices[6][4] = {
                {xpos,     ypos + h, 0.0, 0.0},
                {xpos,     ypos,     0.0, 1.0},
                {xpos + w, ypos,     1.0, 1.0},

                {xpos,     ypos + h, 0.0, 0.0},
                {xpos + w, ypos,     1.0, 1.0},
                {xpos + w, ypos + h, 1.0, 0.0}
        };

        // Draw a glyph texture on the square
        glActiveTexture(GL_TEXTURE0);
        glBindTexture(GL_TEXTURE_2D, ch.TextureID);
        //glUniform1i(m_SamplerLoc, 0);
        checkGLError("RenderTextChinese 2");
        // Update VBO of current character
        glBindBuffer(GL_ARRAY_BUFFER, VBO);
        glBufferSubData(GL_ARRAY_BUFFER, 0, sizeof(vertices), vertices);
        checkGLError("RenderTextChinese 3");
        glBindBuffer(GL_ARRAY_BUFFER, 0);
        // Draw square
        glDrawArrays(GL_TRIANGLES, 0, 6);
        checkGLError("RenderTextChinese 4");
        // Note that the origin of the font is 64 / 1, and the next position of the font is updated
        x += (ch.Advance >> 6) * scale; //(2^6 = 64)
    }
    glBindVertexArray(0);
    glBindTexture(GL_TEXTURE_2D, 0);
}

The specific work of this function:
(1) getCharacter(text[i], ch); According to the Unicode value of the text, read the Character structure stored above. If it cannot be read, it indicates that it is not a common text. makeTextAsGLTexture will be called again in the getcharacter function to generate the corresponding Character of this rare word and return it.
There are few codes, which are directly stated:

void TextSample::getCharacter(const wchar_t oneText, Character &ch) {
    if (mCharacters.find(oneText) != mCharacters.end()) {
        ch = mCharacters[oneText];
    } else {
        LOGCATD("getCharacter, make a new text");
        const wchar_t temp[] = {oneText};
        makeTextAsGLTexture(temp, 1);
        ch = mCharacters[oneText];
    }
}

(2) GLfloat vertices[6][4] generates vertex coordinates corresponding to text, using normalized coordinates, i.e. [- 1.0f ~ 1.0f]. A text quadrilateral corresponds to 2 triangles, so 6 vertices are required. Because it is a 3D scene, each coordinate has four values (x, y, z, w).
(3) glBindTexture binds the texture corresponding to the text
(4) glBindBuffer bind vertices
(5) glDrawArrays draw a text
(6) The for loop traverses steps 1 to 5 and draws a text array.

Let's take a look at the function calling method:

static const wchar_t CHINESE_TEST[] = L"Love Little love HAHA";

glm::vec2 viewport(screenW, screenH);

RenderTextChinese(m_pShader, CHINESE_TEST,
                      sizeof(CHINESE_TEST) / sizeof(CHINESE_TEST[0]) - 1, -0.9f, -0.1f, 1.0f,
                      glm::vec3(0.5f, 0.8f, 0.2f), viewport);

CHINESE_TEST is an array of input text.

Why does the third parameter, sizeof (machine_test) / sizeof (machine_test [0]) - 1 subtract 1? Because there is also a newline symbol \, this does not need to be rendered.

So far, the key work has been explained and completed, but I believe there are many details you want to know. Please look at my source code directly:
OpenGLESDemo

design sketch

reference resources

Learn OpenGL Text-Rendering