Subpixel rendered font bitmaps

LCDs on these small systems are general pretty low resolution and would greatly benefit from subpixel font rendering. The normal way to do that is to triple the horizontal resolution of the font bit map. That works out to 3, 6, 9 or so bit per pixel. Freetype can generate these horizontally expanded bitmaps. I suspect the online bitmap font generation tool is freetype based?

For example the letter ‘l’
RGB RGB - two pixels on display
001 110 - single bit
013 310 - two bit
027 720 - three bit

These horizontally expanded bitmaps compress down a lot. But the decompression and compositing code needs to be aware of the triple horizontal expansion when mapping to 16b or 32b final pixel colors.

This does not have to consume a lot of ram. It may work out that a 6b per pixel font looks better than 8b antialiasing. Note that you can even subpixel render the character width and kerning if you wish.

I noticed this issue when running the demo that prints the blue button with Hello World. The parallel 'l’s have color fringing and turned bluish. I believe subpixel rendering can turn them white.


Is it the same thing as Windows ClearType?

Can you send a screenshot?

This is a basic part of how Windows Cleartype works, the same effect is used in X/Wayland via the Freetype engine. There are more advanced aspects to Cleartype but those take up too much RAM. You need to take a photo to be able to see the rendering issues. That is a portrait LCD, I used a microscope to check.

Note the blue effect on some of the vertical lines, but not all. Also note the striping effect on the W. All of that can be fixed with subpixel rendering. Down at the bottom of the Wikipedia article they show a photo of the letter ‘W’. LittleVGL is rendering like the middle row, it can be improved to render like the bottom row.

The font converter is just updated to produce sharper letters but still you won’t see two " fully covering rectangles" for 'l' letters.
(The new fonts will be included in the next version of LittelvGL. Until that you can test it in the dev branch of lv_font_conv)

I see two problems with subpixel rendering:

  • Relies on pixel order of the screen
  • Relies on the orientation of the display

However, it can be easy to handle in LittlevGL. E.g. the font could have bpp = 8 (in RGB332 format) and then the bit’s values can be translated into real colors.

As I mentioned the font converter was just reworked.
@puzrin What do you think about it?

The bitmaps don’t represent colors, they are still monochrome but expanded 3X in the horizontal direction. The colors don’t come into play until you are compositing onto the background. Since these bitmaps are monochrome they can be shifted on a subpixel basis, that is how you can do subpixel kerning. You are going to pack the three monochrome pixels into the RGB332 format, that works.

The font bitmaps have to be generated to correspond to the pixel layout of the screen – RGB, BGR, pentile, vertical or horizontal stripe. All of this is supported in the Freetype engine.

Note that you should always be using your LCD with the pixel stripe running horizontally. That is a hardware difference between a portrait and landscape format LCD. If you use them in the wrong orientation they are never going to look good.

Don’t worry about the Cleartype patents, this is not part of the patented process and the last Cleartype patent expires Oct 10 anyway - 16 days from now.

I see. I was really misinterpreting it. Converting font as 3 times wider is really required for subpixel kerning.

Let’s see what @puzrin thinks about it (the author of the font converter).

In the LittleVGL context the bitmaps do not need to know about RGB, BGR, etc. It is the compositing code that needs to know that. The compositing code also needs to know about subpixel kerning.

On the desktop this is different since Freetype is doing the kerning and then making the RGB32 pixels.

It is fairly trivial to generate these font bitmaps (just change a parameter when you ask freetype to generate the bitmaps, then you need a new packing format). The effort needs to go into the compositing code, it is going to do compositing pixel by pixel. That is much more memory efficient than expanding into RGB buffers and doing raster ops. You don’t have hardware raster ops so that method is going to be slower and use a lot of memory.

The sub pixel kerning affects “text/letter width” related functionalities. Now fractional width is not supported. Letter width + Kerning is rounded to the nearest integer. That’d be one task to solve.

Regarding the rendering: Let’s assume each pixel has bpp = 8 and the display has RGB888 format. Then the task is to get the next 3 pixels and assigned to the R G B channels. Of course, handle if the letter starts at a fractional pixel (in this case get on 1 or 2 pixel from the font). Is that correct?

It seems more difficult to support fractional width because the whole font engine is built to work with integer width.
On the other hand, this tricky letter drawing is a local update which is easier to do.

I still see advantage in supporting subpixel rendering only (without subpixel kerning). What do you think?

Say a display has these pixel formats

And the monochrome bitmap tells you to light up the third sub-pixel. The compositor needs to know the LCD pixel format so that it knows which color channel is hooked to the third sub-pixel.

Sub-pixel rendering without sub-pixel kerning is a fine place to start. You can always add sub-pixel kerning later.

BGR panels are just RGB panels mounted upside down.

When working on sub-pixel rendering code, you will find a cheap USB microscope very useful. The microscope will let you see the individual sub-pixels and make errors in your code obvious.

Here is the easy way to explain why sub-pixel rendering works. LCD pixel format - RGB RGB RGB RGB. So if you turn on RGB you get white.

Now make the observation that it simply requires three consecutive sub-pixels to make white. It does not have to only be the RGB pixels. If you span two pixels and turn on GBR or BRG you also get white.

Now you have a means of shifting that white dot left/right by one-third of a pixel. Doing that has a large impact on how clearly fonts render.

What works for white similarly works for any color.

I know about subpixel rendering, but available info describes only general princilple, without details. As far as i understand:

  • That will require 3x more memory to store fonts (because we will get 3x more horizontal resolution).
    • Probably, flash size increase can be partially dimmed by compression, but decompression buffer will be 3x more.
  • There should be halos at least from the left and from the right of text. I don’t understand how to compensate that.
  • This is much more sensitive to right gamma correction.

So, technically, it should not be difficult modify conventor & spec to support 3x horizontal resolution bitmaps, but i don’t understand how to process those in details.

May be someone will be more lucky with understanding FT docs and links to outer sites:

I see that on my PC too; at least on Windows. I don’t know if it’s the monitor or the OS. How severe is the halo you are talking about?

How large is the decompression buffer right now? Or, could we make this an optionally supported feature (to reduce flash size on platforms which don’t need it)?

I don’t know. I only explained, that “subpixel rendering” sounds easy only until you don’t start to dig details :).

Now it’s allocated to match largest glyph of all registered fonts. Size is ~H*W*bpp.

The freetype subpixel support is probably further down the pipeline than you want. The output from FT_CONFIG_OPTION_SUBPIXEL_RENDERING has already been built for RGBA32 buffers.

Consider earlier in the pipeline and just generate the 3x expanded monochrome bitmaps. Then do the conversion to RGB16 or RGB24 in LittleVGL. You can add this filtering at that point if you want. You don’t have to implement all of the ClearType effects to get most of the benefit.

This filtering is the subject of those patents that expire in a few days.

I’m responsible for convertor only, and not C programmer at all :). As i said, it should be not too difficult to create 3x wide alpha bitmaps, and that should not break font spec. But i have no idea about implementation cost in LVGL. I guess, when someone will be able to implement that and finish job, everybody will migrate to HiDPI LCD-s, and issue become not critical :).

There are degrees of font perfection. Cleartype is trying for absolute perfection. One way to start would be to generate the 3X wider monochrome fonts and then hack some on the LittleVGL pixel generation code to see how far you can get. If initial hacking is successful then work on optimizations.

Alternatively you could store the output from FT_CONFIG_OPTION_SUBPIXEL_RENDERING in the font bitmaps, that makes the fonts much larger but they are also far easier to use.

I think, lvgl needs new hero :). I see something suspicious in situation, when i should spend my time to improve foreign libs instead of doing my own hobby projects. Currently i did lv_i18n & lv_font_conv, because previous approaches were not acceptable for me. Situation with subpixel rendering is not nice, but acceptable.

From my side, i did everything possible to make font specs & convertor code easy to extend and maintain. If anyone wish to experiment - everything is available on github.

Just thought experiment about composing the pixels (let’s use percentage instead of 0…255 range):

                R   G   B      R   G   B          
BG COLOR        100 0   0      100 0   0
FONT COLOR      0   100 0      0   100 0
FONT DATA       20  40  60     80  100 100
RESULT          80  60  0      20  100 0

Can you confirm it?

If it should work in this way than I can add the rendering part to LittlevGL.
Of course, it should be optional to render the font in 3X wider or normally.

Can you send a Pull Request to lv_font_conv?
I can imagine a --render-subpixel flag to enable this feature.

That is the first step of how it works. The more advanced stuff is adding those filters.