Loading font from filesystem consumes too much RAM

General discussion
,
1

Hey guys, I tried to load a 16MB font from filesystem, and found that LVGL read all the glyphs and bitmaps into the RAM (LVGL 8.2).
In my view, this feature supposes to be designed as “dynamically reading from the file”. If we need to put the font in an external filesystem, that usually means the font is too large to be fitted into the ROM. And generally, the RAM is even smaller than the ROM.
Since Chinese IME has been integrated into LVGL, Chinese font is also necessary. I notice that the Chinese IME can input only a very very small portion of Chinese characters, mainly due to the problem in loading large font.
Looking forward to hearing about any progress on this issue~

2

Hi,

I think the descriptors of the glyphs (box size, position, kerning, etc) should be loaded to RAM to get a reasonable performance. However the bitmaps could be really really loaded on demand.

Although it seems similar but in the practice it’s quite different from the current font loader. Probably it can be extended in some form like this:

  1. Add an option to not load the bitmaps
  2. Create a special font->get_glyph_bitmap which handles only getting the bitmaps on demand from a given bin font file.

What do you think?

It’s only for the built in SimSun font. However you can add a custom dictionary to it…

3

Thanks for your response~

  1. The basic Chinese subset GB2312 contains about 8000 characters, the descriptors will consume 8000x16byte=128KB RAM, which is still kinda large but more acceptable. If the descriptors only record the start position of glyphs, the RAM will be 8000x4byte=32KB.
  2. I have tried on my STM32H7@280MHz, it takes about 6 seconds to load the descriptors from SD card. I think it would be better if descriptors are directly saved as binary and loaded by memcpy.

I think SCATTER LOADING is a good idea and is rather easy to implement, something like this:

  1. Convert the font to both *.c and *.bin.
  2. Use an option like “FONT_LOAD_BITMAP” to control whether to mask the “const uint8_t glyph_bitmap[]” in *.c file. If FONT_LOAD_BITMAP==True, that will be the same as current version, otherwise the bitmap will be empty.
  3. Use a function like “set_font_bitmap_src” to set bitmap path in file system, and get the bitmaps on demand.

In this case, the font will consume 160K ROM, and the startup speed is fast, which I believe is more preferable.

4

The current CJK subset was created by me, but as a non CJK speaker probably it’s not the best. So any suggestions about official or handmade subsets would be highly appreciated.

Could you make an experiment by manually crafting a font? Just cut the bitmap, create a binary file from it and set a new get_bitmap callback on the font, similar to the original, just read a file.

If it work well in the practice too, I’m ok with supporting something like this officially.

5

This blog post has a list of the 3500 (2500+1000) most common Chinese characters and some background info:

6

Thanks, it’s great!

Do you think it’s worth to have a combined CJK built in font or does it make more sense to have separate Chinese, Japanese and Korean fonts?

7

Combined font is probably the future, but not currently widely used in real applications.

Usually we choose only one subset, and these are the most commonly used ones:
GB2312: China mainland, 7000 characters
Big5: Taiwan/Hongkong, 13000 characters
SHIFT-JIS: Japan, 7000 characters

8

I’m ok with using separate fonts in v9. Is there widely accepted free fonts that you can recommend?

9

The Google Noto fonts also have a CJK branch available under SIL Open Font License v1.1. It’s the same license that applies to Montserrat.

10

Sounds great, thanks!