How to use BIDI(LV_USE_BIDI(used for arabic)), any demo examples?

Anda · December 4, 2019, 1:11pm

Description

I want to realize the display of the Arabic font library,
I learned that it can be achieved by BIDI method,
how to test this program?

What MCU/Processor/Board and compiler are you using?

BLE MCU(ARM)

What do you want to achieve?

I want to achieve Arabic font display!

What have you tried so far?

I now use the font generator to generate the Arabic font,
and then the display is not good,
this is not consistent with the actual text content

Code to reproduce

Add a code snippet which can run in the simulator. It should contain only the relevant code that compiles without errors when separated from your main code base.

The code block(s) should be formatted like:

lv_obj_t *arabic_label = lv_label_create(lv_disp_get_scr_act(NULL), NULL);

static lv_style_t text_style;
lv_style_copy(&text_style, lv_obj_get_style(arabic_label));
text_style.text.font = &lv_font_arabic;
text_style.text.color = LV_COLOR_MAGENTA;

lv_label_set_style(arabic_label, LV_LABEL_STYLE_MAIN, &text_style);

char *arabic_text = "مرحبا ، يانغ تشاو ، أنا معجب بك كثيرا ، هيا";
lv_label_set_text(arabic_label, arabic_text);

lv_obj_align(arabic_label, NULL, LV_ALIGN_CENTER, 0, 0);

Screenshot and/or video

If possible, add screenshots and/or videos about the current state.
text: مرحبا ، يانغ تشاو ، أنا معجب بك كثيرا ، هيا

This picture is expected

This image is the effect of the run

@kisvegabor
@amirgon

Looking forward to your reply!
Thanks in advance!

Best Regards.

embeddedt · December 4, 2019, 3:20pm

Mentions only work when there aren’t spaces between the @ and the username.

@kisvegabor
@amirgon

Anda · December 4, 2019, 3:30pm

Thank you very much for your suggestion, I have modified it now.

Best Regards.

embeddedt · December 4, 2019, 4:13pm

You do not need to link to user’s profiles. Just type @ followed by their name. (i.e. @embeddedt)

Anda · December 4, 2019, 4:21pm

Thanks. I get it.

amirgon · December 4, 2019, 8:27pm

Hi @Anda!
I’m glad to see that you are interested in the Bidi feature!
Currently this feature implements Unicode specs for displaying LTR and RTL (also mixed), and orders the characters correctly.
To enable it, you need to define LV_USE_BIDI to 1 on lv_conf.h.

However, the support for Arabic script is not complete, and there are several missing features:

Ligatures
Diacritics
Context-sensitive text shaping (each character can have multiple shapes)
Add Arabic script unicode ranges to lv_bidi_letter_is_rtl (that one is very easy)
Others?

Without these features you may not be able to display Arabic correctly. You can read more about this on https://github.com/littlevgl/lvgl/issues/899

Are you interested in helping implementing these missing features for LittlevGL?

Current work on Bidi involved Hebrew script support, which is much simpler in this aspect and doesn’t require the additional features mentioned above.
If you would like, I can provide you an example of Hebrew Bidi demo.
btw, who is Yang Chao?

Anda · December 5, 2019, 2:36am

Thank you very much for your reply, I now know that the Arabic characters are special, and I also understand the rules of Arabic display(i.e. Ligatures, Diacritics, Text Shaping)

If you would like, I can provide you an example of Hebrew Bidi demo.
Yes, I think this can help me to better achieve Arabic display

btw, who is Yang Chao?
Hahaha, I didn’t seem to mention she. This looks like Rocket Girls 101 – Yang ChaoYue.

Best Regards.

Anda · December 5, 2019, 2:57am

Regarding the Arabic display rule method, I provide the following Arabic match table for reference

// 0621 to 064a deformation code, exceed this part are their own codeword
static const u16 Arabic_PositionTable[][4]= {
// first, last, middle, alone
{ 0xfe80, 0xfe80, 0xfe80, 0xfe80 }, // 0x621
{ 0xfe82, 0xfe81, 0xfe82, 0xfe81 },
{ 0xfe84, 0xfe83, 0xfe84, 0xfe83 },
{ 0xfe86, 0xfe85, 0xfe86, 0xfe85 },
{ 0xfe88, 0xfe87, 0xfe88, 0xfe87 },
{ 0xfe8a, 0xfe8b, 0xfe8c, 0xfe89 },
{ 0xfe8e, 0xfe8d, 0xfe8e, 0xfe8d },
{ 0xfe90, 0xfe91, 0xfe92, 0xfe8f }, // 0x628
{ 0xfe94, 0xfe93, 0xfe93, 0xfe93 },
{ 0xfe96, 0xfe97, 0xfe98, 0xfe95 }, // 0x62A
{ 0xfe9a, 0xfe9b, 0xfe9c, 0xfe99 },
{ 0xfe9e, 0xfe9f, 0xfea0, 0xfe9d },
{ 0xfea2, 0xfea3, 0xfea4, 0xfea1 },
{ 0xfea6, 0xfea7, 0xfea8, 0xfea5 },
{ 0xfeaa, 0xfea9, 0xfeaa, 0xfea9 },
{ 0xfeac, 0xfeab, 0xfeac, 0xfeab }, // 0x630
{ 0xfeae, 0xfead, 0xfeae, 0xfead },
{ 0xfeb0, 0xfeaf, 0xfeb0, 0xfeaf },
{ 0xfeb2, 0xfeb3, 0xfeb4, 0xfeb1 },
{ 0xfeb6, 0xfeb7, 0xfeb8, 0xfeb5 },
{ 0xfeba, 0xfebb, 0xfebc, 0xfeb9 },
{ 0xfebe, 0xfebf, 0xfec0, 0xfebd },
{ 0xfec2, 0xfec3, 0xfec4, 0xfec1 },
{ 0xfec6, 0xfec7, 0xfec8, 0xfec5 }, // 0x638
{ 0xfeca, 0xfecb, 0xfecc, 0xfec9 },
{ 0xfece, 0xfecf, 0xfed0, 0xfecd }, //0x63A
{ 0x063b, 0x063b, 0x063b, 0x063b },
{ 0x063c, 0x063c, 0x063c, 0x063c },
{ 0x063d, 0x063d, 0x063d, 0x063d },
{ 0x063e, 0x063e, 0x063e, 0x063e },
{ 0x063f, 0x063f, 0x063f, 0x063f },
{ 0x0640, 0x0640, 0x0640, 0x0640 }, // 0x640
{ 0xfed2, 0xfed3, 0xfed4, 0xfed1 },
{ 0xfed6, 0xfed7, 0xfed8, 0xfed5 },
{ 0xfeda, 0xfedb, 0xfedc, 0xfed9 },
{ 0xfede, 0xfedf, 0xfee0, 0xfedd },
{ 0xfee2, 0xfee3, 0xfee4, 0xfee1 },
{ 0xfee6, 0xfee7, 0xfee8, 0xfee5 },
{ 0xfeea, 0xfeeb, 0xfeec, 0xfee9 },
{ 0xfeee, 0xfeed, 0xfeee, 0xfeed }, // 0x648
{ 0xfef0, 0xfeef, 0xfef0, 0xfeef },
{0xfef2, 0xfef3, 0xfef4, 0xfef1 }, // 0x64A
};

// Character set one, used to read the previous connection
static const u16 Arabic_CharSet1[23] = {
0x62c, 0x62d, 0x62e, 0x647, 0x639, 0x63a, 0x641, 0x642,
0x62b, 0x635, 0x636, 0x637, 0x643, 0x645, 0x646, 0x62a,
0x644, 0x628, 0x64a, 0x633, 0x634, 0x638, 0x626
};

// Character set two, used to determine the next continuation
static const u16 Arabic_CharSet2[35] = {
0x62c, 0x62d, 0x62e, 0x647, 0x639, 0x63a, 0x641, 0x642,
0x62b, 0x635, 0x636, 0x637, 0x643, 0x645, 0x646, 0x62a,
0x644, 0x628, 0x64a, 0x633, 0x634, 0x638, 0x626,
0x627, 0x623, 0x625, 0x622, 0x62f, 0x630, 0x631, 0x632,
0x648, 0x624, 0x629, 0x649
};

// Special codeword for continuous writing
static const u16 Arabic_CharSpecs[][2] = {
{ 0xFEF5, 0xFEF6 },
{ 0xFEF7, 0xFEF8 },
{ 0xFEF9, 0xFEFA },
{ 0xFEFB, 0xFEFC },
};

Best Regards.

Anda · December 5, 2019, 9:52am

Hi, amirgon, Can you provide a Hebrew BIDI example?

@amirgon

Best Regards.

amirgon · December 5, 2019, 8:04pm

Could you explain these tables you provided and how they are used?
Where were they taken from? Could you provide some references?

Yes.
I’m planning to write a blog post that explains how to use lvgl Bidi with Hebrew script, and provide a complete example.
If, however, you are in a hurry, I can explain the basics and send you the Hebrew font I’m using.

Anda · December 6, 2019, 3:11am

This is my collection from online resources, I can provide some references.

Good, I also want to refer to the Hebrew example.

Best Regards.

Anda · December 6, 2019, 3:36am

Now I provide a simple demo example about Arabic

bool Arabic_GetLigatrueFirstState(u16 now)
{

for(int index = 0; index < ARRAY_COUNT(Arabic_CharSet1); index ++){
  if( now == Arabic_CharSet1[index] ) return true;
}
return false;

}

bool Arabic_GetLigatureLastState(u16 now)
{

for(int index = 0; index < ARRAY_COUNT(Arabic_CharSet2); index ++){
   if( now == Arabic_CharSet2[index] ) return true;
}
return false;

}

bool Arabic_GetLigatureSpecCharState(u16 now, u16 next)
{

if( now != 0x0644 ) return false;

if( next == 0x0622 || next == 0x0623 || next == 0x0625 || next == 0x0627 ) return true;

return false;

}

u16 Arabic_GetChar(u16 prev, u16 now, u16 next)
{

if( now <= 0x0621 || now >= 0x064A ) return now;

u32 offset = now - 0x0621;

bool first_flag = Arabic_GetLigatrueFirstState(prev);
bool last_flag = Arabic_GetLigatureLastState(next);

u8 state = (first_flag << 0) | (last_flag << 1);

u16 code;

switch( state ){
case 0:
{
	now = Arabic_PositionTable[offset][3];
}
break;
case 1:
{
	now = Arabic_PositionTable[offset][0];
}
break;
case 2:
{
    now = Arabic_PositionTable[offset][1];
}
break;
case 3:
{
	now = Arabic_PositionTable[offset][2];
}
break;
default:
{
	now = Arabic_PositionTable[offset][3];
}
break;
}

return now;

}

u16 Arabic_GetLigatureSpecChar(u16 prev, u16 next)
{

u16 code  = 0;
u8  state = Arabic_GetLigatrueFirstState(prev);

switch( next ){
case 0x0622:
{
	code = Arabic_CharSpecs[0][state];
}
break;
case 0x0623:
{
	code = Arabic_CharSpecs[1][state];
}
break;
case 0x0625:
{
	code = Arabic_CharSpecs[2][state];
}
break;
case 0x0627:
{
	code = Arabic_CharSpecs[3][state];
}
break;
}

return code;

}

void Arabic_Process(u16 *in, u32 in_count, u16 *out, u32 *out_count)
{

if( in_count == 0 ){
	*out_count = 0;
	return;
}
if( in_count == 1 ){
	*out = *in;
	*out_count = 1;
	return;
}
u32 index = 0;
u32 count = 0;
while( index < in_count ){
	if( index == 0 ){
		*out = Arabic_GetChar(0, *in, *(in + 1));
		out ++;
	    in ++;
		index ++;
		count ++;
	}else if( index == (in_count - 1) ){
		*out = Arabic_GetChar(*(in - 1), *in, 0);
		index ++;
		count ++;
	}else{
		if( Arabic_GetLigatureSpecCharState(*in, *(in + 1)) ){
			*out = Arabic_GetLigatureSpecChar(*(in - 1), *(in + 1));
			out ++;
		    in += 2;
			index += 2;
		}else{
			*out = Arabic_GetChar(*(in - 1), *in, *(in + 1));
			out ++;
			in ++;
			index ++;
		}
		count ++;
	}
}

*out_count = count;

}

void Arabic_Test(void)
{
// origin text: “مرحبا”;

u16 arabic_text_unicodes[] = {
    0x645,
    0x631,
    0x62D,
    0x628,
    0x627,
};
u16 arabic_text_count = ARRAY_COUNT(arabic_text_unicodes);
u16 arabic_result_unicodes[32];
u16 arabic_result_count = 0;
Arabic_Process(arabic_text_unicodes, arabic_text_count, arabic_result_unicodes, &arabic_result_count);
// reverse(arabic RTL)
for(int index = arabic_result_count - 1; index >= 0; index --){
    u16 letter       = arabic_result_unicodes[index];
    u16 letter_next  = (index == 0) ? 0 : arabic_result_unicodes[index - 1];
    u16 letter_width = lv_font_get_glyph_width(font, letter, letter_next);
    lv_draw_letter(&point, mask, font, letter, color, opa);
    if( letter_width ){
         point.x += letter_width + style->text.letter_space;
    }
}

}

amirgon · December 6, 2019, 10:06am

Thank you for your code examples!

Did you consider sending a Push Request to lvgl (dev 7.0 branch)? It would be helpful and make it much easier for us to review and test your code. This forum is not a good media for sharing code.
I suggest adding the Arabic script code to lv_bidi.c and lv_bidi.h guarded by #if LV_ARABIC or something similar that can be set on lv_conf.h.

Anda · December 6, 2019, 11:21am

Hmm, I ’m not very familiar with Push Request. I have tested that the Arabic text can be displayed normally. It is just a simple test. I will improve this later and look forward to your support.

I can send this code file if you need.

By the way, you can also send me your Hebrew example for reference.

Best Regards.

kisvegabor · December 6, 2019, 11:28am

@Anda
I wrote a short summary about Pull requests. I think you will find easy to send one if you follow the guide.

Anda · December 6, 2019, 11:30am

Okay, I will learn about this first.

amirgon · December 6, 2019, 3:48pm

@Anda - One more important thing, regarding licensing.

As you may have noticed, lvgl is licensed as MIT.
As long as you contribute your own code, a code you have written yourself and didn’t copy from anywhere else - there is no problem. Such code would automatically be licensed as MIT as well.

However, if you copied the code from somewhere (you mentioned “online resources”), we must make sure it’s allowed to include that code in lvgl. For example, a piece of code license as GPL cannot be used in lvgl.

Therefore, in order to avoid legal issues, could you please let us know:

Which parts of your code did you copy from other sources? (you didn’t write yourself)
What is the license agreement of each of these sources?

Anda · December 7, 2019, 2:20am

Regarding the display and processing of Arabic rules, this is the code I rewritten, I also refer to it and have not specified it, such as from some blogs. Of course, this is just for reference, and can be used to understand the display processing of Arabic.

Best Regards.

Anda · December 7, 2019, 10:16am

This code is modified.

void Arabic_Test(void)
{
// origin text: “مرحبا”;

u16 arabic_text_unicodes[] = {
    0x645,
    0x631,
    0x62D,
    0x628,
    0x627,
};
u32 arabic_text_count = ARRAY_COUNT(arabic_text_unicodes);
u16 arabic_result_unicodes[32];
u32 arabic_result_count = 0;
Arabic_Process(arabic_text_unicodes, arabic_text_count, arabic_result_unicodes, &arabic_result_count);
// reverse(arabic RTL)
for(int index = arabic_result_count - 1; index >= 0; index --){
    u16 letter       = arabic_result_unicodes[index];
    u16 letter_next  = (index == 0) ? 0 : arabic_result_unicodes[index - 1];
    u16 letter_width = lv_font_get_glyph_width(font, letter, letter_next);
    lv_draw_letter(&point, mask, font, letter, color, opa);
    if( letter_width ){
         point.x += letter_width + style->text.letter_space;
    }
}

}