Q: correct handling of event.get_param() and event.get_user_data() blobs

[h-milz]

Hi all,

in C, it is easy to access the param or user_data from the event by simply typecasting the returned (void *) to whatever (struct *) we know the data adheres to. In Python, a Blob is returned, and there seems to be no clean / official / documented way of fetching the contained data.

For example, in an lv.EVENT.INSERT event against a textarea, I try to fetch the text to be inserted using ev.get_param() in order to manipulate the string before text insertion. What I found out so far is this:

# fetch the blob, 
# dereference its address, 
# convert to bytearray, 
# split the bytearray at the first null, 
# take the first split,
# convert / decode to string. 
maxlen = 10
txt = bytearray(ev.get_param().__dereference__(maxlen)).split(b'\x00', 1)[0].decode('utf-8')

Although it works for me [TM], it does not look like it was meant to be that way.

What gives?

[kdschlosser]

ev.get_param().__cast__(bytearray)

I believe that is what you need to do.

[h-milz]

No, sadly it refuses any __cast__. In this case

SyntaxError: Can't convert int to bytearray!

[kdschlosser]

ok don’t pass bytearray to __cast__. leave it empty

[h-milz]

Same. I believe I tried all possible combinations. Maybe what is needed is the equivalence of a (char *) typecast as in C. For some other data types, this was implemented. But there is no lv.char_t. Which I think would be pretty simple to implement.

[kdschlosser]

what are you trying to pass in the user_data.

[h-milz]

In this case, nothing. lv.EVENT.INSERT passes the text to be inserted in param. The user_data field is None.

[h-milz]

And yes, the supposedly obvious way of using str() doesn’t work either. This text fragment

def ta_ev_cb_insert(ev):
    obj = ev.get_param()
    print (f"obj is {type(obj)}, {dir(obj)}")
    txt = str(obj)
    print (f"txt is \"{txt}\"")
...
ta.add_event_cb(ta_ev_cb_insert, lv.EVENT.INSERT, None)

gives this result no matter which text is sent to the textarea:

obj is <class 'Blob'>, ['__class__', '__cast__', '__dereference__']
txt is "Blob"

Which is probably fine because “Blob” is not a type that str() is supposed to work with. Trying to use __cast__ reproducibly leads to the error message shown.

SyntaxError: Can’t convert to bytearray!
…
SyntaxError: Can’t convert to str!

The obvious reason is that lv_event_t looks like this:

struct _lv_event_t {
    void * current_target;
    void * original_target;
    lv_event_code_t code;
    void * user_data;
    void * param;
    lv_event_t * prev;
    uint8_t deleted : 1;
    uint8_t stop_processing : 1;
    uint8_t stop_bubbling : 1;
};

event.get_param() just returns the (void *)

void * lv_event_get_param(lv_event_t * e)
{
    return e->param;
}   

And from then on there seems no way of “officially” getting the former (char *) back. Except the ugly hack shown in the first post.

Going one step back, the textarea insert handler does this:

static lv_result_t insert_handler(lv_obj_t * obj, const char * txt)
{
    ta_insert_replace = NULL;
    lv_obj_send_event(obj, LV_EVENT_INSERT, (char *)txt);
    ... 

lv_obj_send_event() is defined as

lv_result_t lv_obj_send_event(lv_obj_t * obj, lv_event_code_t event_code, void * param) 
{
    ...
    e.param = param;
    ...

which is where the implicit type cast happens. I think in C++ you would get a compiler error from this

So I think this should be properly documented somewhere, or someone with better programming skills than I needs to fix this binding. I understand that LVGL for Micropython is supposed to provide the LVGL for C API as closely as possible, but this is just broken.

[kdschlosser]

OK since it is a string that is being stored and you don’t know what the length of the string is correct? a person can only assume that it will be null terminated.

But lets look at the “Blob” data type in the binding…

OK so what i can see in the code is the “Blob” data type is simply a wrapper around other types. It doesn’t have the ability to be iterated over and it doesn’t have the ability to be indexed. It can omnly be cast or converted into a memoryview object. Since it is a string we are dealing with we would need to convert it into a memoryview object but the length of the string is not known. That’s OK tho because there are mechanics built into the binding code to handle that.

def bytes_to_str(data):

    output = b''
    for char in data:
        if char == b'\x00':
            break
        output += char
    try:
        return output.decode('utf-8')
    except:
        return output.decode('latin1')


def ta_ev_cb_insert(ev):
    obj = ev.get_param()
    # __dereference__ converts the blob into a memoryview object which is simply a pointer to
    # the memory address that holds the information we are looking for.
    # by not supplying a parameter to the call it triggers internal mechanics that will collect the 
    # size of the data that is being stored. IF MicroPython has coded in the "to_bytes" function
    # into a memoryview we should be able to collect the stored data as a python bytes object and
    # then convert it to a string. I do not know if the data being stored is a fixed size where the
    # memory used after the last readable character is a null character followed by garbage 
    # data or if the entire address is going to be filled with readable data.
    txt = bytes_to_str(bytes(obj.__dereference__()))
    print (f"txt is \"{txt}\"")

Give that a shot and see what happens.

[kdschlosser]

The reason why this is a goofy issue is because the data doesn’t originate from a python object. It is originating from c code which is the reason why you cannot use an empty call to __cast__. we are dealing with some unknowns the biggest one is the length of the data that is stored the second is if the text is null terminated or not. Looking at the LVGL C code that you posted it appears that the text is going to be null terminated. so the code I posted may or may not work. It might not work because we simply do not know where that null terminated character is and that is what is ultimately is going to be the end of the string. If the char * was allocated dynamically then the size of the data stored at the memory address is not going to be known. we don’t want to feed in some random value to __dereference__ as the length. we need to calculate the proper length of data so we don’t run into any issues with pulling data from some area of memory that has nothing to do with the data that is trying to be collected. Doing that would lead to undefined behavior.

If that is the case then using code like this would be the best approach.

def to_str(obj):
    output = b''
    count = 0
    
    while not output.endswith(b'\x00'):
        count += 1
        output = bytes(obj.__dereference__(count))

    try:
        return output[:-1].decode('utf-8')
    except:
        return output[:-1].decode('latin1')


def ta_ev_cb_insert(ev):
    txt = to_str(ev.get_param())
    print(f"txt is \"{txt}\"")

[picpic020960]

Bonjour ,
exists the same issue with

void * current_target;
void * original_target;

?
And with the ‘event’ use ?

(‘cast’ and ‘dereference’ )

[h-milz]

@picpic020960
No, because the functions get_target_obj() and get_current_target_obj() do the type cast and return a correct lv_obj_t object.

[h-milz]

@kdschlosser
Yes, I had a similar approach before using a loop. Now I dereference with a length that I know from my application will not be surpassed (these strings are all < 5 or 6 characters). The blob pointer appears to point at null terminated strings that are just concatenated (somewhere on the heap I suppose). The first one is the one I am looking for. But instead of collecting the characters until \x00 and assembling the string in a Python loop I figured that doing as much as possible in C would be faster, hence my approach in my first post.

txt = bytearray(ev.get_param().__dereference__(maxlen)).split(b'\x00', 1)[0].decode('utf-8')

does exactly what you outlined but is uses functions implemented in C.

Still …

Do you think we could set up a feature request that in a similar way to get_current_obj() does a get_string_obj()?

OK here’s a hack. In lib/lvgl/src/misc/lv_event.c I added

char * lv_obj_get_str_from_blob(void * blob)
{
    return (char *) blob;
}       

and in lv_event.h the corresponding prototype. Now in my insert callback, I can say

blob = ev.get_param()
txt = lv.obj.get_str_from_blob(blob)
print (f"type txt: {type(txt)}")
print (f"txt = \"{txt}\"")

and the result is (after pressing the “pi” key):

type txt: <class ‘str’>
txt = “pi”

OK, the place may not the correct one, but you get the idea. I admit that the lv.EVENT.INSERT event is probably the only place where this is useful because afaik most other widgets have their specific get_something functions, where the cast happens in the underlying C code anyway. Which is exactly what the textarea insert event lacks.

This works equally well when passing a string as user_data (to any event). When passing a dict, casting it using blob.__cast__() works as designed in mp_blob_cast().

[picpic020960]

Not a blob as return ?
Édit :
OK ,I see my error ; exists get_target and get_target_obj : this last IS good for read text label of button in cb.

[kdschlosser]

a char * is not a python str object. it is closer to a bytes object but it still isn’t even one of those.

From what I am seeing in the generated binding code a blob supports the buffer protocol which means it can be passed directly to a bytearray. When the bytearray is constructed is check to see if the passed in parameter supports the buffer protocol and if it does then the method that is attached to collect the buffer data get called which will populate the bytearray, once it is a bytearray we are able to split it on the first NULL collecting the section before the NULL and converting that section from a bytearray to a string by using the decode method.

data = bytearray(ev.get_param()).split(b'\x00', 1)[0].decode('utf-8')

give the code above a try and see if it works. That is going to be the fastest way to do it if it works. For some reason I believe it is only going to show the first 4 bytes tho. I say that because of the use of sizeof in the binding code and it is using on the char * that is stored and that I believe is being dynamically allocated.

This is what the code is for the collection of the data using the buffer protocol.

    bufinfo->buf = &self->data;
    bufinfo->len = sizeof(self->data);
    bufinfo->typecode = BYTEARRAY_TYPECODE;

I believe that sizeof is going to return the size of the type for self->data and that type is void *

what would size of show if I did this…

char * buf = (char *)malloc(sizeof(char) * 20);

sizeof(buf) = ???

That is going to return the size of a char * which is a pointer and on the ESP32 pointers are 32 bits or 4 bytes. in order for this to work properly it would have to iterate over the data to locate the NULL and set the length to the position before that NULL. In order to do that the type that is being stored in self->data would need to be known to be able to correctly determine what to do.

I would think this is a bug in the binding code because the length is always going to be set to 4.

But If you look at the code you can see that the buffer data is being set using & and the data field has a type of void *.

Since the & is being used to point to a pointer should that mean that what is being passed as the buffer is the memory address? that would make sense that the length is being set to 4 bytes.

If that is the case then there is another way to collect the data using the uctypes structure.

Or I could add a couple of helper functions to collect the data. One would return a bytes object which would be a copy of the data. The other would return a bytearray which would allow for an in place modification of the data and the third would be to return an actual python str which would be a copy of the data.

[kdschlosser]

@h-milz

Now I dereference with a length that I know from my application will not be surpassed

You say that it will not be surpassed. But that means it can be shorter than that length so what is going to happen is it is going to be adding portions of memory that have nothing to do with the text you are trying to collect. You are basically having a memory overrun…

say you have the 1 byte blocks of memory

so as an exmaple if you set the max length to 10 bytes and the data that is stored in memory is seen like what is below.

T  |  E  |  S  |  T  |  NULL  |  S  |  O  |   M  |  E  |  O  |   T  |  H  |  E  |  R  |  D  |  A  |  T  |  A

The returned memoryview from __dereference__ is going to point to

T  |  E  |  S  |  T  |  NULL  |  S  |  O  |   M  |  E  |  O

It is going to contain the data from some other things that is stored in memory. While I know you are splitting on the NULL so it shouldn’t be an issue but if you do something in the same fashion you could end up mucking up the data stored in memory for something else. You need to be careful doing what you are doing.

In the example where I loop increasing the number of bytes that get dereferenced ensures that no data past that first NULL will get messed with. It’s safer code even if it mmight be slower. Now that being said, if it’s speed you are after then you can do this…

This will run at the speed of C code because we are using the viper code emitter. The viper code emitter deals with memory addresses that is what the prt8 does. It uses the memory address and from there you can access 8 bits at a time using an index. index 0 is the data that is stored at the memory address that is passed and 1 is the next memory location. we return the total number of bytes that have been added to the buffer that was passed as a memory location just like you would see in C code when an array gets passed to a function so it can be filled. This guards against accessing areas of memory that contain data that is not the text you are wanting to collect

This doesn’t allow you to make an in place change of the as it makes a copy of the data.

import micropython


@micropython.viper
def get_bytes(blob: ptr8, buf: ptr8) -> int:
    count = 0
    # blob has 2 fields the first field has a type of mp_obj_base_t and that
    # structure only has a single field which is a pointer to mp_obj_type_t.
    # so the size of mp_obj_base_t is going to be 4 bytes.
    # so we want to start at the 4th byte in from the memory address
    # being pointed to by the blob parameter. which is what is being
    # done on the next line
    data = ptr32(blob[4])

    while True:
        if uint(data[count]) == 0:
            break
        buf[count] = uint(data[count])

        count += 1

    return count


def ta_ev_cb_insert(ev):
    # you can set the bytearray to max_len.
    buf = bytearray(max_len)
    blob = ev.get_param()
    
    length = get_bytes(blob, buf)
    print(f"txt is \"{buf[:length].decode("utf-8")}\"")

[h-milz]

I see your point. My desire to have things as fast as possible comes from the fact that I’m using the lv.EVENT.INSERT to preprocess (and potentially, manipulate) every string that comes from the keyboard, which means I need to know the text to be inserted in the first place. But then I may be actually barking up the wrong tree because the subsequent if-elif-else construct has far more influence on the timing of key entry. On the Unix port, this is practically unnoticable, but on the ESP32 I may have to make up a different strategy where insert manipulations are kept at a minimum.

BTW I agree that char * is not a Python str object. But the binding that is automatically generated in lib/micropython/ports/{port}/{build-dir}/lv_mp.c by gen/lvgl_api_gen_mpy.py does the right thing, as shown:

static mp_obj_t mp_lv_obj_get_str_from_blob(size_t mp_n_args, const mp_obj_t *mp_args, void *lv_func_ptr)
{
    void *blob = mp_to_ptr(mp_args[0]);
    char * _res = ((char *(*)(void *))lv_func_ptr)(blob);
    return convert_to_str((void*)_res);
}

static MP_DEFINE_CONST_LV_FUN_OBJ_STATIC_VAR(mp_lv_obj_get_str_from_blob_mpobj, 1, mp_lv_obj_get_str_from_blob, lv_obj_get_str_from_blob);

The other python code

data = bytearray(ev.get_param()).split(b'\x00', 1)[0].decode('utf-8')

imho is not supposed to work because get_param() just returns a 4-byte address to the blob, and there is not much that can be split. It just spits out a UnicodeError.

OK, back to the drawing board as far as keyboard handling I think I can do without or with only minor input string manipulation by splitting the keyboard (i.e. multiple keyboard objects) in several parts not all of which will write into the textarea. To give you an idea what I’ve been babbling about the whole time, below is a pic of the current less-than-prototype. The challenge comes from the need to have a couple of silent keys like SHIFT, ALPHA, hyp, or AC. Plus a longer list of case decisions. Away with it!

By the way I also experimented with kb.get_button_text() but it appears that in this setup, the behaviour is even more laggy, also because the text would be inserted in the textarea by the default kb handler and I had to delete it again for the silent keys.

[kdschlosser]

With your specific use case it is going to be better if you roll your own keyboard plugin.

One of the things that I have learned with using LVGL is to use the lv.obj as a container. What you do is you create a style that sets all colors to black, all of the opacity to 0. set the pads, margins, border widths, shadow settings, outline widths all to 0.

here is an example of what I am talking about…

import lvgl as lv


CALC_BUTTON_PRESSED = lv.event_register_id()


class Button(object):

    def __init__(self, parent, label, keycode):
        self.parent = parent
        self.keycode = keycode
        self.obj = obj = lv.button(parent.obj)
        self.label = lv.label(obj)
        self.label.set_text(label)

        obj.set_style_text_color(lv.color_hex(0x00FF00), lv.PART.MAIN)
        obj.set_style_text_color(lv.color_hex(0xFF0000), lv.STATE.PRESSED)

        obj.set_style_bg_color(lv.color_hex(0x000000), lv.PART.MAIN)
        obj.set_style_bg_opa(255, lv.PART.MAIN)

        obj.set_style_border_width(2, lv.PART.MAIN)
        obj.set_style_border_color(lv.color_hex(0x00FF00), lv.PART.MAIN)
        obj.set_style_border_opa(255, lv.PART.MAIN)
        obj.set_style_border_opa(0, lv.STATE.PRESSED)

        obj.set_style_margin_bottom(0, lv.PART.MAIN)
        obj.set_style_margin_left(0, lv.PART.MAIN)
        obj.set_style_margin_right(0, lv.PART.MAIN)
        obj.set_style_margin_top(0, lv.PART.MAIN)

        obj.set_style_outline_color(lv.color_hex(0xFF0000), lv.PART.MAIN)
        obj.set_style_outline_pad(0, lv.PART.MAIN)
        obj.set_style_outline_width(2, lv.PART.MAIN)
        obj.set_style_outline_opa(0, lv.PART.MAIN)
        obj.set_style_outline_opa(255, lv.STATE.PRESSED)

        obj.set_style_pad_left(0, lv.PART.MAIN)
        obj.set_style_pad_right(0, lv.PART.MAIN)
        obj.set_style_pad_top(0, lv.PART.MAIN)
        obj.set_style_pad_bottom(0, lv.PART.MAIN)

        obj.set_style_radius(5, lv.PART.MAIN)

        obj.set_style_shadow_offset_x(2, lv.PART.MAIN)
        obj.set_style_shadow_offset_y(2, lv.PART.MAIN)
        obj.set_style_shadow_color(lv.color_hex(0xC0C0C0), lv.PART.MAIN)
        obj.set_style_shadow_opa(210, lv.PART.MAIN)
        obj.set_style_shadow_opa(0, lv.STATE.PRESSED)

        obj.set_style_shadow_spread(5, lv.PART.MAIN)
        obj.set_style_shadow_width(3, lv.PART.MAIN)

        obj.remove_flag(lv.obj.FLAG.EVENT_BUBBLE)
        obj.remove_flag(lv.obj.FLAG.SCROLLABLE)

        obj.add_event_cb(self.on_pressed, lv.EVENT.PRESSED, None)

    def on_pressed(self, evt):
        evt.stop_bubbling()
        self.obj.send_event(CALC_BUTTON_PRESSED, self)

    def set_pos(self, x, y):
        self.obj.set_pos(x, y)

    def set_size(self, width, height):
        self.obj.set_size(width, height)
        self.obj.layout()
        self.label.center()

    def get_keycode(self):
        if self.keycode is None:
            return self.label.get_text()

        return self.keycode


class Container(object):
    def __init__(self, parent):
        if hasattr(parent, 'obj'):
            self.parent = parent
            self.obj = obj = lv.obj(parent.obj)
        else:
            self.parent = None
            self.obj = obj = lv.obj(parent)

        obj.set_style_bg_color(lv.color_hex(0x000000), lv.PART.MAIN)
        obj.set_style_bg_opa(0, lv.PART.MAIN)
        obj.set_style_border_opa(0, lv.PART.MAIN)
        obj.set_style_border_width(0, lv.PART.MAIN)
        obj.set_style_margin_bottom(0, lv.PART.MAIN)
        obj.set_style_margin_left(0, lv.PART.MAIN)
        obj.set_style_margin_right(0, lv.PART.MAIN)
        obj.set_style_margin_top(0, lv.PART.MAIN)
        obj.set_style_outline_opa(0, lv.PART.MAIN)
        obj.set_style_outline_pad(0, lv.PART.MAIN)
        obj.set_style_outline_width(0, lv.PART.MAIN)
        obj.set_style_pad_left(0, lv.PART.MAIN)
        obj.set_style_pad_right(0, lv.PART.MAIN)
        obj.set_style_pad_top(0, lv.PART.MAIN)
        obj.set_style_pad_bottom(0, lv.PART.MAIN)
        obj.set_style_radius(0, lv.PART.MAIN)
        obj.set_style_shadow_offset_x(0, lv.PART.MAIN)
        obj.set_style_shadow_offset_y(0, lv.PART.MAIN)
        obj.set_style_shadow_opa(0, lv.PART.MAIN)
        obj.set_style_shadow_spread(0, lv.PART.MAIN)
        obj.set_style_shadow_width(0, lv.PART.MAIN)
        
        obj.remove_flag(lv.obj.FLAG.SCROLLABLE)
        
    def set_size(self, width, height):
        self.obj.set_size(width, height)

    def set_pos(self, x, y):
        self.obj.set_pos(x, y)


class ButtonGroup(Container):
    
    def __init__(self, parent):
        super().__init__(parent)
        
        self.obj.add_flag(lv.obj.FLAG.HIDDEN)

    def add_button(self, label, keycode, x, y, width, height):
        button = Button(self, label, keycode)
        button.set_pos(x, y)
        button.set_size(width, height)

    def layout(self):
        self.obj.layout()
        
    def show(self, flag):
        if flag:
            self.obj.remove_flag(lv.obj.FLAG.HIDDEN)
        else:
            self.obj.add_flag(lv.obj.FLAG.HIDDEN)


class NumberPad(ButtonGroup):

    def __init__(self, parent):
        super().__init__(parent)

        self.add_button('1', None, 0,  0, 30, 20)
        self.add_button('2', None,35, 0, 30, 20)
        self.add_button('3', None,70, 0, 30, 20)

        self.add_button('4', None,0, 25, 30, 20)
        self.add_button('5', None,35, 25, 30, 20)
        self.add_button('6', None,70, 25, 30, 20)

        self.add_button('7', None,0, 30, 30, 20)
        self.add_button('8', None,35, 30, 30, 20)
        self.add_button('9', None,70, 30, 30, 20)

        self.add_button('9', None,0, 55, 100, 20)
        

class ArithmeticPad(ButtonGroup):

    def __init__(self, parent):
        super().__init__(parent)

        self.add_button('+', None, 0,  0, 30, 20)
        self.add_button('-', None,35, 0, 30, 20)
        self.add_button('*', None,70, 25, 30, 20)
        self.add_button('/', None,70, 50, 30, 20)


class Keyboard(Container):

    def __init__(self, parent):
        super().__init__(parent)

        self.number_pad = NumberPad(self)
        self.number_pad.set_size(100, 95)
        self.number_pad.set_pos(0, 25)
        self.number_pad.show(True)

        self.arithmetic_pad = ArithmeticPad(self)
        self.arithmetic_pad.set_size(100, 70)
        self.arithmetic_pad.set_pos(35, 0)
        self.arithmetic_pad.show(True)


class Calculator(Container):

    def __init__(self):
        scrn = lv.screen_active()

        super().__init__(scrn)
        self.obj.set_style_bg_opa(255, lv.PART.MAIN)

        width = scrn.get_width()
        height = scrn.get_height()

        self.obj.set_size(width, height)

        half_height = int(height / 2) - 15
        width -= 20

        self.text_area = lv.text_area(self.obj)
        self.text_area.set_size(width, half_height)
        self.text_area.set_pos(10, 10)

        self.keyboard = Keyboard(self)
        self.keyboard.set_size(width, half_height)
        self.keyboard.set_pos(10, half_height + 20)

        self.obj.add_event_cb(self.button_pressed_cb, CALC_BUTTON_PRESSED, None)

    def button_pressed_cb(self, e):
        button = e.get_param().__cast__()
        self.text_area.append_text(button.get_keycode())



calc = Calculator()

[h-milz]

True. Over the weekend I came up with something similar albeit a bit more specific still. I now create and place buttons individually and depending on the type of function I call only 3 or 4 different callbacks with very few case decisions. The downside of higher memory requirements are no big deal on an esp32 with large psram.

[kdschlosser]

By grouping the buttons together like numbers, trig functions, geometry etc you can create what looks like tabs that can be clicked on to change the buttons that are being displayed. you group them together as children of an object that you have all colors set to have an opacity of 0. Basically what you are creating is a container and you can add or remove the hidden flag from that container and all children in that container will hide along with it. Doing this does have the higher cost of memory but as you said with the ESP32 and the PSRAM this is a non issue. What you gain is the performance from not having to create and delete buttons over and over again and you get the speed of not having to iterate over a list of buttons hiding them one at a time. With the container is becomes simple because LVGL when rendering will see the hidden object and it not going to bother checking it’s children.

By removing the clickable flag for the container it not only becomes transparent visually but also physically. so you can group buttons together that are actually on opposite sides of the keypad with a different group sitting in the middle.

What I do when I am creating a GUI that is this complex is I open up a program called “Paint.NET” and I will create a new image the same size as the display. I then create a layer for each container object I am using. I draw in each widget that is in the containers. I get all of the spacing and things worked out that way. I hammer out all of the layout stuff that way.