Opening files on ESP32 issues

mhepp · January 7, 2020, 11:40pm

Description

I have problems with loading images from file on ESP32. When I open file manualy, image is shown on screen, but when I try to open file via img.set_src('/image.bin'), nothing happened.

Deep investigation points to problem with opening files by lvgl on ESP’s internal FS. All lv.fs* functions ends with ret code 3 (LV_FS_RES_FS_ERR)

Platform

ESP32

Question:

If I understand well, I must define file driver (lv.fs_drv_t()) to be able access files on internal FS?

Code:

Non-functional code:

import lvgl as lv
lv.init()

from ili9341 import ili9341
from xpt2046 import xpt2046

disp = ili9341()
touch = xpt2046(cal_x0=441, cal_y0=3810, cal_x1=3406, cal_y1=443)

scr = lv.obj()
lv.scr_load(scr)
lv.img.cache_set_size(16)

img = lv.img(scr)
img.set_src('/vz2.bin')

img.align(scr, lv.ALIGN.CENTER, 0, 0)

This code DO NOT load image onto screen.

This is output on console:

Info: lv_init ready     (../../lib/lv_bindings/lvgl/src/lv_core/lv_obj.c #121)
Warn: lv_init: already inited   (../../lib/lv_bindings/lvgl/src/lv_core/lv_obj.c #83)
ILI9341 initialization completed
Enable backlight
Double buffer
Info: Screen create ready       (../../lib/lv_bindings/lvgl/src/lv_core/lv_obj.c #234)
Info: Screen create ready       (../../lib/lv_bindings/lvgl/src/lv_core/lv_obj.c #234)
Info: Screen create ready       (../../lib/lv_bindings/lvgl/src/lv_core/lv_obj.c #234)
Info: Screen create ready       (../../lib/lv_bindings/lvgl/src/lv_core/lv_obj.c #234)
Info: image created     (../../lib/lv_bindings/lvgl/src/lv_objx/lv_img.c #110)
Info: image draw: cache miss, cached to an empty entry      (../../lib/lv_bindings/lvgl/src/lv_draw/lv_img_cache.c #120)
Warn: Image draw cannot open the image resource         (../../lib/lv_bindings/lvgl/src/lv_draw/lv_img_cache.c #129)
Warn: Image draw error  (../../lib/lv_bindings/lvgl/src/lv_draw/lv_draw_img.c #62)
Info: image draw: cache miss, cached to an empty entry  (../../lib/lv_bindings/lvgl/src/lv_draw/lv_img_cache.c #120)
Warn: Image draw cannot open the image resource         (../../lib/lv_bindings/lvgl/src/lv_draw/lv_img_cache.c #129)
Warn: Image draw error  (../../lib/lv_bindings/lvgl/src/lv_draw/lv_draw_img.c #62)
Info: image draw: cache miss, cached to an empty entry  (../../lib/lv_bindings/lvgl/src/lv_draw/lv_img_cache.c #120)
Warn: Image draw cannot open the image resource         (../../lib/lv_bindings/lvgl/src/lv_draw/lv_img_cache.c #129)
Warn: Image draw error  (../../lib/lv_bindings/lvgl/src/lv_draw/lv_draw_img.c #62)
Info: image draw: cache miss, cached to an empty entry  (../../lib/lv_bindings/lvgl/src/lv_draw/lv_img_cache.c #120)
Warn: Image draw cannot open the image resource         (../../lib/lv_bindings/lvgl/src/lv_draw/lv_img_cache.c #129)
Warn: Image draw error  (../../lib/lv_bindings/lvgl/src/lv_draw/lv_draw_img.c #62)

It looks like unable to open file.

Working code:

is based on load png example, but implements only header reader:


def get_img_info(decoder, src, header):
    # Only decode header from image

    if lv.img.src_get_type(src) != lv.img.SRC.VARIABLE:
        return lv.RES.INV

    img_header = bytes(lv.img_dsc_t.cast(src).data.__dereference__(4))

    hdr = struct.unpack(">L", img_header)[0]

    cf = (hdr >> 24) & 0x1f
    w = (hdr >> 18) & 0x3f | (hdr & 0x1f00) >> 2
    h = (hdr >> 13) & 0x07 | (hdr & 0xff) << 3

    header.always_zero = 0
    header.w = w
    header.h = h
    header.cf = cf

    print("width=%d, height=%d, cf=%d" % (header.w, header.h, header.cf))

    return lv.RES.OK


def show_img(lv_img, path):
    # wrapper to load image via one function call

    with open(path,'rb') as f:
       img_data = f.read()
    f.close()

    img_dsc = lv.img_dsc_t()
    img_dsc.data = img_data
    img_dsc.data_size = len(img_data)    

    raw_dsc = lv.img_dsc_t()
    get_img_info(None, img_dsc, raw_dsc.header)
    raw_dsc.data = img_data[4:] # stripped header
    raw_dsc.data_size = raw_dsc.header.w * raw_dsc.header.h * lv.color_t.SIZE
    
    lv_img.set_src(raw_dsc)
    

disp = ili9341()
touch = xpt2046(cal_x0=441, cal_y0=3810, cal_x1=3406, cal_y1=443)
    
scr = lv.obj()
lv.scr_load(scr)
lv.img.cache_set_size(2)

img = lv.img(scr)
lp.show_img(img, '/vz2.bin')

img.align(scr, lv.ALIGN.CENTER, 0, 0)

Same image is displayed…

amirgon · January 8, 2020, 7:43am

I think you need to register a file-system driver.
You can do that from your micropython script by calling lv.fs_drv_register. First you need to create a lv.fs_drv_t struct, initialize the callbacks and pass it to lv.fs_drv_register.

Actually I never tried this, but I believe it should work.

@kisvegabor, @embeddedt do we need to initialize all the callbacks in lv_fs_drv_t? Do you have an example?

In your working code, how do you actually read the file from the file system?

mhepp · January 8, 2020, 9:13am

@amirgon: I am using standard python call open(). It is in show_img():

def show_img(lv_img, path):
    # wrapper to load image via one function call
    with open(path,'rb') as f:
        img_data = f.read()
    f.close()
    ...

Yes, ok, I thought it is this situation. As I look into documentation:

@kisvegabor, @embeddedt: example should be very fine, I think, I must create some wrappers to standard python calls, for example:

def my_open_cb(fs_file, filename, mode):
    with open(filename, mode) as f:
        fs_file.file_d = f
        return 0
    return 1

This should be a part of lvgl initialization process…

embeddedt · January 8, 2020, 1:24pm

No. You only need to implement the functionality that you need. Please see the filesystem documentation.

mhepp · January 8, 2020, 3:15pm

OK, I have call-back functions for open, read, seek, tell, write and close:

def esp32fs_open_cb( fs_file, path, mode):

    p_mode = 'rb' if mode & 0x2 else ''
    p_mode = 'wb' if mode & 0x1 else ''
    p_mode = 'rb+' if mode & 0x3 else ''

    if p_mode == '':
        return 11

    with open(path, p_mode) as f:
        fs_file.file_d = f
        return 0 
    return 4


def esp32fs_read_cb( fs_file, buf, btr, br):

    try:
        buf = fs_file.file_d.read(btr)
    except:
        return 3

    br = len(buf)

    return 0

# this are similar to read...
def esp32fs_seek_cb( fs_file, pos):
    ...

def esp32fs_tell_cb( fs_file, pos):
    ...

def esp32fs_write_cb( fs_file, buf, btw, bw):
    ...

def esp32fs_close_cb( fs_file):
    ...

Then I set fs_drv:

fs_drv = lv.fs_drv_t()
fs_drv.letter = 2
fs_drv.open_cb = esp32fs_open_cb
fs_drv.read_cb = esp32fs_read_cb
fs_drv.seek_cb = esp32fs_seek_cb
fs_drv.tell_cb = esp32fs_tell_cb
fs_drv.close_cb = esp32fs_close_cb
lv.fs_drv_register(fs_drv)

fs_drv.letter = 2 is result of experiments with various values. Partitions are here:

<Partition type=1, subtype=2, address=36864, size=24576, label=nvs, encrypted=0>
<Partition type=1, subtype=1, address=61440, size=4096, label=phy_init, encrypted=0>
<Partition type=0, subtype=0, address=65536, size=2097152, label=factory, encrypted=0>
<Partition type=1, subtype=129, address=2162688, size=2031616, label=vfs, encrypted=0>

Try to load image from file:

scr = lv.obj()
lv.scr_load(scr)
lv.img.cache_set_size(16)

img = lv.img(scr)
img.set_src('2:/vz2.bin')

And result is: Image is not shown and no message written:

Info: lv_init ready     (../../lib/lv_bindings/lvgl/src/lv_core/lv_obj.c #121)
ILI9341 initialization completed
Enable backlight
Double buffer
Info: Screen create ready       (../../lib/lv_bindings/lvgl/src/lv_core/lv_obj.c #234)
Info: Screen create ready       (../../lib/lv_bindings/lvgl/src/lv_core/lv_obj.c #234)
Info: Screen create ready       (../../lib/lv_bindings/lvgl/src/lv_core/lv_obj.c #234)
Info: Screen create ready       (../../lib/lv_bindings/lvgl/src/lv_core/lv_obj.c #234)
Info: image created     (../../lib/lv_bindings/lvgl/src/lv_objx/lv_img.c #110)

I am on end of my possibilities now. Do you have any idea what I am doing wrong or what next?

Thanks, mhepp.

amirgon · January 8, 2020, 8:55pm

@mhepp There are some problems with your code.
First, I suggest you don’t try to catch all exceptions (or at least print them out), so you would known what problems you have.

For example, esp32fs_read_cb() calls fs_file.file_d.read().
However, fs_file.file_d is a void * variable and needs to be casted back to file object.
Casting a generic python object to C pointer and back can be done using dict as a mediator:
Something like

fs_file.file_d = {'file': f}
...
d = fs_file.file_d.cast()
buf = d['file'].read(btr)

In case of a generic pointer such as file_d, lvgl binding only allows converting it to/from dict. (or to/from lvgl structs, which is not the case here).

mhepp · January 9, 2020, 11:53am

I think, there is something wrong in fs driver. I changed functions – added cast() and debug print(). But output is without change – it looks like cb functions are not called. Output is still same.

When I create lvf = lv.fs_file_t(), then lvf.drvis None. Is this correct?

fs_drv = lv.fs_drv_t()
fs_drv.letter = 2
fs_drv.open_cb = file_drv.esp32fs_open_cb
fs_drv.read_cb = file_drv.esp32fs_read_cb
fs_drv.seek_cb = file_drv.esp32fs_seek_cb
fs_drv.tell_cb = file_drv.esp32fs_tell_cb
fs_drv.close_cb =file_drv.esp32fs_close_cb
    
lv.fs_drv_register(fs_drv)
    
scr = lv.obj()
lv.scr_load(scr)
lv.img.cache_set_size(16)
    
img = lv.img(scr)
img.set_src('2:/vz2.bin')

Please, how to determine correct partition number? And is filepath in set_src correct?

file_drv.txt (1.9 KB)

amirgon · January 9, 2020, 10:13pm

Please read carefully the filesystem doc.

Here is a simple example that does nothing, but shows that the callbacks are called.

import lvgl as lv
lv.init()

# Register your display and input driver here

fs_drv = lv.fs_drv_t()
lv.fs_drv_init(fs_drv)

def open_cb(drv, file_p, path, mode):
    print("open_cb called!")
    return lv.FS_RES.OK

fs_drv.letter = ord('S')
fs_drv.open_cb = open_cb
lv.fs_drv_register(fs_drv)
    
img = lv.img()
img.set_src('S:/vz2.bin')

When running this, I’m getting open_cb called! as expected.
I think that you were missing the ord function in your code when setting drv letter.

mhepp · January 10, 2020, 8:33am

Jupí, @amirgon, thank you very much. Problem was in letter. If I understand this well, this letter is for pairing file driver with specific filepath. So, I can use any letter I want, but must be same in fs driver and filepath. Is it correct?

Callback functions are called after fixing this.

amirgon · January 10, 2020, 8:41am

That’s correct.

mhepp · January 11, 2020, 2:07pm

I have a update. Function to open file is OK, but read (for start) is ending with error. Code is here:

def esp32fs_open_cb(drv, fs_file, path, mode):

    print("Info: opening file '{}'".format(path))

    p_mode = 'rb' if mode & 0x2 else ''
    p_mode = 'wb' if mode & 0x1 else ''
    p_mode = 'rb+' if mode & 0x3 else ''

    if p_mode == '':
        print("Warn: open mode error: {}".format(mode))
        return lv.FS_RES.INV_PARAM
    cast_fs = lv.fs_file_t.cast(fs_file)

    print(type(cast_fs.file_d), cast_fs.file_d)

    with open(path, p_mode) as f:
        cast_fs.file_d = { 'file': f }
        print("Info: File {} opened".format(path))
        tmp = cast_fs.file_d.cast()
        print(type(tmp), tmp)
        return lv.FS_RES.OK

    print("Warn: Opening file {} failed".format(path))
    return lv.FS_RES.FS_ERR


def esp32fs_read_cb(drv, fs_file, buf, btr, br):
    print("Info: called read( {})".format(btr))
    cast_fs = lv.fs_file_t.cast(fs_file)

    try:
        print("read(): castinng")
        tmp = cast_fs.file_d.cast()
        print(type(tmp), tmp)
        buf = tmp['file'].read(btr)

    except Exception as e:
        print("Read() caused exception (probably OSError)", uerrno.errorcode[e.args[0]])
        return 3

    br = len(buf)

    print("Info: read {}B".format(br))
    return 0

Calling this ends with EINVAL, which is not very specific, so I add printf() to file_obj_read() in extmod/vfs_fat_file.c.

And result is there:

Info: opening file 'vz2.bin'
<class 'Blob'> Blob
Info: File vz2.bin opened
<class 'dict'> {'file': <io.FileIO 3f81ef30>}
Info: called read( 4)
read(): castinng
<class 'dict'> {'file': <io.FileIO 3f81ef30>}
VFAT file_obj_read: fp: 1065479988, buf: 1065480352, size: 4, res: 9
Read() caused exception (probably OSError) EINVAL
Info: called close()
Info: File descriptor closed

res 9 means FR_INVALID_OBJECT, this should points to messing with address and I have probably winner: <class 'dict'> {'file': <io.FileIO 3f81ef30>} line is struct with filedescriptor and 0x3f81ef30 is its address (I hope…). BUT, in file_obj_read() have filedescriptor address 1065479988 (0x3f81ef34), so 4B more. And it is 4B everytime.

~~It looks like problem with trasformation of mpy’s lv.fs_file_t.file_d to C’s pyb_file_obj_t in file_obj_read().~~

Struct pyb_file_obj_t contains more things, so maybe address of filedescriptor is offsetted by size of this things…

amirgon · January 11, 2020, 7:47pm

mhepp:

    with open(path, p_mode) as f:
        cast_fs.file_d = { 'file': f }
        print("Info: File {} opened".format(path))
        tmp = cast_fs.file_d.cast()
        print(type(tmp), tmp)
        return lv.FS_RES.OK

In python, a “with” statement declares a context manager control structure.
The idea is that the resource is freed when exiting the “with” block.

It’s wrong to use context manager here, because the resource (the file) must not be freed (closed) when exiting the “with” block. That’s because a reference to the file is saved on file_d and used later on.

The correct place to close the file is close_cb callback.

mhepp · January 12, 2020, 10:54pm

@amirgon, yes, of course… You are right. I don’t know where I left my mind… Fixed:

def esp32fs_open_cb(drv, fs_file, path, mode): 
 
    print("Info: opening file '{}', mode {}".format(path, mode)) 
 
    p_mode = '' 
    if mode == 1 : 
        p_mode = 'wb' 
    elif mode == 2: 
        p_mode = 'rb'
    elif mode == 3:
        p_mode = 'rb+'
    else:
        p_mode = ''

    if p_mode == '':
        print("Warn: open mode error: {} ({})".format(mode))
        return lv.FS_RES.INV_PARAM

    cast_fs = lv.fs_file_t.cast(fs_file)

    try:
        f = open(path, p_mode)
        cast_fs.file_d = { 'file': f }
    except Exception as e:
        print("Warn: Opening file {} failed with {}".format(path, uerrno.errorcode[e.args[0]]))
        return lv.FS_RES.FS_ERR

    print("Info: File {} opened with mode {}".format(path, p_mode))
    return lv.FS_RES.OK


def esp32fs_read_cb(drv, fs_file, buf, btr, br):
    print("Info: called read({})".format(btr))
    cast_fs = lv.fs_file_t.cast(fs_file)

    try:
        buf = cast_fs.file_d.cast()['file'].read(btr)
    except Exception as e:
        print("Read() caused exception (probably OSError)", uerrno.errorcode[e.args[0]])
        return 3

    br = len(buf)
    print("Info: read() {}B: ".format(br), buf)

    return lv.FS_RES.OK

### Other functions are equivalent

Now, I am able to open file, read from them, but I have probably problem with returning content of buf, because it is probably zero-ed after read_cb ends.

I added debug prints (those begin with lv_img…) to trace code and I have output:


Info: image draw: cache miss, cached to an empty entry  (../../lib/lv_bindings/lvgl/src/lv_dra w/lv_img_cache.c #120)
lv_img_cache_open: Calling lv_img_decoder_open...
lv_img_decoder_open: d->info_cb = 1074818996, d->open_cb = 1074819456;
lv_img_decoder_built_in_info: entering
lv_img_decoder_built_in_info: is a file
Info: opening file 'vz2.bin', mode 2
Info: File vz2.bin opened with mode rb
Info: called read(4)
Info: read() 4B: b'\x04\x88\xa0\x05'
Info: called close()
Info: File descriptor closed
lv_img_decoder_built_in_info: header->cf = 0, header->w = 0, header->h = 0, CF_BUILT_IN_FIRST = 4, CF_BUILT_IN_LAST = 14
lv_img_decoder_open: d->info_cb(d, src, &dsc->header) res = 0, LV_RES_OK = 1, LV_RES_INV = 0
lv_img_decoder_open: result 0
lv_img_cache_open: Called lv_img_decoder_open with result 0, LV_RES_OK == 1, LV_RES_INV == 0
Warn: Image draw cannot open the image resource         (../../lib/lv_bindings/lvgl/src/lv_draw/lv_img_cache.c #133)
Warn: Image draw error  (../../lib/lv_bindings/lvgl/src/lv_draw/lv_draw_img.c #62)

Most important lines of output are those 2:

Info: read() 4B: b'\x04\x88\xa0\x05'
lv_img_decoder_built_in_info: header->cf = 0, header->w = 0, header->h = 0, CF_BUILT_IN_FIRST = 4, CF_BUILT_IN_LAST = 14

First one is from my read_cb, and readed data – first 4B, header of image, but after return from read_cb to lv_img_decoder_built_in_info is header zero (second line).

Call in lv_img_decoder_built_in_info, around line 290 (I don’t know exact line number because of my printf calls):

            res = lv_fs_read(&file, header, sizeof(lv_img_header_t), &rn);

I am sorry for my probably stupid question, but I am working on it too long, solution is probably very simple, but I don’t know…

amirgon · January 12, 2020, 11:37pm

Don’t be sorry, working with C pointers in micropython can be tricky.

The problem is in read_cb:

def esp32fs_read_cb(drv, fs_file, buf, btr, br):
    print("Info: called read({})".format(btr))
    cast_fs = lv.fs_file_t.cast(fs_file)

    try:
        buf = cast_fs.file_d.cast()['file'].read(btr)
    except Exception as e:
        print("Read() caused exception (probably OSError)", uerrno.errorcode[e.args[0]])
        return 3

    br = len(buf)
    print("Info: read() {}B: ".format(br), buf)

    return lv.FS_RES.OK

Let’s have a look at read_cb C prototype:

lv_fs_res_t (*read_cb)(struct _lv_fs_drv_t * drv, void * file_p, void * buf, uint32_t btr, uint32_t * br);

void * buf is an output parameter.
What you should do is copy the file data to the buffer pointed to by buf.
Changing buf pointer value itself will not achieve this, not in C and not in micropython.

Try something like this:

    try:        
        buf.__dereference__(btr)[0:btr] = cast_fs.file_d.cast()['file'].read(btr)

buf.__dereference__(btr) returns a memoryview to the memory pointed by buf (with size of btr bytes)
buf.__dereference__(btr)[0:btr] = ... copied btr bytes from the file read buffer to buf.

mhepp · January 13, 2020, 10:20am

You are right, it is really tricky…

And same problem is with br and equivalent in other functions:

    try:
        tmp_data = cast_fs.file_d.cast()['file'].read(btr)
        br.__dereference__(4)[0:4] = struct.pack("<L", len(tmp_data))
        buf.__dereference__(btr)[0:br] = tmp_data

And this is finally functional.

Whole file_drv.zip (1.2 KB) is attached with example usage. Feel free to use it where you want.

Thanks @amirgon, this was little bit over my possibilities.

amirgon · January 13, 2020, 10:43am

That’s great!
Would you like to contribute to lv_binding_micropython by opening a pull request on GitHub?

A good location for a file system driver could be: https://github.com/littlevgl/lv_binding_micropython/tree/master/lib

mhepp · January 13, 2020, 12:28pm

OK, I will add some comments and open pull request…