Problems with interface after connecting to network

[Tiago_Almeida]

Device: M5stack
Chip: Esp32 , 520kb ram, no psram
Micropython

I am having problems when connecting to network.
At first I was having a DMA-able memory allocation error when I tried to connect to network and initialize the display which was solved in Not enough DMA-able memory to allocate display buffer by changing the factor in the ili9341.py driver from 4 to 8 or 16.

Now when I connect to network it messes with my program, mainly when I have some kind of list and I try to interact with it using buttons (check video : https://imgur.com/YJBO0JU ). It looks like network is conflicting with some internal loop (maybe event loop?).

Also, when I turn off the wifi my program works as expected… I can use the buttons and select an item from the list. When I reconnect it conflicts again.

I am using the test code bellow (you need to have the main.py and the m5_lvgl.py )

main.py

import machine
import gc
import network
import lvgl as lv
from m5_lvgl import ButtonsInputEncoder, EncoderInputDriver
from ili9341 import ili9341
import gc
import utime

import micropython
micropython.alloc_emergency_exception_buf(100)


AUTHENTICATED = False

OPTION1 = False
OPTION2 = False
OPTION3 = False
OPTION4 = False


def connect():
 
    ssid = "xxx"
    password =  "xxx"
 
    station = network.WLAN(network.STA_IF)
 
    if station.isconnected() == True:
        print("Already connected")
        return
 
    station.active(True)
    station.connect(ssid, password)
 
    while station.isconnected() == False:
        pass
 
    print("Connection successful")
    print(station.ifconfig())

disp = ili9341() # Create a display driver

connect()

def event_handler(obj, event):
    """
    Called when a button is released.
    Parameters
    ----------
    btn :
        The Button that triggered the event.
    event :
        The triggering event.
    """
    global OPTION1, OPTION2, OPTION3, OPTION4
    if event == lv.EVENT.RELEASED:
        print("Clicked: %s" % lv.list.get_btn_text(obj))
        if lv.list.get_btn_text(obj) == "Option1":
            
            OPTION1 = True
 
        elif lv.list.get_btn_text(obj) == "Option2":
            
            OPTION2 = True

        elif lv.list.get_btn_text(obj) == "Option3":

            OPTION3 = True
            
        elif lv.list.get_btn_text(obj) == "Option4":
            
            OPTION4 = True

screen = lv.obj()

button_encoder = ButtonsInputEncoder()
button_driver = EncoderInputDriver(button_encoder)

list1 = lv.list(screen)
AUTHENTICATED = True
if AUTHENTICATED:
    list1.set_size(300, 154)
    list1.align(None, lv.ALIGN.CENTER, 0, -5)   
    
            # Add buttons to the list

    list_btn = list1.add_btn(lv.SYMBOL.FILE, "Option1")
    list_btn.set_event_cb(event_handler)
    
    list_btn = list1.add_btn(lv.SYMBOL.DIRECTORY, "Option2")
    list_btn.set_event_cb(event_handler)
    
    list_btn = list1.add_btn(lv.SYMBOL.DIRECTORY, "Option3")
    list_btn.set_event_cb(event_handler)     
else:
    list1.set_size(300, 100)
    list1.align(None, lv.ALIGN.CENTER, 0, -5) 
    
            # Add buttons to the list
    list_btn = list1.add_btn(lv.SYMBOL.FILE, "Option2")
    list_btn.set_event_cb(event_handler)
    
    list_btn = list1.add_btn(lv.SYMBOL.DIRECTORY, "Option3")
    list_btn.set_event_cb(event_handler)
    
group = lv.group_create() # Create a group
lv.group_add_obj(group, list1)
button_driver.group =group
lv.group_set_style_mod_cb(group, None)
lv.group_set_style_mod_edit_cb(group,None)
lv.group_set_editing(group, True)
   
lv.scr_load(screen)

m5_lvgl.py

import gc
import struct

import lvgl as lv
import lvesp32
import machine
import utime

DEFAULT_ENCODER_ADDR = 0x5E  # (94)


__all__ = ['ButtonsInputEncoder', 'FacesEncoderInputEncoder',
           'EncoderInputDriver', 'general_event_handler', 'init_ili9341']


class ButtonsInputEncoder:
    def __init__(self, left=39, right=37, press=38):
        self._left = 0
        self._right = 0
        self._pressed = False

        def on_press_left(*args):
            self._left_time = utime.ticks_ms()
            self._left += 1

        def on_press_right(*args):
            self._right_time = utime.ticks_ms()
            self._right += 1

        def on_toggle_press(pin):
            self._press_time = utime.ticks_ms()
            self._pressed = not pin.value()

        btn_left = machine.Pin(left, machine.Pin.IN, machine.Pin.PULL_UP)
        btn_left.irq(trigger=machine.Pin.IRQ_RISING, handler=on_press_left)
        btn_right = machine.Pin(right, machine.Pin.IN, machine.Pin.PULL_UP)
        btn_right.irq(trigger=machine.Pin.IRQ_RISING, handler=on_press_right)
        btn_press = machine.Pin(press, machine.Pin.IN, machine.Pin.PULL_UP)
        btn_press.irq(trigger=machine.Pin.IRQ_FALLING | machine.Pin.IRQ_RISING,
                      handler=on_toggle_press)

    @property
    def diff_peek(self):
        return self._right - self._left

    @property
    def diff(self):
        diff = self._right - self._left
        self._left = 0
        self._right = 0
        return diff

    @property
    def pressed(self):
        return self._pressed


class EncoderInputDriver:
    def __init__(self, encoder, group=None):
        def input_callback(drv, data):
            data.enc_diff = encoder.diff
            if encoder.pressed:
                data.state = lv.INDEV_STATE.PR
            else:
                data.state = lv.INDEV_STATE.REL
            gc.collect()
            return False

        self.drv = lv.indev_drv_t()
        self.encoder = encoder
        lv.indev_drv_init(self.drv)
        self.drv.type = lv.INDEV_TYPE.ENCODER
        self.drv.read_cb = input_callback
        self.win_drv = lv.indev_drv_register(self.drv)
        self.group = group

    @property
    def group(self):
        return self._group

    @group.setter
    def group(self, value):
        self._group = value
        if self._group is not None:
            lv.indev_set_group(self.win_drv, self._group)

[Tiago_Almeida]

As suggested by @amirgon :

I’m trying to understand if this is related to rendering / screen refresh or to input device handling.
Please try to add some animation which is not triggered by an input device.
Then try to see if the animation behaves differently when WiFi is on/off.
If animation works fine with WiFi, that would narrow the problem to your input device (buttons/encoder)

I have done the following:

main.py

import gc
import lvgl as lv
from ili9341 import ili9341
import micropython
micropython.alloc_emergency_exception_buf(100)

import network

disp = ili9341() # Create a display driver

def connect():
    ssid = "xxx"
    password =  "xxx"

    station = network.WLAN(network.STA_IF)

    if station.isconnected() == True:
        print("Already connected")
        return
        

    station.active(True)
    station.connect(ssid, password)

    while station.isconnected() == False:
        pass

    print("Connection successful")
    print(station.ifconfig())


connect()

screen = lv.obj()

# Create a style for the Preloader
style = lv.style_t()
lv.style_copy(style, lv.style_plain)
style.line.width = 10                          # 10 px thick arc
style.line.color = lv.color_hex3(0x258)        # Blueish arc color

style.body.border.color = lv.color_hex3(0xBBB) # Gray background color
style.body.border.width = 10
style.body.padding.left = 0

# Create a Preloader object
preload = lv.preload(screen)
preload.set_size(100, 100)
preload.align(None, lv.ALIGN.CENTER, 0, 0)
preload.set_style(lv.preload.STYLE.MAIN, style)

lv.scr_load(screen) 

At first look it is completely fine. I manage to connect to my wifi and then the preload ring refreshes as expected.

So I guess the problem is related with the input device handling…

Also I know I am connected to Wifi because I am using my phone as an hotspot and it says that the esp is connected. If I turn off my hotspot the ring keeps working. If I connect my hotspot again the esp connects to my network again (I can see that it is connected on my phone)

[amirgon]

Let’s see if the problem is related to LittlevGL at all.

Try removing all lvgl code from your script, but keep the machine.Pin calls and your event handlers that are triggered by IRQ_RISING and IRQ_FALLING.
In the event handlers (on_press_left, on_press_right, on_toggle_press) you can try printing something, or increment a counter.

Then check if the callback behavior change when turning WiFi on/off.

[Tiago_Almeida]

So I simply made print(“left”), print(“right”) etc in each of the handlers. When I ran the program I got left left left left left … If I turn the wifi off it stops…

If I try with my normal program and I disable the left button it now works normally! I can use the other 2 buttons…

Why on earth is my Wifi conflicting with my left button…?! (Going back to the memory problem, can it be related? Although I think not…). My guess is that it is a defect with my board…

@fstengel since you are working with the same setup that I am, would you be able to test my code on yours (the one from the beggining of this post) and tell me if it works fine or if you have the same problem? I would be very very grateful!

@amirgon Any other idea?
Thank you so much for your help, it has proven very valuable!

[amirgon]

@Tiago_Almeida

Why on earth is my Wifi conflicting with my left button…?! (Going back to the memory problem, can it be related? Although I think not…). My guess is that it is a defect with my board…

Now it’s worth checking whether WiFi affects the pin state or the IRQ.
Try reading the pin value in a loop, instead of in an IRQ handler. Do you still get a weird behavior when WiFi is turned on?

If not - it’s IRQ. Maybe it’s shared with WiFi. Need to dig in the ESP32 hardware refernce.
If you do - maybe something is connected to the same pin in your hardware. Maybe you should use a pull-down instead of pull-up. Maybe the antenna in your hardware is too close to than pin…
If you suspect it’s some analog side product you can try to lower transmission power and see if it makes a difference.
You can also try another M5stack board. Maybe the piece you are using has some defect.

It may be worth searching M5stack and ESP32 forums / GitHub issues for something related. If you don’t find anything, write a minimal C program that reproduces it - and open a bug on ESP32 or M5stack.

[fstengel]

Just my 0.1 cent here: I have discovered that the M5Stack has somehow sticky buttons. By thin I mean that one can push a button and that this button does not release properly. On mine I need to deliberately push/release the button to see anything happening.

So if your left button gets stuck in the pressed position, then you will get an endless stream of “left”…

@Tiago_Almeida I’ll try your code this afternoon

[fstengel]

I just had a quick try of your code. On my M5Stack, which has PSRAM, your example has problems with ooor without wifi. I tried another input driver I wrote on the side (see https://github.com/fstengel/m5LvglFiles) and I discovered:

1. Using keys that do not navigate the list (KEY.PREV, KEY.NEXT), my console shows the keys pressed, when I press them.
2. If I use the proper navigation keys, then, well, things get hairier. There is strong asynchronicity: I sometimes get feedback on the console much later or not at all. It is as if lvgl is preventing the irqs from happening. This is the case when the pressed physical button is not properly released and stays in the pressed state.

By the way, your display initializer does not work with me. I need to use the “long” version with all the arguments (miso, mosi etc.)

[Tiago_Almeida]

Yup. Bingo. If I create a simple loop like

while(True):
    utime.sleep_ms(30)
    if btn_left.value()==0:
        print("left")

I no longer get infinite presses… So the problem might be related with the IRQ. That sucks… I’ll have to refactor my code. Maybe some lvgl tasks for detecting button presses?

So I searched a little more about what is happening and found that it is a common error, for example:

Calling iot_button_create() causes RTCWDT_RTC_RESET (IDFGH-3167)

M5.BtnA.wasPressed() conflicts with WiFi?

People are getting the same issue with the exact same button… Can’t really understand if it is a software or hardware problem… Some say that with an older esp-idf version the problem doesn’t happen… or changing from digital read to analog helps to sort out the problem.

Meanwhile I also contacted m5stack support via email, directed them to this post and they told me this:

"…btn read can`t use irq event, must use poll to read btn state. In 10ms, btna keeps in low value, can be recognized as ‘Pressing Left Button’ "

So you really are into something @amirgon thank you!

@fstengel It is true that m5stack have sticky buttons sometime but that is not the case here, because if I turn off the Wifi everything works well, when it is on I don’t even have to press once to get it like it was pressed.

Thank you so much for trying my code. I will give a try to your code in a moment! For me my code works well without wifi or if I disable A (left) button. I have hardcoded the arguments inside my driver so I don’t need to provide them everytime, that is why it doesnt work for you I guess.

Regarding #2, yup that looks like what is happening to me. But with what we have seen so far, and in conjunction with the links I provide above I don’t think it is a lvgl problem. But a hardware and/or software problem with esp-idf or m5stack hardware. Are you using IRQ or polling? I’ll have a look anyway. Thank you!

[Tiago_Almeida]

@fstengel can you do me another favor? I am curious to know if this happen in your m5stack version. If you have a working code just add this bit to the beggining and let us know if your program starts to behave weirdly after connecting to WiFi, mainly your A (left button). Also if you connect it to your phone hotspot it is easy to turn on and off your wifi and check for differences.

def connect():
 
    ssid = "xxx"
    password =  "xxx"
 
    station = network.WLAN(network.STA_IF)
 
    if station.isconnected() == True:
        print("Already connected")
        return
 
    station.active(True)
    station.connect(ssid, password)
 
    while station.isconnected() == False:
        pass
 
    print("Connection successful")
    print(station.ifconfig())

connect()

[fstengel]

Wow. It’s a lot to digest. I especially like M5Stack’s answer. Basically all the bits and bobs of code I have read about getting something from a button are done using irqs and are therefore… wrong. I had started with Mika Tuupola’s micropython classes for the M5Stack (he uses an irq) and also fell into the WDT trap. I thought it was due to irqs taking too mutch time. My M5Stack is a grey one with MPU9250 and 4MB PSRAM.

I’ll refactor my code tomorrow to polling. I had tried connecting to wifi first using your code and then testing. I saw no real difference apart from the sticky butons. Still, lets poll…

I’ll tell you more tomorrow.

[Tiago_Almeida]

Great! Happy that this is being useful at a greater level than just my project. I know that right now, since the problem is not related only with lvgl, this might seem a bit off topic. But I believe that it will be usefull to anyone using m5stack + lvgl.

Right now I just made some quick fixes. Changed the irq’s for a lvgl task with 130ms of interval and for now it works. I’ll have to change for a more robust option later. Any downsides of using tasks for this effect? Would they consume more resources than necessary? I think it is better than simply poling and sleeping mid code anyway…

[amirgon]

@Tiago_Almeida

I no longer get infinite presses… So the problem might be related with the IRQ. That sucks… I’ll have to refactor my code. Maybe some lvgl tasks for detecting button presses?

It’s great to see you are making progress!

You can try reading buttons in polling mode, and you don’t even need to use lvgl tasks.
Your input_callback which you register to your indev lvgl driver is called periodically by lvgl. You can simply read the pins there.

But be careful - you can lose clicks if they are fast enough. I’ve seen that before.
To overcome this, ESP32 has PCNT hardware (Pulse Counter) which you can also use in Micropython, so the hardware, not software, would count how many times the button was clicked.
For more details please see:

Example for working with a rotary encoder?

You are right. I don’t see a reason why this wouldn’t be implemented for ESP32, maybe worth opening an issue on micropython upstream repo. This really sounds useful, and it will probably be added soon to micropython. See this forum thread. Another option is using the espidf module, which provides raw esp-idf micropython API. I’m using it today to access SPI and I2S directly from micropython. (see for example the micropython ili9341 driver). I’ve added pcnt support to espidf module (it re…

[Tiago_Almeida]

@amirgon

Your input_callback which you register to your indev lvgl driver is called periodically by lvgl. You can simply read the pins there.

I’ll certainly try this. It would save me the memory and cpu time, and battery, of creating and maintaining more tasks.

@amirgon

To overcome this, ESP32 has PCNT hardware (Pulse Counter) which you can also use in Micropython, so the hardware, not software, would count how many times the button was clicked.
For more details please see

Didn’t know about this, I will also give it a look! Thank you so much. Learning a lot with you!

[fstengel]

@Tiago_Almeida I have started toying with the pulse counters to read clicks from a button. I’ll keep you posted on my progress.

My current idea for push buttons is:

• set up a unit whose counter increases every time an edge occurs (rising and falling)
• if the counter is even the button is released, if the counter is odd, the button is pushed
• (1+counter)//2 can give me the number of times there was a push. That way I can count misses

Basically, rather than polling the pin value, I now poll the counter, resetting it from time to time.

There is a bouncing problem. Even with “filtering” I can have bounces…

@amirgon The espidf module is an amazing treasure trove. What I have to find is a way to use all the callbacks that the idf uses.

[fstengel]

I can confirm there is something weird going on . There is definitely something of an interaction beween button A of the M5Stack and Wifi. If I activate wifi and connect to a hotspot, then the pulse counting hardware counts a short pulse about once a second on button A (gpio 39)
I cannot say how short the pulse is. The only thing I can say is that it is at least 1023 clock cycles long that is more than 13us long: it is not filtered out by the pulse counter.

One thing though: if I press button A then there is no pulse going through…

[edit] As far as I can tell the pulse is shorter than the time it takes, in microPython, to read the counter, compare it to the previous value and loop…

[amirgon]

@fstengel

@amirgon The espidf module is an amazing treasure trove. What I have to find is a way to use all the callbacks that the idf uses.

Using callbacks it tricky, because the callback must also receive some “user_data” that contains the Python callable object.
In LittlevGL we solved this with “callback conventions”, but ESP IDF doesn’t follow them, of course.
When there is some callback you want to register in ESP IDF, there can be two approaches:

• Wrap the callback registration and the corresponding callback function with C wrapper functions that obey the callback conventions supported by the binding script.
• Or - Improve the binding script to support some other callback conventions.

If there is some specific callback you need, please let me know and we can discuss what is the best approach.

[fstengel]

Thanks @amirgon. I had figured out about the same as your explanation . At this time, I do not really need to use a callback/isr that is used by the espidf and is not already transcoded somewhrere else in uPy/lvgl. So the question was more academic than anything else.

@Tiago_Almeida Things are progressing well. With the pulse count approach, I was able to adapt your examples hand have them work. I have not tried to do more with my wifi than connecting to it.

Anyway. For all, a word of advice:

Do not use ISR/IRQs to read buttons with an ESP32!

They are flaky, and there is some interference on the M5Stack implementation with the wifi module (hardware or esp-idf related). Use the pulse counter instead

I’ll polish my work, and I’ll publish it on my github during the week-end.

[fstengel]

See M5Stack and its buttons for an attempt to use the PCNT hardware to read the buttons reliably. The files are in my github: https://github.com/fstengel/m5LvglFiles

[Tiago_Almeida]

@fstengel glad that you are making progress! Thank you for sharing your code, I will test it soon!

Atleast now if someone gets the same behavior they will know why!