Recommend some simple basic test framework for core and conversion functions

Feature request
1

The core lvgl code seems to have no unit tests.
Does the team have any opinion about adding tests and the preferred way to add and run them?
Would they be open to a PR to add this?
I am just talking about basic unit function tests.
Nothing about UI or automation testing.

Thanks,
Pv

2

Hi,

I’ve just added an “ASSERT system” to the object’s API functions. You can configure LittlevGL to check a few things about the objects in run time.
Config options
Example usage

Regarding the unit test, what do you think, which units should be tested?

3

At some point, we would like to add CI to prevent build breakage. I don’t think there is any objection to unit tests in the meantime. Feel free to open a PR so we can discuss.

4

Just brainstorming…

  1. Color conversions
  2. Object position after alignment (on known [emulated] screen size)
  3. File system (generic functionality; not suggesting it initially test anything platform specific)
  4. Area mutation
  5. Asyncs fire
  6. Linked lists would be an almost perfect first candidate
  7. Math functionality
  8. Memory seems like a good candidate, but could open up black-hole of low RoI problematic edge cases or false results
  9. printf
  10. Tasks fire
  11. Text mutation/manipulation/calculation
  12. Utils
  13. Fonts might be difficult
  14. Style copying
  15. Theme verification
  • Objects may be hard.
  • Drawing will be difficult only in that it will require verifying results from screen capture, and that isn’t trivial to verify

Again, my initial proposal is just for basic core functionality tests where judiciously determined to be worthwhile to minimize regressions.

5

At some point, we would like to add CI to prevent build breakage. I don’t think there is any objection to unit tests in the meantime. Feel free to open a PR so we can discuss.

Being new to MCUs, my main concern is not knowing yet if there is already a mechanism to easily run tests in an emulator/simulator/container and get pass/fail results.
Thus the request for a discussion here before anything is coded for a PR.

Perhaps first step would be to document [or find where it is already documented] whatever testing is currently feasible.

Pv

6

Provided that there is a logical definition for pass/fail, the most primitive way do it is probably to modify the PC simulator to print out OK if a test passes, and have a script that runs various builds of the PC simulator and checks that OK is printed for each one.

Of course, I have never worked with any unit testing framework before, so I’m sure there’s a much better way of doing it.

Another project on GitHub that I’ve used in some web projects before does this: https://github.com/0xfe/vexflow/wiki/Visual-Regression-Tests

7

Another project on GitHub that I’ve used in some web projects before does this: https://github.com/0xfe/vexflow/wiki/Visual-Regression-Tests

Ergo: non-trivial! :slight_smile:

I was an SDET for 10+ years (at Microsoft, Google, and Motorola), so I have a bit of desktop experience, especially w/ xUnit types of test frameworks, but how this can be applied to MCUs is all new to me. I default to ASSuMEing that folks magnitudes smarter than I solved this years ago and have open sourced it! :slight_smile:

8

I actually think it’s not too difficult to do ourselves. :slightly_smiling_face:

Remember that we don’t need to run the tests on an MCU - they can be run in the PC simulator. That makes it a lot easier to do advanced things.

From what I can tell, the test can be broken down into two steps:

  1. Getting a screenshot out of the PC simulator in bitmap form.
  2. Running a perceptual hash on that screenshot and a stored “known good” screenshot and comparing the results.
9

I use PIO to run tests in CI. It uses built-in Unity testing lib. In latest pio beta they added some simple static analyzers call, but no yet coverage reporter call (i guess, can be added manually via platformio.ini). https://github.com/puzrin/dispenser/blob/dev/.travis.yml

As been noted above, usually you don’t need run most of tests on bare metal. Almost everything can be tested on PC with appropriate mocks.

10

I like the idea to test these components. They seem “well-testable”.

Ultimately it’d be great to automatically run tests with Travis (or similar) when new PR arrives.
Do you have experience with that?

11

Just getting back around to this open tab.

Yes, I have used Travis and Jenkins a ton, mostly in the area of Android and iOS build and test, and not with micro/embedded code.
My main issue right now is getting a usable simulator working on my main work machine which is a Mac.
I have Windows PCs at home that I can work on, but most of my development is on Mac, so getting that to work would be preferred.

Obviously getting a Travis OS image booted and used is irrelevant to my getting an environment running on my Mac, but psychologically/motivationally I would like to get the latter working first before I start volunteering to get Travis CI tests up and running.

12

Have you tried the PC simulator yet? It should work on a Mac. Be sure to clone the latest version from GitHub.

13

Which one?
I am a bit discouraged to do this on a Mac due to:

  1. https://github.com/littlevgl/pc_simulator_sdl_visual_studio
    “This is a pre-configured Visual Studio project to try LittlevGL on a Windows PC. The project uses the SDL library which is copied and linked to the project, so you can compile it without any extra dependencies. The 64 bit libraries are used so it will work out-of-the-box on 64-bit systems.”
  2. https://github.com/littlevgl/lv_platformio/issues/11#issuecomment-538635767
    “This demo was tested only with linux.”
    “Closing as wontfix . If anyone has knowledge and time to make native demo work in Windows/macOS - PR welcome”

Pv

14

@paulpv
pc_simulator_sdl_eclipse should work well on Mac too.
See this guide about installing SDL on OSX.