[Yocto BSP 6.0.0] Issue with toradex-sanity

Doing the following:

$ repo init -u git://git.toradex.com/toradex-manifest.git -b kirkstone-6.x.y -m tdxref/default.xml
$ repo sync
$ . export
$ bitbake -k tdx-reference-multimedia-image

throws the folloing error message:

ERROR: Execution of event handler 'defaulttoradex_check_sanity_eventhandler' failed
Traceback (most recent call last):
  File "/develop/projects/toradex/build/../layers/meta-toradex-bsp-common/classes/toradex-sanity.bbclass", line 57, in defaulttoradex_check_sanity_eventhandler(e=<bb.event.SanityCheck object at 0x7f21bc32bf10>):
         if bb.event.getName(e) == "SanityCheck":
    >        sanity_data = copy_data(e)
             if e.generateevents:
NameError: name 'copy_data' is not defined

ERROR: Command execution failed: Traceback (most recent call last):
  File "/develop/projects/toradex/layers/openembedded-core/bitbake/lib/bb/command.py", line 123, in runAsyncCommand
    self.cooker.updateCache()
  File "/develop/projects/toradex/layers/openembedded-core/bitbake/lib/bb/cooker.py", line 1648, in updateCache
    bb.event.fire(bb.event.SanityCheck(False), self.databuilder.mcdata[mc])
  File "/develop/projects/toradex/layers/openembedded-core/bitbake/lib/bb/event.py", line 222, in fire
    fire_class_handlers(event, d)
  File "/develop/projects/toradex/layers/openembedded-core/bitbake/lib/bb/event.py", line 123, in fire_class_handlers
    execute_handler(name, handler, event, d)
  File "/develop/projects/toradex/layers/openembedded-core/bitbake/lib/bb/event.py", line 93, in execute_handler
    ret = handler(event)
          ^^^^^^^^^^^^^^
  File "/develop/projects/toradex/build/../layers/meta-toradex-bsp-common/classes/toradex-sanity.bbclass", line 57, in defaulttoradex_check_sanity_eventhandler
    sanity_data = copy_data(e)
                  ^^^^^^^^^
NameError: name 'copy_data' is not defined

The function copy_data is defined in the default OE sanity checker meta/classes/sanity.bbclass as:

# Create a copy of the datastore and finalise it to ensure appends and 
# overrides are set - the datastore has yet to be finalised at ConfigParsed
def copy_data(e):
    sanity_data = bb.data.createCopy(e.data)
    sanity_data.finalize()
    return sanity_data

The Python namespace is local to the class hence the function would not be visible to toradex_sanity.bblcass. It would have to be defined inside that class.

What surprises me is that the exact same thing worked two weeks ago. Apparently a change was made and not properly validated.

Same problem here!

Hi @luciolis and @RudolfStreif,

Thanks for the report. Can you please share with me the default.xml file?

$ cat .repo/manifests/tdxref/default.xml

Also, can you share which module are you using? Is it the Verdin iMX8 Mini?

Best Regards,
Hiago.

<?xml version="1.0" encoding="UTF-8"?>
<manifest>
  <remote alias="repo" fetch="https://github.com/meta-qt5" name="githq"/>

  <include name="base/pinned.xml"/>
  <include name="bsp/pinned-nxp.xml"/>
  <include name="bsp/pinned-tdx.xml"/>

  <project name="meta-qt5.git" path="layers/meta-qt5" remote="githq" revision="44d44933200287f7d17cf6981af4b4a0961c308d" upstream="kirkstone"/>
  <project name="meta-toradex-demos.git" path="layers/meta-toradex-demos" remote="tdx" revision="842174e46db477167f44870160edadb46c7a3468" upstream="kirkstone-6.x.y"/>
  <project name="meta-toradex-distro.git" path="layers/meta-toradex-distro" remote="tdx" revision="2712e46aadd2ec74711d69503b61dba3548e0aa4" upstream="kirkstone-6.x.y">
    <copyfile dest="export" src="buildconf/export"/>
  </project>
</manifest>

It’s IMX8MP for me

Hi @luciolis and @RudolfStreif,

I was able to detect the problem here on my side. Let me show you how to solve it for now and then let me know if you guys have any questions about it.

The copy_data function was replaced in the openembedded layer:

commit 06e088ef6e961f05ca600612adcc71bff91f09be
Author: Richard Purdie <richard.purdie@linuxfoundation.org>
Date:   Fri Nov 11 22:58:27 2022 +0000

    sanity: Drop data finalize call
    
    This call was effectively like update_data and no longer did anything
    in bitbake. Drop it as it is obsolete.
    
    Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
    Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
    (cherry picked from commit d3eb4531aae28a07cb7e52ed5fe1102445d2effd)
    Signed-off-by: Steve Sakoman <steve@sakoman.com>

diff --git a/meta/classes/sanity.bbclass b/meta/classes/sanity.bbclass
index f2b2e4dfaf..293e405f62 100644
--- a/meta/classes/sanity.bbclass
+++ b/meta/classes/sanity.bbclass
@@ -999,13 +999,6 @@ def check_sanity(sanity_data):
     if status.messages != "":
         raise_sanity_error(sanity_data.expand(status.messages), sanity_data, status.network_error)
 
-# Create a copy of the datastore and finalise it to ensure appends and 
-# overrides are set - the datastore has yet to be finalised at ConfigParsed
-def copy_data(e):
-    sanity_data = bb.data.createCopy(e.data)
-    sanity_data.finalize()
-    return sanity_data
-
 addhandler config_reparse_eventhandler
 config_reparse_eventhandler[eventmask] = "bb.event.ConfigParsed"
 python config_reparse_eventhandler() {
@@ -1016,13 +1009,13 @@ addhandler check_sanity_eventhandler
 check_sanity_eventhandler[eventmask] = "bb.event.SanityCheck bb.event.NetworkTest"
 python check_sanity_eventhandler() {
     if bb.event.getName(e) == "SanityCheck":
-        sanity_data = copy_data(e)
+        sanity_data = bb.data.createCopy(e.data)
         check_sanity(sanity_data)
         if e.generateevents:
             sanity_data.setVar("SANITY_USE_EVENTS", "1")
         bb.event.fire(bb.event.SanityCheckPassed(), e.data)
     elif bb.event.getName(e) == "NetworkTest":
-        sanity_data = copy_data(e)
+        sanity_data = bb.data.createCopy(e.data)
         if e.generateevents:
             sanity_data.setVar("SANITY_USE_EVENTS", "1")
         bb.event.fire(bb.event.NetworkTestFailed() if check_connectivity(sanity_data) else bb.event.NetworkTestPassed(), e.data)

This was also fixed on the meta-toradex-bsp-common:

commit 24c7c028fdeedfc4b5e1b21ce4951a82dbfc2667
Author: Max Krummenacher <max.krummenacher@toradex.com>
Date:   Thu Nov 17 15:35:14 2022 +0100

    toradex-sanity.bbclass: follow upstream change
    
    openembedded-core commit d3eb4531aa ("sanity: Drop data finalize call")
    dropped a python function. Replace it with its aquivalent.
    
    Fixes:
    | ERROR: Execution of event handler 'defaulttoradex_check_sanity_eventhandler' failed
    | Traceback (most recent call last):
    |     File ".../meta-toradex-bsp-common/classes/toradex-sanity.bbclass", line 57, in defaulttoradex_check_sanity_eventhandler(e=<bb.event.SanityCheck object at 0x7f50de8bb3c8>):
    |         if bb.event.getName(e) == "SanityCheck":
    |     >       sanity_data = copy_data(e)
    |             if e.generateevents:
    | NameError: name 'copy_data' is not defined
    
    Signed-off-by: Max Krummenacher <max.krummenacher@toradex.com>
    (cherry picked from commit 706a6f65b6dfc01f18fca1161498467cafa47822)
    openembedded-core backported commit d3eb4531aa to kirkstone
    Signed-off-by: Max Krummenacher <max.krummenacher@toradex.com>

diff --git a/classes/toradex-sanity.bbclass b/classes/toradex-sanity.bbclass
index 19a38cd..bfb95f3 100644
--- a/classes/toradex-sanity.bbclass
+++ b/classes/toradex-sanity.bbclass
@@ -54,7 +54,7 @@ toradex_check_sanity_eventhandler[eventmask] = "bb.event.SanityCheck"
 
 python toradex_check_sanity_eventhandler() {
     if bb.event.getName(e) == "SanityCheck":
-        sanity_data = copy_data(e)
+        sanity_data = bb.data.createCopy(e.data)
         if e.generateevents:
             sanity_data.setVar("SANITY_USE_EVENTS", "1")
         reparse = toradex_check_sanity(sanity_data)

However, since 6.1 is still in development, the manifest wasn’t updated yet and it will be on the next quarterly release:

<?xml version="1.0" encoding="UTF-8"?>
<manifest>
  <remote alias="repo" fetch="https://git.toradex.com" name="tdx"/>
  <project name="meta-toradex-bsp-common.git" path="layers/meta-toradex-bsp-common" remote="tdx" revision="33b78f3508d5170a63e407a9216182026b5ebe08" upstream="kirkstone-6.x.y"/>
  <project name="meta-toradex-nxp.git" path="layers/meta-toradex-nxp" remote="tdx" revision="a23c4b6d01831d45634d4156b513746dda051161" upstream="kirkstone-6.x.y"/>
</manifest>

As you can see, the manifest gets one commit behind where the issue was solved. So there is a couple of options to fix this issue:

  • First, stick with the 6.0.0 quarterly release. This will roll back the openembedded layer where the copy_data is still used. For that, you can use:
$ repo init -u git://git.toradex.com/toradex-manifest.git -b refs/tags/6.0.0 -m tdxref/default.xml

This is the recommended option since 6.1 is still under development.

  • If you still want to use the 6.1 you can simply manually edit the revision inside manifests/bsp/pinned-tdx.xml and change the revision from 33b78f3508d5170a63e407a9216182026b5ebe08 to 24c7c028fdeedfc4b5e1b21ce4951a82dbfc2667, where the issue is resolved.

  • Or you can also go to the meta-toradex-bsp-common folder which was downloaded by repo and checkout to 24c7c028fdeedfc4b5e1b21ce4951a82dbfc2667.

I hope this helps, let me know if you have any questions.

Best Regards,
Hiago.

Thank you, @hfranco.tx. I fixed it the same way by patching the file. It was not a show stopper for me. I just wanted to bring it to your attention.

Hi @RudolfStreif,

I’m glad you could resolve it yourself. Thanks for the report! I created a ticket internally so I can follow the status as well.

Thanks again!

Best Regards,
Hiago.

Hi @hfranco.tx,

One more question related to the 6.x.y or 6.1 release: is Qt6 going to be supported with it?
I added it myself but had to modify the Qt package group of course (or more specifically added one for Qt6).

Thanks.

Hi @RudolfStreif,

Sorry for the delay. The only mention of QT6 that I found is inside meta-freescale:

PACKAGECONFIG[qt5] = "-DWITH_QT=ON -DOE_QMAKE_PATH_EXTERNAL_HOST_BINS=${STAGING_BINDIR_NATIVE} -DCMAKE_PREFIX_PATH=${STAGING_BINDIR_NATIVE}/cmake,-DWITH_QT=OFF,qtbase qtbase-native,"
PACKAGECONFIG[qt6] = "-DWITH_QT=ON -DQT_HOST_PATH=${RECIPE_SYSROOT_NATIVE}${prefix_native},-DWITH_QT=OFF,qtbase qtbase-native,"
PACKAGECONFIG[tests-imx] = "-DINSTALL_TESTS=ON -DOPENCV_TEST_DATA_PATH=${S}/../extra/testdata, -DINSTALL_TESTS=OFF,"
PACKAGECONFIG[tim-vx] = "-DWITH_TIMVX=ON -DTIMVX_INSTALL_DIR=${STAGING_DIR_HOST}${libdir},-DWITH_TIMVX=OFF,tim-vx"

# Disable cvv module in opencv_contrib as it is not yet suppported for Qt6
# (opencv debug framework)
EXTRA_OECMAKE:append = " -DBUILD_opencv_cvv=OFF"

Which modifications you had to do to add the QT6?

I will check the status of QT6 internally and let you know.

Best Regards,
Hiago.