No devtool in the SDK produced?

Fide · January 24, 2022, 4:51pm

Hi,

I followed the instructions here and compiled nightly build for Verdin iMX8M-Plus and get tx-reference-minimal-image.

Then I created an SDK with below command and all completed without any errors.
clear; DISTRO="tdx-xwayland-rt" bitbake tdx-reference-minimal-image -c populate_sdk

Then I installed the sdk ./tdx-xwayland-rt-glibc-x86_64-Reference-Minimal-Image-rt-aarch64-verdin-imx8mp-toolchain-5.6.0.sh produced. (no problem)

Finally I entered to the installation folder and typed
. environment-setup-aarch64-tdx-linux which works ok.

But then when I try to run devtool, I get following error
bash: devtool: command not found

Any idea why?

Thank you.

DaveM · January 24, 2022, 5:30pm

I think devtool is in the extensible sdk…not sure if there are toradex-specific instructions for it, but try:
bitbake tdx-reference-minimal-image -c populate_sdk_ext

Fide · January 25, 2022, 12:47am

Thank you but populate_sdk_ext bitbake ends up with following error:

Loading cache: 100% |##############################################################################################################################| Time: 0:00:00
Loaded 3902 entries from dependency cache.
Parsing recipes: 100% |############################################################################################################################| Time: 0:00:01
Parsing of 2748 .bb files complete (2739 cached, 9 parsed). 3911 targets, 238 skipped, 0 masked, 0 errors.
NOTE: Resolving any missing task queue dependencies

Build Configuration:
BB_VERSION           = "1.46.0"
BUILD_SYS            = "x86_64-linux"
NATIVELSBSTRING      = "universal"
TARGET_SYS           = "aarch64-tdx-linux"
MACHINE              = "verdin-imx8mp"
DISTRO               = "tdx-xwayland-rt"
DISTRO_VERSION       = "5.6.0-devel-20220124180941+build.0"
TUNE_FEATURES        = "aarch64"
TARGET_FPU           = ""
meta-toradex-nxp     = "HEAD:e7751bf81c3634161c14d16a0857d02ca0be02cb"
meta-freescale       = "HEAD:6d2922bee372157454e422df518b6cf0bfcf6700"
meta-freescale-3rdparty = "HEAD:c52f64973cd4043a5e8be1c7e29bb9690eb4c3e5"
meta-toradex-tegra   = "HEAD:a03fde04d777cb3f9549d6fbf234924211c867c2"
meta-toradex-bsp-common = "HEAD:ac81ce5eb533a78af2a84acecc389315dfb54385"
meta-oe              
meta-filesystems     
meta-gnome           
meta-xfce            
meta-initramfs       
meta-networking      
meta-multimedia      
meta-python          = "HEAD:7889158dcd187546fc5e99fd81d0779cad3e8d17"
meta-freescale-distro = "HEAD:5d882cdf079b3bde0bd9869ce3ca3db411acbf3b"
meta-toradex-demos   = "HEAD:eeaa1fa0f22496a84d42c2b54e4ae23b567b07f2"
meta-qt5             = "HEAD:b4d24d70aca75791902df5cd59a4f4a54aa4a125"
meta-toradex-distro  = "HEAD:432c37ce55d3d5c03bd67d17348d1e667070f661"
meta-poky            = "HEAD:8a9d89df2c50128130a40119a4827ace761c236c"
meta                 = "HEAD:44b1970c40e9d73f6e63fb10cdc55837a26f5921"

Initialising tasks: 100% |#########################################################################################################################| Time: 0:00:04
Sstate summary: Wanted 452 Found 256 Missed 196 Current 2080 (56% match, 92% complete)
NOTE: Executing Tasks
ERROR: tdx-reference-minimal-image-1.0-r0 do_populate_sdk_ext: Error executing a python function in exec_func_python() autogenerated:

The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_func_python() autogenerated', lineno: 2, function: <module>
     0001:
 *** 0002:copy_buildsystem(d)
     0003:
File: '/home/mf/yocto-projects/toradexbsp/build/../layers/openembedded-core/meta/classes/populate_sdk_ext.bbclass', lineno: 209, function: copy_buildsystem
     0205:        workspace_name = 'orig-workspace'
     0206:    else:
     0207:        workspace_name = None
     0208:
 *** 0209:    corebase, sdkbblayers = buildsystem.copy_bitbake_and_layers(baseoutpath + '/layers', workspace_name)
     0210:    conf_bbpath = os.path.join('layers', corebase, 'bitbake')
     0211:
     0212:    for path in os.listdir(baseoutpath + '/layers'):
     0213:        relpath = os.path.join('layers', path, oe_init_env_script)
File: '/home/mf/yocto-projects/toradexbsp/build/../layers/openembedded-core/meta/lib/oe/copy_buildsystem.py', lineno: 130, function: copy_bitbake_and_layers
     0126:
     0127:                if os.path.exists(os.path.join(layerdestpath, 'conf/layer.conf')):
     0128:                    bb.note("Skipping layer %s, already handled" % layer)
     0129:                else:
 *** 0130:                    _smart_copy(layer, layerdestpath)
     0131:
     0132:            if workspace:
     0133:                # Make some adjustments original workspace layer
     0134:                # Drop sources (recipe tasks will be locked, so we don't need them)
File: '/home/mf/yocto-projects/toradexbsp/build/../layers/openembedded-core/meta/lib/oe/copy_buildsystem.py', lineno: 25, function: _smart_copy
     0021:    if stat.S_ISDIR(mode):
     0022:        bb.utils.mkdirhier(dest)
     0023:        cmd = "tar --exclude='.git' --exclude='__pycache__' --xattrs --xattrs-include='*' -chf - -C %s -p . \
     0024:        | tar --xattrs --xattrs-include='*' -xf - -C %s" % (src, dest)
 *** 0025:        subprocess.check_output(cmd, shell=True, stderr=subprocess.STDOUT)
     0026:    else:
     0027:        shutil.copyfile(src, dest)
     0028:        shutil.copymode(src, dest)
     0029:
File: '/usr/lib/python3.8/subprocess.py', lineno: 415, function: check_output
     0411:        else:
     0412:            empty = b''
     0413:        kwargs['input'] = empty
     0414:
 *** 0415:    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
     0416:               **kwargs).stdout
     0417:
     0418:
     0419:class CompletedProcess(object):
File: '/usr/lib/python3.8/subprocess.py', lineno: 516, function: run
     0512:            # We don't call process.wait() as .__exit__ does that for us.
     0513:            raise
     0514:        retcode = process.poll()
     0515:        if check and retcode:
 *** 0516:            raise CalledProcessError(retcode, process.args,
     0517:                                     output=stdout, stderr=stderr)
     0518:    return CompletedProcess(process.args, retcode, stdout, stderr)
     0519:
     0520:
Exception: subprocess.CalledProcessError: Command 'tar --exclude='.git' --exclude='__pycache__' --xattrs --xattrs-include='*' -chf - -C /home/mf/yocto-projects/toradexbsp/layers/meta-freescale -p .         | tar --xattrs --xattrs-include='*' -xf - -C /home/mf/yocto-projects/toradexbsp/build/tmp/work/verdin_imx8mp-tdx-linux/tdx-reference-minimal-image/1.0-r0/sdk-ext/image//usr/local/oe-sdk-hardcoded-buildpath/layers/openembedded-core/../meta-freescale' returned non-zero exit status 2.

Subprocess output:
abort()ing pseudo client by server request. See https://wiki.yoctoproject.org/wiki/Pseudo_Abort for more details on this.
Check logfile: /home/mf/yocto-projects/toradexbsp/build/tmp/work/verdin_imx8mp-tdx-linux/tdx-reference-minimal-image/1.0-r0/pseudo//pseudo.log
Aborted (core dumped)
tar: Unexpected EOF in archive
tar: Unexpected EOF in archive
tar: Error is not recoverable: exiting now

ERROR: Logfile of failure stored in: /home/mf/yocto-projects/toradexbsp/build/tmp/work/verdin_imx8mp-tdx-linux/tdx-reference-minimal-image/1.0-r0/temp/log.do_populate_sdk_ext.715183
ERROR: Task (/home/mf/yocto-projects/toradexbsp/build/../layers/meta-toradex-demos/recipes-images/images/tdx-reference-minimal-image.bb:do_populate_sdk_ext) failed with exit code '1'
NOTE: Tasks Summary: Attempted 5805 tasks of which 5245 didn't need to be rerun and 1 failed.
NOTE: Writing buildhistory
NOTE: Writing buildhistory took: 3 seconds

Summary: 1 task failed:
  /home/mf/yocto-projects/toradexbsp/build/../layers/meta-toradex-demos/recipes-images/images/tdx-reference-minimal-image.bb:do_populate_sdk_ext
Summary: There was 1 ERROR message shown, returning a non-zero exit code.

DaveM · January 25, 2022, 1:18am

Bummer…maybe a different image supports it? Doubt it though, as that is “minimal”. We’ll see what the toradex folks say?

Fide · January 25, 2022, 10:58am

I have also tried it with BSP 5.5.0 and tdx-xwayland distro instead of rt. Same error.

Parsing recipes: 100% |############################################################################################################################################################################################################################################| Time: 0:01:03
Parsing of 2748 .bb files complete (0 cached, 2748 parsed). 3911 targets, 238 skipped, 0 masked, 0 errors.
Removing 1 recipes from the verdin_imx8mp sysroot: 100% |##########################################################################################################################################################################################################| Time: 0:00:01
NOTE: Resolving any missing task queue dependencies

Build Configuration:
BB_VERSION           = "1.46.0"
BUILD_SYS            = "x86_64-linux"
NATIVELSBSTRING      = "universal"
TARGET_SYS           = "aarch64-tdx-linux"
MACHINE              = "verdin-imx8mp"
DISTRO               = "tdx-xwayland"
DISTRO_VERSION       = "5.5.0-devel-20220125083911+build.0"
TUNE_FEATURES        = "aarch64"
TARGET_FPU           = ""
meta-toradex-nxp     = "HEAD:4d9bf383a656759a574b3d82cb8b90b0e2663e50"
meta-freescale       = "HEAD:6d2922bee372157454e422df518b6cf0bfcf6700"
meta-freescale-3rdparty = "HEAD:c52f64973cd4043a5e8be1c7e29bb9690eb4c3e5"
meta-toradex-tegra   = "HEAD:84b975b3e5fd834f9bec5ea398edf0c0ab911cd1"
meta-toradex-bsp-common = "HEAD:4b876a0ecbe04eb924abf2442249c2cae3231d21"
meta-oe              
meta-filesystems     
meta-gnome           
meta-xfce            
meta-initramfs       
meta-networking      
meta-multimedia      
meta-python          = "HEAD:7889158dcd187546fc5e99fd81d0779cad3e8d17"
meta-freescale-distro = "HEAD:5d882cdf079b3bde0bd9869ce3ca3db411acbf3b"
meta-toradex-demos   = "HEAD:eeaa1fa0f22496a84d42c2b54e4ae23b567b07f2"
meta-qt5             = "HEAD:b4d24d70aca75791902df5cd59a4f4a54aa4a125"
meta-toradex-distro  = "HEAD:e835edf3bc368fc6beb64acefa96a349ec6da281"
meta-poky            = "HEAD:8a9d89df2c50128130a40119a4827ace761c236c"
meta                 = "HEAD:44b1970c40e9d73f6e63fb10cdc55837a26f5921"

Initialising tasks: 100% |#########################################################################################################################################################################################################################################| Time: 0:00:05
Sstate summary: Wanted 364 Found 236 Missed 128 Current 2168 (64% match, 94% complete)
NOTE: Executing Tasks
ERROR: tdx-reference-minimal-image-1.0-r0 do_populate_sdk_ext: Error executing a python function in exec_func_python() autogenerated:

The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_func_python() autogenerated', lineno: 2, function: <module>
     0001:
 *** 0002:copy_buildsystem(d)
     0003:
File: '/home/mf/yocto-projects/toradexbsp/build/../layers/openembedded-core/meta/classes/populate_sdk_ext.bbclass', lineno: 209, function: copy_buildsystem
     0205:        workspace_name = 'orig-workspace'
     0206:    else:
     0207:        workspace_name = None
     0208:
 *** 0209:    corebase, sdkbblayers = buildsystem.copy_bitbake_and_layers(baseoutpath + '/layers', workspace_name)
     0210:    conf_bbpath = os.path.join('layers', corebase, 'bitbake')
     0211:
     0212:    for path in os.listdir(baseoutpath + '/layers'):
     0213:        relpath = os.path.join('layers', path, oe_init_env_script)
File: '/home/mf/yocto-projects/toradexbsp/build/../layers/openembedded-core/meta/lib/oe/copy_buildsystem.py', lineno: 133, function: copy_bitbake_and_layers
     0129:
     0130:                if os.path.exists(os.path.join(layerdestpath, 'conf/layer.conf')):
     0131:                    bb.note("Skipping layer %s, already handled" % layer)
     0132:                else:
 *** 0133:                    _smart_copy(layer, layerdestpath)
     0134:
     0135:            if workspace:
     0136:                # Make some adjustments original workspace layer
     0137:                # Drop sources (recipe tasks will be locked, so we don't need them)
File: '/home/mf/yocto-projects/toradexbsp/build/../layers/openembedded-core/meta/lib/oe/copy_buildsystem.py', lineno: 28, function: _smart_copy
     0024:    if stat.S_ISDIR(mode):
     0025:        bb.utils.mkdirhier(dest)
     0026:        cmd = "tar --exclude='.git' --exclude='__pycache__' --xattrs --xattrs-include='*' -chf - -C %s -p . \
     0027:        | tar --xattrs --xattrs-include='*' -xf - -C %s" % (src, dest)
 *** 0028:        subprocess.check_output(cmd, shell=True, stderr=subprocess.STDOUT)
     0029:    else:
     0030:        shutil.copyfile(src, dest)
     0031:        shutil.copymode(src, dest)
     0032:
File: '/usr/lib/python3.8/subprocess.py', lineno: 415, function: check_output
     0411:        else:
     0412:            empty = b''
     0413:        kwargs['input'] = empty
     0414:
 *** 0415:    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
     0416:               **kwargs).stdout
     0417:
     0418:
     0419:class CompletedProcess(object):
File: '/usr/lib/python3.8/subprocess.py', lineno: 516, function: run
     0512:            # We don't call process.wait() as .__exit__ does that for us.
     0513:            raise
     0514:        retcode = process.poll()
     0515:        if check and retcode:
 *** 0516:            raise CalledProcessError(retcode, process.args,
     0517:                                     output=stdout, stderr=stderr)
     0518:    return CompletedProcess(process.args, retcode, stdout, stderr)
     0519:
     0520:
Exception: subprocess.CalledProcessError: Command 'tar --exclude='.git' --exclude='__pycache__' --xattrs --xattrs-include='*' -chf - -C /home/mf/yocto-projects/toradexbsp/layers/meta-freescale -p .         | tar --xattrs --xattrs-include='*' -xf - -C /home/mf/yocto-projects/toradexbsp/build/tmp/work/verdin_imx8mp-tdx-linux/tdx-reference-minimal-image/1.0-r0/sdk-ext/image//usr/local/oe-sdk-hardcoded-buildpath/layers/openembedded-core/../meta-freescale' returned non-zero exit status 2.

Subprocess output:
abort()ing pseudo client by server request. See https://wiki.yoctoproject.org/wiki/Pseudo_Abort for more details on this.
Check logfile: /home/mf/yocto-projects/toradexbsp/build/tmp/work/verdin_imx8mp-tdx-linux/tdx-reference-minimal-image/1.0-r0/pseudo//pseudo.log
Aborted (core dumped)
tar: Unexpected EOF in archive
tar: Unexpected EOF in archive
tar: Error is not recoverable: exiting now

ERROR: Logfile of failure stored in: /home/mf/yocto-projects/toradexbsp/build/tmp/work/verdin_imx8mp-tdx-linux/tdx-reference-minimal-image/1.0-r0/temp/log.do_populate_sdk_ext.1099199
ERROR: Task (/home/mf/yocto-projects/toradexbsp/build/../layers/meta-toradex-demos/recipes-images/images/tdx-reference-minimal-image.bb:do_populate_sdk_ext) failed with exit code '1'
NOTE: Tasks Summary: Attempted 5805 tasks of which 5593 didn't need to be rerun and 1 failed.
NOTE: Writing buildhistory
NOTE: Writing buildhistory took: 2 seconds

Summary: 1 task failed:
  /home/mf/yocto-projects/toradexbsp/build/../layers/meta-toradex-demos/recipes-images/images/tdx-reference-minimal-image.bb:do_populate_sdk_ext
Summary: There was 1 ERROR message shown, returning a non-zero exit code.

One more info, if I try to populate sdk ext for core-image-minimal from poky using verdin-imx8mp machine with tdx-xwayland-rt distro:
clear; DISTRO="tdx-xwayland-rt" bitbake core-image-minimal -c populate_sdk_ext

It populates the sdk successfully.

It seems that the source of the issue is somewhere in tdx-reference-minimal-image bb content.

henrique.tx · January 26, 2022, 1:56pm

Hi @Fide !

From what I understand, devtool is made available after the initial source for the Yocto build environment (in Toradex’s case, after the . export).

So, you need to source the environment before using the devtool, or keep using the terminal where you built your image.

Best regards,

Fide · January 26, 2022, 2:48pm

Hi @henrique.tx ,

Thank you for the reply, this approach is new to me, it seems that I can continue my tasks with that way. But we need to provide a SDK to software developers in the company so that they can also continue their developments.

After sourcing the SDK environment, they usually use

devtool modify app_recipe
then modify the source code using VSCode or directly with VS and VisualGDB
devtool build app_recipe
finally devtool deploy-target app_recipe root@ip

It would be difficult to ask each of them to prepare a yocto environment. So eventually this SDK is needed.

Any suggestion for the initial issue?

Thank you.

henrique.tx · January 26, 2022, 3:39pm

Hi @Fide !

The Yocto environment that I am referring to are the layers, so you can source the export file and obtain the build environment (not necessarily having to actually build something).

The devtool is a python script that depends on some other files within layers/openembedded-core/scripts

The 7 devtool Quick Reference — The Yocto Project ® 4.3.999 documentation gives us the hint:

The devtool command-line tool provides a number of features that help you build, test, and package software. This command is available alongside the bitbake command. Additionally, the devtool command is a key part of the extensible SDK.

You can check its location:

pokyuser@6c6a1da76e61:/workdir/tdxref-bsp-5$ ls
build  export  layers  sstate-cache
pokyuser@6c6a1da76e61:/workdir/tdxref-bsp-5$ . export

### Shell environment set up for builds. ###

...
...
    
pokyuser@6c6a1da76e61:/workdir/tdxref-bsp-5/build$ which devtool
/workdir/tdxref-bsp-5/layers/openembedded-core/scripts/devtool
pokyuser@6c6a1da76e61:/workdir/tdxref-bsp-5/build$

Seems like devtool only needs the openembedded-core folder. If someone needs to use the devtool, they need at least the openembedded-core folder and, of course, some layers/recipes to work on.

I didn’t quite get the difficulty here… Can you elaborate on that?

Best regards,