Kernel is not within downloads/*tar.gz but within dynamic folder build/tmp-glibc/workshared/...?

Technical Support
, , ,
1

Hello,
we are building apalis tk1 images on a off-network build machine. Therefore we have to fetch all tarballs first and checkin on a local svn/git repository. This works successfully with all the tar.gz files within the downloads folder.
BUT where is the kernel? ERROR: linux-toradex-3.10.40+gitrAUTOINC+e9636255d1-2.7.4 do_fetch.
It seems that the kernel sources are fetched onthefly into the dynamically created folder build/tmp-glibc/workshared/apalis-tk1/kernel-source. Is there any defined tar.gz or other static downloads location for this which can be found by the recipe?
Best regards
Dirk

2

Yes, it should just clone off the bare git repo it previously cloned into your downloads folder when it did have a proper Internet connection.

3

But how can I get

  1. the cloned toradex kernel e.g.“downloads/git2/git.toradex.com.linux-toradex.git”
  2. prefetched completely
  3. and with the correct branch/version (toradex_tk1_l4t_r21.5)
  4. to a valid download folder (“downloads/git2/linux-toradex”?)
  5. to prevent fetching from the internet again during the offnetwork build?
4

“bitbake angstrom-lxde-image -c fetchall” apparently doesn’t fetch all the needed kernel files

5

Well it works here.
e.g.

mkdir fetch_test; cd fetch_test
repo init -u http://git.toradex.com/toradex-bsp-platform.git -b LinuxImageV2.7
repo sync
. export
MACHINE=apalis-tk1 bitbake angstrom-lxde-image -c fetchall
cd downloads/git2/git.toradex.com.linux-toradex.git/
git log --decorate toradex_tk1_l4t_r21.5

The branch/commit has been fetched.

Then disconnecting the network and starting the kernel build succeeds.

cd -
MACHINE=apalis-tk1 bitbake virtual/kernel
6

Fetch just fills /tmp-glibc/work and /tmp-glibc/work-shared with /temp folders containing log files, but where are all the “arch”, “lib”, “net”, “drivers”, “block”,… sources of the kernel?

7

The fetch task fetches the sources into the directory defined with DL_DIR.
(local.conf, ‘DL_DIR ?= “${TOPDIR}/downloads”’) e.g. default build/downloads.

Populating the sources in /tmp-glibc/work and /tmp-glibc/work-shared from the downloaded sources in DL_DIR is then later done with the unpack and patch tasks.

See also the relevant chapter in the Yocto documentation.

8

~/fetch_test$ repo init -u Index of /toradex-bsp-platform.git -b LinuxImageV2.7
Get https://gerrit.googlesource.com/git-repo/clone.bundle
Get git-repo - Git at Google
Get Index of /toradex-bsp-platform.git
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:–:-- --:–:-- --:–:-- 0
curl: (22) The requested URL returned error: 404 Not Found

9

Well, of course one can not repo init anything without a proper Internet connection! That part (e.g. all layers and such) would need to be copied from the good known already populated (aka repo initted) machine.

10

No! “MACHINE=apalis-tk1 bitbake angstrom-lxde-image -c fetchall”
populates build/tmp-glibc/work and work-shared and sstate-control and stamps and sysroots and…
Btw I have read the relevant chapter w/i yocto doc: "This variation of the BitBake command guarantees that you have all the sources for that BitBake target should you disconnect from the Internet and want to do the build later offline."´
That’s what I expected, but apparently doesn’t solely populate “build/downloads”.
However, offline build started. So let’s see if this time really everything has been fetched in which folder whatever…

11

online
MACHINE=apalis-tk1 bitbake angstrom-lxde-image -c fetchall → ok

offline
MACHINE=apalis-tk1 bitbake virtual/kernel → ok

offline
MACHINE=apalis-tk1 bitbake angstrom-lxde-image → nok fatal: unable to access ‘Index of /backports-toradex.git’: Received HTTP code 502 from proxy after CONNECT

12

Again

online
repo sync → ok, new (previously not fetched objects)

online
MACHINE=apalis-tk1 bitbake angstrom-lxde-image -c fetchall → ok, no access errors

offline
MACHINE=apalis-tk1 bitbake angstrom-lxde-image → nok fatal: unable to access ‘Index of /backports-toradex.git’: Received HTTP code 502 from proxy after CONNECT

As I mentioned before: There are packages that are NOT fetched before the unpack, configure, compile…

13

cd downloads/git2/git.toradex.com.backports-toradex.git

git log --decorate toradex_tk1_l4t_r21.5

fatal: ambiguous argument ‘toradex_tk1_l4t_r21.5’: unknown revision or path not in the working tree.
Use ‘–’ to separate paths from revisions, like this:
‘git […] – […]’

14

ERROR: Task (/home/user/fetch_test/build/…/layers/meta-toradex-bsp-common/recipes-kernel/backports/backports_4.11.bb:do_unpack) failed with exit code ‘1’

Kernel build succeeded, but recipes-kernel fails???

15

I believe an error within backports_4.11.bb that has no reference to downloads folder:

SUMMARY = “Backported kernel drivers”
HOMEPAGE = “https://backports.wiki.kernel.org”
SECTION = “kernel/modules”
LICENSE = “GPLv2”
LIC_FILES_CHKSUM = “file://COPYING;md5=d7810fab7487fb0aad327b76f1be7cd7”
INC_PR = “r1”

SRCREV = “toradex-${PV}”
SRC_URI = "
git://git.toradex.com/backports-toradex.git;protocol=git;branch=${SRCREV}
file://makefile.patch
file://config
"

S = “${WORKDIR}/git”

inherit module

MAKE_TARGETS = “modules”
MODULES_INSTALL_TARGET = “modules_install”

KERNEL_MODULE_PACKAGE_PREFIX = “${BPN}-”

EXTRA_OEMAKE = " KLIB=${STAGING_KERNEL_DIR} KLIB_BUILD=${STAGING_KERNEL_BUILDDIR} "

do_configure() {

unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS
make CC="${BUILD_CC}" LD="${BUILD_LD}" AR="${BUILD_AR}" \
     -C ${S}/kconf  O=${S}/kconf conf

cp ${WORKDIR}/config ${S}/.config
unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS
oe_runmake oldconfig

}

16

offline
MACHINE=apalis-tk1 bitbake backports → nok missing backports-toradex

online
MACHINE=apalis-tk1 bitbake backports -c fetchall → ok, nothing to fetch?

offline
MACHINE=apalis-tk1 bitbake backports → nok, tries to fetch something

17

Hi

I see. I so far only checked the kernel sources. The backports recipe has a bug for your use case.
As it does not define a specific git hash but only the git branch in SRCREV the fetcher needs to access the network on each build to check what is latest and greatest.

By fixing that you should be able to build backports without network access.

I pushed the following change to meta-toradex-bsp-common, morty-next:

diff --git a/recipes-kernel/backports/backports_4.11.bb b/recipes-kernel/backports/backports_4.11.bb
index b84f378..c6523b1 100644
--- a/recipes-kernel/backports/backports_4.11.bb
+++ b/recipes-kernel/backports/backports_4.11.bb
@@ -5,9 +5,9 @@ LICENSE = "GPLv2"
 LIC_FILES_CHKSUM = "file://COPYING;md5=d7810fab7487fb0aad327b76f1be7cd7"
 INC_PR = "r1"
 
-SRCREV = "toradex-${PV}"
+SRCREV = "ad615b7c1f2bfdb766f225805569f0a1495edc69"
 SRC_URI = " \
-    git://git.toradex.com/backports-toradex.git;protocol=git;branch=${SRCREV} \
+    git://git.toradex.com/backports-toradex.git;protocol=git;branch=toradex-${PV} \
     file://makefile.patch \
     file://config \
     "
18

Hi,
thank you very much for your help. How can I access your changes within morty-next?
My repo is initted to the branch LinuxImageV2.7.
Regards
Dirk

19

We updated the repo manifest to include the fix so you may just repo sync it now.

20

The manifest worked fine until Friday.
Now even with online network connection:

1 repo init -u Index of /toradex-bsp-platform.git -b LinuxImageV2.7

2 repo sync

3 . export

4 MACHINE=apalis-tk1 bitbake angstrom-lxde-image -c fetchall

leads to multiple errors:RROR: u-boot-toradex-2016.11-2.7.5+gitrAUTOINC+52259cf22a do_fetch: Fetcher failure for URL: ‘git://git.toradex.com/u-boot-toradex.git;protocol=git;branch=2016.11-toradex’. Unable to fetch URL from any source.