Been trying over a week to install pvlib, which seems to require a ton of other packages like pandas, numpy, scipy, which in turn need their own compilers, etc… its quite a headache, especially when you consider that nearly every Stackoverflow or equivalent forum expects that you can just type whatever you want into your dockerfile willy-nilly, unlike Toradex’s approach with the configuration panel.
So here is where I stand, I get the following excerpt from a gigantic output:
Questions
- Is there a way to stop the system from overwriting the dockerfile?
- Is there a way to add multiple ENV variables? something like:
ENV BLAS_SRC=“BLAS (Basic Linear Algebra Subprograms)” \ ENV BLAS=“BLAS (Basic Linear Algebra Subprograms)”
- Here are the extrapackages I am installing, some of them seemed to have helped because I get “FOUND” instead of “NOT AVAILABLE” but I seem to have a long way to go:
python3-scipy libblas-dev liblapack-dev libatlas-base-dev libblas3 liblapack3 liblapack-doc build-essential iproute2 python-dev python3-pexpect tzdata rename iputils-ping gfortran
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 62.8/62.8 kB 3.6 MB/s eta 0:00:00
[08-27 15:09:14.845] Collecting h5py
[08-27 15:09:15.409] Downloading h5py-3.7.0.tar.gz (392 kB)
[08-27 15:09:15.514] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 392.4/392.4 kB 861.6 kB/s eta 0:00:00
[08-27 15:09:15.947] Installing build dependencies: started
[08-27 15:10:34.106] Installing build dependencies: still running…
[08-27 15:12:07.646] Installing build dependencies: still running…
[08-27 15:12:15.858] Installing build dependencies: finished with status ‘error’
[08-27 15:12:16.080] e[91m error: subprocess-exited-with-error× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> [293 lines of output]
Ignoring Cython: markers ‘python_version < “3.8”’ don’t match your environment
Ignoring Cython: markers ‘python_version == “3.8”’ don’t match your environment
Collecting setuptools
Downloading setuptools-65.4.0-py3-none-any.whl (1.2 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 194.8 kB/s eta 0:00:00
Collecting wheel
Downloading wheel-0.37.1-py2.py3-none-any.whl (35 kB)
Collecting oldest-supported-numpy
Downloading oldest_supported_numpy-2022.8.16-py3-none-any.whl (3.9 kB)
Collecting pkgconfig
Downloading pkgconfig-1.5.5-py3-none-any.whl (6.7 kB)
Collecting Cython>=0.29.15
Downloading Cython-0.29.32-py2.py3-none-any.whl (986 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 986.3/986.3 kB 273.3 kB/s eta 0:00:00
Collecting numpy==1.19.3
Downloading numpy-1.19.3.zip (7.3 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7.3/7.3 MB 535.5 kB/s eta 0:00:00
Installing build dependencies: started
Installing build dependencies: finished with status ‘done’
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status ‘done’
Preparing metadata (pyproject.toml): started
Preparing metadata (pyproject.toml): still running…
Preparing metadata (pyproject.toml): finished with status ‘error’
error: subprocess-exited-with-error× Preparing metadata (pyproject.toml) did not run successfully. │ exit code: 1 ╰─> [254 lines of output] Running from numpy source directory. setup.py:480: UserWarning: Unrecognized setuptools command, proceeding with generating Cython sources and expanding templates run_build = parse_setuppy_commands() Processing numpy/random/_bounded_integers.pxd.in Processing numpy/random/_pcg64.pyx Processing numpy/random/_philox.pyx Processing numpy/random/mtrand.pyx Processing numpy/random/_generator.pyx Processing numpy/random/_common.pyx Processing numpy/random/_mt19937.pyx Processing numpy/random/_sfc64.pyx Processing numpy/random/_bounded_integers.pyx.in Processing numpy/random/bit_generator.pyx Cythonizing sources blas_opt_info: blas_mkl_info: customize UnixCCompiler libraries mkl_rt not found in ['/usr/local/lib', '/usr/lib', '/usr/lib/arm-linux-gnueabihf'] NOT AVAILABLE blis_info: libraries blis not found in ['/usr/local/lib', '/usr/lib', '/usr/lib/arm-linux-gnueabihf'] NOT AVAILABLE openblas_info: libraries openblas not found in ['/usr/local/lib', '/usr/lib', '/usr/lib/arm-linux-gnueabihf'] NOT AVAILABLE atlas_3_10_blas_threads_info: Setting PTATLAS=ATLAS libraries tatlas not found in ['/usr/local/lib', '/usr/lib', '/usr/lib/arm-linux-gnueabihf/atlas', '/usr/lib/arm-linux-gnueabihf'] NOT AVAILABLE atlas_3_10_blas_info: libraries satlas not found in ['/usr/local/lib', '/usr/lib', '/usr/lib/arm-linux-gnueabihf/atlas', '/usr/lib/arm-linux-gnueabihf'] NOT AVAILABLE atlas_blas_threads_info: Setting PTATLAS=ATLAS libraries ptf77blas,ptcblas,atlas not found in ['/usr/local/lib', '/usr/lib', '/usr/lib/arm-linux-gnueabihf/atlas', '/usr/lib/arm-linux-gnueabihf'] NOT AVAILABLE atlas_blas_info: get_default_fcompiler: matching types: '['gnu95', 'intel', 'lahey', 'pg', 'nv', 'absoft', 'nag', 'vast', 'compaq', 'intele', 'intelem', 'gnu', 'g95', 'pathf95', 'nagfor']' customize Gnu95FCompiler Found executable /usr/bin/gfortran customize Gnu95FCompiler customize Gnu95FCompiler using config compiling '_configtest.c': /* This file is generated from numpy/distutils/system_info.py */ void ATL_buildinfo(void); int main(void) { ATL_buildinfo(); return 0; } C compiler: arm-linux-gnueabihf-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -ffile-prefix-map=/build/python3.9-jS0VHk/python3.9-3.9.2=. -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -ffile-prefix-map=/build/python3.9-jS0VHk/python3.9-3.9.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC compile options: '-c' arm-linux-gnueabihf-gcc: _configtest.c arm-linux-gnueabihf-gcc -pthread _configtest.o -lf77blas -lcblas -latlas -o _configtest FOUND: language = c define_macros = [('HAVE_CBLAS', None), ('NO_ATLAS_INFO', -1)] libraries = ['f77blas', 'cblas', 'atlas', 'f77blas', 'cblas'] library_dirs = ['/usr/lib/arm-linux-gnueabihf'] FOUND: language = c define_macros = [('HAVE_CBLAS', None), ('NO_ATLAS_INFO', -1)] libraries = ['f77blas', 'cblas', 'atlas', 'f77blas', 'cblas'] library_dirs = ['/usr/lib/arm-linux-gnueabihf'] non-existing path in 'numpy/distutils': 'site.cfg' lapack_opt_info: lapack_mkl_info: libraries mkl_rt not found in ['/usr/local/lib', '/usr/lib', '/usr/lib/arm-linux-gnueabihf'] NOT AVAILABLE openblas_lapack_info: libraries openblas not found in ['/usr/local/lib', '/usr/lib', '/usr/lib/arm-linux-gnueabihf'] NOT AVAILABLE openblas_clapack_info: libraries openblas,lapack not found in ['/usr/local/lib', '/usr/lib', '/usr/lib/arm-linux-gnueabihf'] NOT AVAILABLE flame_info: libraries flame not found in ['/usr/local/lib', '/usr/lib', '/usr/lib/arm-linux-gnueabihf'] NOT AVAILABLE atlas_3_10_threads_info: Setting PTATLAS=ATLAS libraries lapack_atlas not found in /usr/local/lib libraries tatlas,tatlas not found in /usr/local/lib libraries lapack_atlas not found in /usr/lib libraries tatlas,tatlas not found in /usr/lib libraries lapack_atlas not found in /usr/lib/arm-linux-gnueabihf/atlas libraries tatlas,tatlas not found in /usr/lib/arm-linux-gnueabihf/atlas libraries tatlas,tatlas not found in /usr/lib/arm-linux-gnueabihf <class 'numpy.distutils.system_info.atlas_3_10_threads_info'> NOT AVAILABLE atlas_3_10_info: libraries lapack_atlas not found in /usr/local/lib libraries satlas,satlas not found in /usr/local/lib libraries lapack_atlas not found in /usr/lib libraries satlas,satlas not found in /usr/lib libraries lapack_atlas not found in /usr/lib/arm-linux-gnueabihf/atlas libraries satlas,satlas not found in /usr/lib/arm-linux-gnueabihf/atlas libraries satlas,satlas not found in /usr/lib/arm-linux-gnueabihf <class 'numpy.distutils.system_info.atlas_3_10_info'> NOT AVAILABLE atlas_threads_info: Setting PTATLAS=ATLAS libraries lapack_atlas not found in /usr/local/lib libraries ptf77blas,ptcblas,atlas not found in /usr/local/lib libraries lapack_atlas not found in /usr/lib libraries ptf77blas,ptcblas,atlas not found in /usr/lib libraries lapack_atlas not found in /usr/lib/arm-linux-gnueabihf/atlas libraries ptf77blas,ptcblas,atlas not found in /usr/lib/arm-linux-gnueabihf/atlas libraries ptf77blas,ptcblas,atlas not found in /usr/lib/arm-linux-gnueabihf <class 'numpy.distutils.system_info.atlas_threads_info'> NOT AVAILABLE atlas_info: libraries lapack_atlas not found in /usr/local/lib libraries f77blas,cblas,atlas not found in /usr/local/lib libraries lapack_atlas not found in /usr/lib libraries f77blas,cblas,atlas not found in /usr/lib libraries lapack_atlas not found in /usr/lib/arm-linux-gnueabihf/atlas libraries f77blas,cblas,atlas not found in /usr/lib/arm-linux-gnueabihf/atlas <class 'numpy.distutils.system_info.atlas_info'> get_default_fcompiler: matching types: '['gnu95', 'intel', 'lahey', 'pg', 'nv', 'absoft', 'nag', 'vast', 'compaq', 'intele', 'intelem', 'gnu', 'g95', 'pathf95', 'nagfor']' customize Gnu95FCompiler customize Gnu95FCompiler customize Gnu95FCompiler using config compiling '_configtest.c': /* This file is generated from numpy/distutils/system_info.py */ void ATL_buildinfo(void); int main(void) { ATL_buildinfo(); return 0; } C compiler: arm-linux-gnueabihf-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -ffile-prefix-map=/build/python3.9-jS0VHk/python3.9-3.9.2=. -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -ffile-prefix-map=/build/python3.9-jS0VHk/python3.9-3.9.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC compile options: '-c' arm-linux-gnueabihf-gcc: _configtest.c arm-linux-gnueabihf-gcc -pthread _configtest.o -lf77blas -lcblas -latlas -o _configtest FOUND: language = f77 libraries = ['lapack', 'f77blas', 'cblas', 'atlas', 'f77blas', 'cblas'] library_dirs = ['/usr/lib/arm-linux-gnueabihf'] define_macros = [('NO_ATLAS_INFO', -1)] FOUND: language = f77 libraries = ['lapack', 'f77blas', 'cblas', 'atlas', 'f77blas', 'cblas'] library_dirs = ['/usr/lib/arm-linux-gnueabihf'] define_macros = [('NO_ATLAS_INFO', -1)] /usr/lib/python3.9/distutils/dist.py:274: UserWarning: Unknown distribution option: 'define_macros' warnings.warn(msg) running dist_info running build_src build_src building py_modules sources creating build creating build/src.linux-armv7l-3.9 creating build/src.linux-armv7l-3.9/numpy creating build/src.linux-armv7l-3.9/numpy/distutils building library "npymath" sources creating build/src.linux-armv7l-3.9/numpy/core creating build/src.linux-armv7l-3.9/numpy/core/src creating build/src.linux-armv7l-3.9/numpy/core/src/npymath conv_template:> build/src.linux-armv7l-3.9/numpy/core/src/npymath/npy_math_internal.h adding 'build/src.linux-armv7l-3.9/numpy/core/src/npymath' to include_dirs. conv_template:> build/src.linux-armv7l-3.9/numpy/core/src/npymath/ieee754.c conv_template:> build/src.linux-armv7l-3.9/numpy/core/src/npymath/npy_math_complex.c None - nothing done with h_files = ['build/src.linux-armv7l-3.9/numpy/core/src/npymath/npy_math_internal.h'] building library "npysort" sources creating build/src.linux-armv7l-3.9/numpy/core/src/common conv_template:> build/src.linux-armv7l-3.9/numpy/core/src/common/npy_sort.h adding 'build/src.linux-armv7l-3.9/numpy/core/src/common' to include_dirs. creating build/src.linux-armv7l-3.9/numpy/core/src/npysort conv_template:> build/src.linux-armv7l-3.9/numpy/core/src/npysort/quicksort.c conv_template:> build/src.linux-armv7l-3.9/numpy/core/src/npysort/mergesort.c conv_template:> build/src.linux-armv7l-3.9/numpy/core/src/npysort/timsort.c conv_template:> build/src.linux-armv7l-3.9/numpy/core/src/npysort/heapsort.c conv_template:> build/src.linux-armv7l-3.9/numpy/core/src/npysort/radixsort.c conv_template:> build/src.linux-armv7l-3.9/numpy/core/src/common/npy_partition.h conv_template:> build/src.linux-armv7l-3.9/numpy/core/src/npysort/selection.c conv_template:> build/src.linux-armv7l-3.9/numpy/core/src/common/npy_binsearch.h conv_template:> build/src.linux-armv7l-3.9/numpy/core/src/npysort/binsearch.c None - nothing done with h_files = ['build/src.linux-armv7l-3.9/numpy/core/src/common/npy_sort.h', 'build/src.linux-armv7l-3.9/numpy/core/src/common/npy_partition.h', 'build/src.linux-armv7l-3.9/numpy/core/src/common/npy_binsearch.h'] building library "npyrandom" sources building extension "numpy.core._multiarray_tests" sources creating build/src.linux-armv7l-3.9/numpy/core/src/multiarray conv_template:> build/src.linux-armv7l-3.9/numpy/core/src/multiarray/_multiarray_tests.c building extension "numpy.core._multiarray_umath" sources Traceback (most recent call last): File "/usr/local/lib/python3.9/dist-packages/pip/_vendor/pep517/in_process/_in_process.py", line 363, in <module> main() File "/usr/local/lib/python3.9/dist-packages/pip/_vendor/pep517/in_process/_in_process.py", line 345, in main json_out['return_val'] = hook(**hook_input['kwargs']) File "/usr/local/lib/python3.9/dist-packages/pip/_vendor/pep517/in_process/_in_process.py", line 164, in prepare_metadata_for_build_wheel return hook(metadata_directory, config_settings) File "/tmp/pip-build-env-ag4p0hdp/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 157, in prepare_metadata_for_build_wheel self.run_setup() File "/tmp/pip-build-env-ag4p0hdp/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 248, in run_setup super(_BuildMetaLegacyBackend, File "/tmp/pip-build-env-ag4p0hdp/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 142, in run_setup exec(compile(code, __file__, 'exec'), locals()) File "setup.py", line 508, in <module> setup_package() File "setup.py", line 500, in setup_package setup(**metadata) File "/tmp/pip-install-xg_3w20w/numpy_46ca8b7ef3c74b5ba5ec053c51ebbcba/numpy/distutils/core.py", line 169, in setup return old_setup(**new_attr) File "/tmp/pip-build-env-ag4p0hdp/overlay/lib/python3.9/site-packages/setuptools/__init__.py", line 165, in setup return distutils.core.setup(**attrs) File "/usr/lib/python3.9/distutils/core.py", line 148, in setup dist.run_commands() File "/usr/lib/python3.9/distutils/dist.py", line 966, in run_commands self.run_command(cmd) File "/usr/lib/python3.9/distutils/dist.py", line 985, in run_command cmd_obj.run() File "/tmp/pip-build-env-ag4p0hdp/overlay/lib/python3.9/site-packages/setuptools/command/dist_info.py", line 31, in run egg_info.run() File "/tmp/pip-install-xg_3w20w/numpy_46ca8b7ef3c74b5ba5ec053c51ebbcba/numpy/distutils/command/egg_info.py", line 24, in run self.run_command("build_src") File "/usr/lib/python3.9/distutils/cmd.py", line 313, in run_command self.distribution.run_command(command) File "/usr/lib/python3.9/distutils/dist.py", line 985, in run_command cmd_obj.run() File "/tmp/pip-install-xg_3w20w/numpy_46ca8b7ef3c74b5ba5ec053c51ebbcba/numpy/distutils/command/build_src.py", line 144, in run self.build_sources() File "/tmp/pip-install-xg_3w20w/numpy_46ca8b7ef3c74b5ba5ec053c51ebbcba/numpy/distutils/command/build_src.py", line 161, in build_sources self.build_extension_sources(ext) File "/tmp/pip-install-xg_3w20w/numpy_46ca8b7ef3c74b5ba5ec053c51ebbcba/numpy/distutils/command/build_src.py", line 318, in build_extension_sources sources = self.generate_sources(sources, ext) File "/tmp/pip-install-xg_3w20w/numpy_46ca8b7ef3c74b5ba5ec053c51ebbcba/numpy/distutils/command/build_src.py", line 378, in generate_sources source = func(extension, build_dir) File "numpy/core/setup.py", line 436, in generate_config_h moredefs, ignored = cocache.check_types(config_cmd, ext, build_dir) File "numpy/core/setup.py", line 47, in check_types out = check_types(*a, **kw) File "numpy/core/setup.py", line 292, in check_types raise SystemError( SystemError: Cannot compile 'Python.h'. Perhaps you need to install python-dev|python-devel. [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. error: metadata-generation-failed × Encountered error while generating package metadata. ╰─> See above for output. note: This is an issue with the package mentioned above, not pip. hint: See above for details. [end of output]note: This error originates from a subprocess, and is likely not a problem with pip.
e[0m
[08-27 15:12:16.086] e[91merror: subprocess-exited-with-error× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> See above for output.note: This error originates from a subprocess, and is likely not a problem with pip.
e[0m
[08-27 15:12:32.572] Local docker exception.
[08-27 15:12:32.573] Error (530) - Docker exception: The command '/bin/sh -c dos2unix /requirements.txt && dos2unix /setup.sh && dos2unix /cleanup.sh && chmod a+x /setup.sh && ch