Hello dotnet team, I am having trouble with a cross compiled dotnet hello world …application.
I am building on linux 64bit for an ARM processor (SAMA5D31) with the following information:
```
processor : 0
model name : ARMv7 Processor rev 1 (v7l)
BogoMIPS : 351.43
Features : half thumb fastmult vfp edsp thumbee vfpv3 vfpv3d16 tls vfpv4
CPU implementer : 0x41
CPU architecture: 7
CPU variant : 0x0
CPU part : 0xc05
CPU revision : 1
Hardware : Atmel SAMA5
Revision : 0000
Serial : 0000000000000000
```
I keep receiving "Illegal Instruction" when running my code on the ARM machine.
I have been following the directions from "Build CoreCLR on Linux" (docker method), and the following issues:
https://github.com/dotnet/runtime/issues/9969
https://github.com/dotnet/coreclr/pull/27890
https://github.com/dotnet/runtime/pull/546
https://github.com/dotnet/runtime/pull/471
I have built both dotnet/coreclr (git hash a9f3fc16483eecfc47fb79c362811d870be02249 dotnet version 3.1) and dotnet/runtime (git hash 516956a01fee9aa732b199012a6ea0bcc2bd789f dotnet version 5.0.0-dev).
My command to build the runtime for dotnet coreclr is:
`sudo docker run --rm -v /media/calder/secondary/git/coreclr:/coreclr -w /coreclr -e ROOTFS_DIR=/crossrootfs/arm mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-16.04-cross-14.04-23cacb0-20190923200213 ./build.sh -arm -release -cross -stripsymbols -cmakeargs -DCLR_ARM_FPU_CAPABILITY=0x3 -cmakeargs -DCLR_ARM_FPU_TYPE=vfpv3-d16`
My command to build the runtime for dotnet runtime is:
`sudo docker run --rm -v /media/calder/secondary/git/runtime:/runtime -w /runtime -e ROOTFS_DIR=/crossrootfs/arm mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-16.04-cross-14.04-23cacb0-20191023143847 ./build.sh --arch arm --cross --cmakeargs -DCLR_ARM_FPU_CAPABILITY=0x3 --cmakeargs -DCLR_ARM_FPU_TYPE=vfpv3-d16 --clang9 --configuration Release`
Not sure if this is relevant but out dotnet comment section is:
```
String dump of section '.comment':
[ 0] GCC: (Ubuntu/Linaro 4.8.4-2ubuntu1~14.04.4) 4.8.4
[ 32] clang version 9.0.1-svn374858-1~exp1~20191015044046.62 (branches/release_90)
```
My simple hello world application is made using my standard dotnet command to compile helloworld.
I have tried building just the byte code and using it with the runtime (this is using my usual linux64 dotnet compiler):
`dotnet build -c Release`
I then pass my bytecode in (bin/Release/netcoreapp3.1/) along with the compiled runtime to my ARM machine and run with:
`~/dotnet-lib/dotnet helloworld.dll`
I have tried this with both 3.1/5.0.0-dev.
I have also tried building a self contained executable:
`dotnet publish --self-contained -r linux-arm`
And then replacing dlls and .so files with my runtime libraries, but the same issue occurs.
Dependencies such as stdlibc++6 has been compiled for ARM as well.
I requested help from my colleague who is more experienced with embedded systems. He provided some stacktraces:
```
Program received signal SIGILL, Illegal instruction.
std::__detail::_Rehash_base<std::string, std::pair<std::string const, std::string>, std::allocator<std::pair<std::string const, std::string> >, std::__detail::_Select1st, std::equal_to<std::string>, std::hash<std::string>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::reserve (this=0x4246c8, __n=24)
at /crossrootfs/arm/usr/lib/gcc/arm-linux-gnueabihf/4.8/../../../../include/c++/4.8/bits/hashtable_policy.h:819
819 /crossrootfs/arm/usr/lib/gcc/arm-linux-gnueabihf/4.8/../../../../include/c++/4.8/bits/hashtable_policy.h: No such file or directory.
#0 std::__detail::_Rehash_base<std::string, std::pair<std::string const, std::string>, std::allocator<std::pair<std::string const, std::string> >, std::__detail::_Select1st, std::equal_to<std::string>, std::hash<std::string>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::reserve (this=0x4246c8, __n=24)
at /crossrootfs/arm/usr/lib/gcc/arm-linux-gnueabihf/4.8/../../../../include/c++/4.8/bits/hashtable_policy.h:819
#1 std::unordered_map<std::string, std::string, std::hash<std::string>, std::equal_to<std::string>, std::allocator<std::pair<std::string const, std::string> > >::reserve
(this=0x4246c8, __n=24)
at /crossrootfs/arm/usr/lib/gcc/arm-linux-gnueabihf/4.8/../../../../include/c++/4.8/bits/unordered_map.h:743
#2 coreclr_property_bag_t::coreclr_property_bag_t (this=0x4246c8)
at /runtime/src/installer/corehost/cli/hostpolicy/coreclr.cpp:224
#3 0xb6bdbae0 in hostpolicy_context_t::hostpolicy_context_t (this=0x424698)
at /runtime/src/installer/corehost/cli/hostpolicy/hostpolicy_context.h:15
#4 0xb6bd7ee4 in (anonymous namespace)::create_hostpolicy_context (
hostpolicy_init=..., args=..., breadcrumbs_enabled=true)
at /runtime/src/installer/corehost/cli/hostpolicy/hostpolicy.cpp:120
#5 0xb6bd7cf8 in corehost_main (argc=2, argv=<optimized out>)
at /runtime/src/installer/corehost/cli/hostpolicy/hostpolicy.cpp:381
#6 0xb6c2a42e in execute_app (impl_dll_dir=..., init=0x424c98,
argc=<optimized out>, argv=<optimized out>)
---Type <return> to continue, or q <return> to quit---
at /runtime/src/installer/corehost/cli/fxr/fx_muxer.cpp:145
#7 (anonymous namespace)::read_config_and_execute (host_command=..., host_info=...,
app_candidate=..., opts=..., new_argc=2, new_argv=0xbefffcb4,
mode=<optimized out>, out_buffer=<optimized out>, buffer_size=<optimized out>,
required_buffer_size=<optimized out>)
at /runtime/src/installer/corehost/cli/fxr/fx_muxer.cpp:501
#8 fx_muxer_t::handle_exec_host_command (host_command=..., host_info=...,
app_candidate=..., opts=..., argc=2, argv=0xbefffcb4, argoff=1, mode=muxer,
result_buffer=0x0, buffer_size=0, required_buffer_size=0x0)
at /runtime/src/installer/corehost/cli/fxr/fx_muxer.cpp:958
#9 0xb6c2909c in fx_muxer_t::execute (host_command=..., argc=2, argv=0xbefffcb4,
host_info=..., result_buffer=0x0, buffer_size=0, required_buffer_size=0x0)
at /runtime/src/installer/corehost/cli/fxr/fx_muxer.cpp:547
#10 0xb6c24644 in hostfxr_main_startupinfo (argc=1103101952, argv=0x41c00000,
host_path=<optimized out>, dotnet_root=<optimized out>,
app_path=0x4230f4 "/home/dlmaint-sn/dotnet/dotnet.dll")
at /runtime/src/installer/corehost/cli/fxr/hostfxr.cpp:33
#11 0x00409fa4 in exe_start (argc=2, argv=0xbefffcb4)
at /runtime/src/installer/corehost/corehost.cpp:219
#12 0x0040a692 in main (argc=2, argv=0xbefffcb4)
at /runtime/src/installer/corehost/corehost.cpp:286
```
And then this conversation:
> 0xb6bcdae6 <+122>: vcvt.f64.f32 d17, s18
> 11:32
> Looks like it's trying to use D17 as a register which wouldn't exist on this FPU. So it is likely still the fpu issue.
> 11:35
> __builtin_ceil(__n / max_load_factor())
> That builtin function for ceil is causing it.
> 11:39
> That's being called from coreclr_property_bag_t()
> That call is made inside a stdc++ header file. The builtin itself is defined in the compiler. (edited)
So from what I can tell, one of two things could be happening:
1. I am missing something either in compilation steps or in dependencies.
2. There is some missed case of applying the vfpv3-d16 flag maybe?
Any assistance would be greatly appreciated.
Thanks,
Calder