Trying to use a NETBSD_4.0 evbarm VM to compile smb binaries by myself

[tbex78]

Following the answer in #144

Thx for your help, I looked your link, in fact I had already previously successfully installed NetBSD-10.1-evbarm-aarch64.iso with qemu and can access it with ssh remotely.
I thought this version couldn't compile properly the compatible smb binaries for a NetBSD 6.0 and 4.0, so I discarded it.
so from here, I git clone TimeCapsuleSMB repo and use the build directory?

[tbex78]

before trying to use the TimeCapsuleSMB build directory, on your NetBSD VM, you need to download the NetBSD sources and compile the cross-toolchain. To do that :
with root :

cd /usr
mkdir -p /usr/obj
chmod 755 /usr/obj
git clone https://github.com/NetBSD/src.git src

(very long to process, take lot of space, I advice you to create your vm image with at least 15G)

cd /usr/src
./build.sh -m evbarm -a earmv4 tools

(the compilation process is very, very, VERY long too)

[tbex78]

cd /usr/src
./build.sh -m evbarm -a earmv4 tools

(the compilation process is very, very, VERY long too)

You can drastically speedup the compilation using multicore.
add -j parameter to indicate your multicore cpu number
./build.sh -j 16 -m evbarm -a earmv4 tools

[jamesyc]

You can just run https://github.com/jamesyc/TimeCapsuleSMB/blob/main/build/downloadoldle.sh and then https://github.com/jamesyc/TimeCapsuleSMB/blob/main/build/bootstrapoldle.sh to download and build the netbsd4 tools/distribution

[jamesyc]

Or read https://github.com/jamesyc/TimeCapsuleSMB/blob/main/build/Makefile

[tbex78]

Some useful terminal commands post VM installation :

#log as root
su
export PKG_PATH=https://cdn.netbsd.org/pub/pkgsrc/packages/NetBSD/aarch64/10.0_2026Q1/All/
pkg_add pkgin
pkgin update
pkgin install gcc7 gmake git wget
pkgin install python27
ln -s /usr/pkg/bin/python2.7 /usr/pkg/bin/python
wget https://cdn.netbsd.org/pub/pkgsrc/current/pkgsrc.tar.gz
tar -xzf pkgsrc.tar.gz /usr/
git clone https://github.com/jamesyc/TimeCapsuleSMB.git
cd TimeCapsuleSMB
pkgin install python314
ln -s /usr/pkg/bin/python3.14 /usr/pkg/bin/python3
#didn't find packages dependances for samba4 so I compiled samba3 from source
#pkgin install samba3
cd /usr/pkgsrc/net/samba
unset PKG_PATH
make clean
make
pkgin install sshpass
cd /home/username/TimeCapsuleSMB
./tcapsule bootstrap
.venv/bin/tcapsule configure
cd /home/username/TimeCapsuleSMB/build
cp .env.example .env
#vi .env and replace timecapsule.local by ip of the time capsule

#for NetBSD4
make sdk-download-netbsd4le
#./downloadoldle.sh
make sdk-bootstrap-netbsd4le
#./bootstrapoldle.sh
#(very long)

make sdk-download-netbsd4be
#./downloadoldbe.sh
make sdk-bootstrap-netbsd4be
#./bootstrapoldbe.sh
#(very long)


#for NetBSD6
make sdk-download-netbsd7
#./download.sh
make sdk-bootstrap-netbsd7
#./bootstrap.sh
#(very long)

[tbex78]

my qemu NetBSD 10.1 VM scripts one for the installation, one post installation

#start-netbsd-10_1-install.sh (Installation)

#!/bin/sh

qemu-system-aarch64 \
  -accel hvf \
  -M virt \
  -cpu host \
  -smp 14 \
  -m 16G \
  -drive file=NetBSD-10.1-evbarm-aarch64.iso,media=cdrom,format=raw \
  -drive if=pflash,format=raw,file=/opt/homebrew/share/qemu/edk2-aarch64-code.fd,readonly=on \
  -drive file=netbsd10.img,if=virtio,format=qcow2 \
  -nic vmnet-bridged,ifname=en0,model=virtio-net-pci \
  -boot d \
  -nographic

#start-netbsd-10_1.sh (post Installation)

#!/bin/sh

qemu-system-aarch64 \
  -accel hvf \
  -M virt \
  -cpu host \
  -smp 14 \
  -m 16G \
  -drive if=pflash,format=raw,file=/opt/homebrew/share/qemu/edk2-aarch64-code.fd,readonly=on \
  -drive file=netbsd10.img,if=virtio,format=qcow2 \
  -nic vmnet-bridged,ifname=en0,model=virtio-net-pci \
  -boot d \
  -nographic

scripts to execute with sudo (root privilege) if you want the vm to be be seen on your home network by ic vmnet-bridged qemu argument

-accel hvf
hardware acceleration (optional argument)
-smp 14
is the cpu core number of your host machine (optional argument)
-m 16G
the ram you want the VM to have

[tbex78]

What are the differences between netbsd4le and netbsd4be?
Is there a way that the compile process on make samba4-netbsd* stop asking repeatdly several times the Time Capsule password?

[jamesyc]

Gen 1-2 are Big Endian
Gen 3-4 are Little Endian

[tbex78]

Is there a way that the compile process on make samba4-netbsd* stop asking repeatdly several times the Time Capsule password?

fill those variables in .env file
TC_NETBSD4LE_PASSWORD
TC_NETBSD4BE_PASSWORD
TC_NETBSD7_PASSWORD

[jamesyc]

You can use a ssh key for login instead of a password

[tbex78]

You can use a ssh key for login instead of a password

already did 😉

[tbex78]

Successfully compile

make advertisers-all
make mdns-all
make nbns-all

but for :

make samba4-netbsd4le
make samba4-netbsd4be
make samba4-netbsd7

ask for the time capsule password lot of times, I didn't go all the way and CTRL+C. Why it asks for it? Did it deploy the binaries after the compile?

[jamesyc]

That's the configure probes for cross exec

See https://github.com/jamesyc/TimeCapsuleSMB/blob/main/build/samba4-cross-exec.sh

[jamesyc]

You can generate the answers once via https://github.com/jamesyc/TimeCapsuleSMB/blob/main/build/generate-samba4x-cross-answers-oldle.sh and reuse this

[tbex78]

So now the password issue is resolved, but for

make samba4-netbsd4le
make samba4-netbsd4be
make samba4-netbsd7

the three fail

make samba4-netbsd4le
./downloadsamba4oldle.sh
Samba 4 download complete.
Log: /root/tc-earmv4-netbsd4/downloadsamba4old.log
./samba4oldle.sh
*** Error code 1

Stop.
make: stopped in /home/tbex/TimeCapsuleSMB/build

cat /root/tc-earmv4-netbsd4/samba4old.log
[ 115/1847] Compiling ASN1 source4/heimdal/lib/asn1/kx509.asn1
22:08:18 runner  cd "/root/tc-samba4-netbsd4le/samba/bin/default/source4/heimdal/lib/asn1" && "/root/tc-samba4-netbsd4le/samba/bin/asn1_compile"   --one-code-file "/root/tc-samba4-netbsd4le/samba/source4/heimdal/lib/asn1/kx509.asn1" kx509_asn1 
sh: Cannot execute ELF binary /root/tc-samba4-netbsd4le/samba/bin/asn1_compile
sh: Cannot execute ELF binary /root/tc-samba4-netbsd4le/samba/bin/asn1_compile
Waf: Leaving directory `/root/tc-samba4-netbsd4le/samba/bin'
Build failed:
 -> task failed (err #2): 
	{task: HEIMDAL_DIGEST_ASN1_ASN1 digest.asn1 -> asn1_digest_asn1.x,digest_asn1.hx,digest_asn1-priv.hx}
 -> task failed (err #2): 
	{task: HEIMDAL_KX509_ASN1_ASN1 kx509.asn1 -> asn1_kx509_asn1.x,kx509_asn1.hx,kx509_asn1-priv.hx}

cat /root/tc-samba4-netbsd4be/samba/bin/config.log
['/root/tc-armeb-netbsd4/tools/bin/armeb--netbsdelf-gcc', 'default/test_1.o', '-o', '/root/tc-samba4-netbsd4be/samba/bin/.conf_check_0/testbuild/default/testprog', '-static', '-L/root/tc-armeb-netbsd4/obj/destdir.evbarm/lib', '-L/root/tc-armeb-netbsd4/obj/destdir.evbarm/usr/lib', '-B/root/tc-armeb-netbsd4/obj/destdir.evbarm/usr/lib', '-B/root/tc-armeb-netbsd4/obj/destdir.evbarm/usr/lib/csu']

Warning: Permanently added '192.168.1.19' (RSA) to the list of known hosts.
Warning: Permanently added '192.168.1.19' (RSA) to the list of known hosts.
Refusing NetBSD 4 be cross-exec on root@192.168.1.19: /bin/sh endianness marker is '\001$'.
Use a build/.env file for a matching oldle/oldbe device before running the build.

returncode 1

cat /root/tc-samba4-netbsd7/samba/bin/config.log
[1/2] Compiling test.c
['/root/tc-earmv4-netbsd7/tools/bin/armv4--netbsdelf-eabi-gcc', '--sysroot=/root/tc-earmv4-netbsd7/obj/destdir.evbarm', '-Os', '-ffunction-sections', '-fdata-sections', '-fomit-frame-pointer', '-fno-unwind-tables', '-fno-asynchronous-unwind-tables', '-fno-ident', '-fno-pie', '-I/root/tc-samba4-netbsd7/samba/lib/replace/../../buildtools/wafsamba', '-MD', '-I/root/tc-earmv4-netbsd7/obj/destdir.evbarm/usr/include', '-D_NETBSD_SOURCE', '-D_LARGEFILE_SOURCE', '-D_FILE_OFFSET_BITS=64', '-D_LARGE_FILES', '-Idefault', '-I..', '-Idefault', '-I..', '-D_SAMBA_BUILD_=4', '-DHAVE_CONFIG_H=1', '../test.c', '-c', '-o', 'default/test_1.o']
[2/2] Linking default/testprog
['/root/tc-earmv4-netbsd7/tools/bin/armv4--netbsdelf-eabi-gcc', '--sysroot=/root/tc-earmv4-netbsd7/obj/destdir.evbarm', 'default/test_1.o', '-o', '/root/tc-samba4-netbsd7/samba/bin/.conf_check_0/testbuild/default/testprog', '-static', '-Wl,--gc-sections', '-Wl,-Map=/root/tc-netbsd7/sbin/smbd.map', '-L/root/tc-earmv4-netbsd7/obj/destdir.evbarm/lib', '-L/root/tc-earmv4-netbsd7/obj/destdir.evbarm/usr/lib']

[1]   Bad system call         "/tmp/tc-samba-p...

returncode 140

[jamesyc]

That error is caused by running the NetBSD 4 big-endian build lane against a little-endian Time Capsule.

The guard in build/samba4-cross-exec.sh will SSH into to the configured device and reads byte 5 of /bin/sh’s ELF header:

\001$ means little-endian ELF
\002$ means big-endian ELF

The failing lane is NETBSD4_ABI=be, so it expects \002$. The device at root@192.168.1.19 returned \001$, so the script refuses to upload/run a big-endian configure probe on a little-endian machine.

[jamesyc]

For your device, configure the .env file, then run:

./downloadoldle.sh
./bootstrapoldle.sh
./downloadsamba4xoldle.sh
./samba4xoldle.sh

[jamesyc]

notice every file is *le.sh

[tbex78]

For your device, configure the .env file, then run:

./downloadoldle.sh
./bootstrapoldle.sh
./downloadsamba4xoldle.sh
./samba4xoldle.sh

I finally succeeded to compile the samba binaries.
I restarded all over again :

• deleted and reinstalled the VM from scratch
• configured it
• git cloned TimeCapsuleSMB
• went in the build directory
• I only focused on doing "le" binaries nothing else
• copied .env.example to .env
• modified it to my need
• executed those command lines :

make sdk-download-netbsd4le
make sdk-bootstrap-netbsd4le
./downloadoldle.sh
./bootstrapoldle.sh
./downloadsamba4xoldle.sh
./samba4xoldle.sh

./samba4xoldle.sh failed with this in the log file :
cat /root/tc-earmv4-netbsd4/samba4xold.log

[210/214] Compiling packaging/systemd/smb.service.in
[211/214] Compiling packaging/systemd/nmb.service.in
[212/214] Compiling packaging/systemd/winbind.service.in
[213/214] Compiling packaging/systemd/samba.service.in
[214/214] Compiling packaging/systemd/samba-bgqd.service.in
[215/262] Processing third_party/heimdal/lib/com_err/parse.y
10:59:10 runner ['/usr/pkg/bin/bison', '-d', '--no-line', '/root/tc-samba4x-netbsd4le/samba/third_party/heimdal/lib/com_err/parse.y', '-o', 'parse.tab.c']
[216/262] Compiling third_party/heimdal/lib/com_err/lex.l
10:59:10 runner ['/usr/bin/flex', '-t', '--noline', '../../third_party/heimdal/lib/com_err/lex.l']
[217/262] Compiling lib/replace/replace.c
10:59:10 runner ['/usr/pkg/gcc7/bin/gcc', '-O', '-fcommon', '-fgnu89-inline', '-D_SAMBA_HOSTCC_', '-O', '-fcommon', '-fgnu89-inline', '-DSTATIC_LIBREPLACE_HOSTCC_MODULES=NULL', '-DSTATIC_LIBREPLACE_HOSTCC_MODULES_PROTO=extern void __LIBREPLACE_HOSTCC_dummy_module_proto(void)', '-Ilib/replace', '-I../../lib/replace', '-Iinclude/public', '-I../../include/public', '-Isource4', '-I../../source4', '-Ilib', '-I../../lib', '-Isource4/lib', '-I../../source4/lib', '-Isource4/include', '-I../../source4/include', '-Iinclude', '-I../../include', '-I.', '-I../..', '../../lib/replace/replace.c', '-c', '-o/root/tc-samba4x-netbsd4le/samba/bin/default/lib/replace/replace.c.1.o', '-D__GNUC_GNU_INLINE__']
[218/262] Compiling lib/replace/strptime.c
10:59:10 runner ['/usr/pkg/gcc7/bin/gcc', '-O', '-fcommon', '-fgnu89-inline', '-D_SAMBA_HOSTCC_', '-O', '-fcommon', '-fgnu89-inline', '-DSTATIC_LIBREPLACE_HOSTCC_MODULES=NULL', '-DSTATIC_LIBREPLACE_HOSTCC_MODULES_PROTO=extern void __LIBREPLACE_HOSTCC_dummy_module_proto(void)', '-Ilib/replace', '-I../../lib/replace', '-Iinclude/public', '-I../../include/public', '-Isource4', '-I../../source4', '-Ilib', '-I../../lib', '-Isource4/lib', '-I../../source4/lib', '-Isource4/include', '-I../../source4/include', '-Iinclude', '-I../../include', '-I.', '-I../..', '../../lib/replace/strptime.c', '-c', '-o/root/tc-samba4x-netbsd4le/samba/bin/default/lib/replace/strptime.c.1.o', '-D__GNUC_GNU_INLINE__']
In file included from ../../lib/replace/strptime.c:25:0:
../../lib/replace/replace.h:45:2: error: #error -D__STDC_WANT_LIB_EXT1__=1 required
 #error -D__STDC_WANT_LIB_EXT1__=1 required
  ^~~~~

In file included from ../../lib/replace/replace.c:26:0:
../../lib/replace/replace.h:45:2: error: #error -D__STDC_WANT_LIB_EXT1__=1 required
 #error -D__STDC_WANT_LIB_EXT1__=1 required
  ^~~~~

Waf: Leaving directory `/root/tc-samba4x-netbsd4le/samba/bin/default'
Build failed
 -> task in 'LIBREPLACE_HOSTCC' failed with exit status 1: 
	{task 269871896900128: c strptime.c -> strptime.c.1.o}
['/usr/pkg/gcc7/bin/gcc', '-O', '-fcommon', '-fgnu89-inline', '-D_SAMBA_HOSTCC_', '-O', '-fcommon', '-fgnu89-inline', '-DSTATIC_LIBREPLACE_HOSTCC_MODULES=NULL', '-DSTATIC_LIBREPLACE_HOSTCC_MODULES_PROTO=extern void __LIBREPLACE_HOSTCC_dummy_module_proto(void)', '-Ilib/replace', '-I../../lib/replace', '-Iinclude/public', '-I../../include/public', '-Isource4', '-I../../source4', '-Ilib', '-I../../lib', '-Isource4/lib', '-I../../source4/lib', '-Isource4/include', '-I../../source4/include', '-Iinclude', '-I../../include', '-I.', '-I../..', '../../lib/replace/strptime.c', '-c', '-o/root/tc-samba4x-netbsd4le/samba/bin/default/lib/replace/strptime.c.1.o', '-D__GNUC_GNU_INLINE__']
 -> task in 'LIBREPLACE_HOSTCC' failed with exit status 1: 
	{task 269871896899744: c replace.c -> replace.c.1.o}
['/usr/pkg/gcc7/bin/gcc', '-O', '-fcommon', '-fgnu89-inline', '-D_SAMBA_HOSTCC_', '-O', '-fcommon', '-fgnu89-inline', '-DSTATIC_LIBREPLACE_HOSTCC_MODULES=NULL', '-DSTATIC_LIBREPLACE_HOSTCC_MODULES_PROTO=extern void __LIBREPLACE_HOSTCC_dummy_module_proto(void)', '-Ilib/replace', '-I../../lib/replace', '-Iinclude/public', '-I../../include/public', '-Isource4', '-I../../source4', '-Ilib', '-I../../lib', '-Isource4/lib', '-I../../source4/lib', '-Isource4/include', '-I../../source4/include', '-Iinclude', '-I../../include', '-I.', '-I../..', '../../lib/replace/replace.c', '-c', '-o/root/tc-samba4x-netbsd4le/samba/bin/default/lib/replace/replace.c.1.o', '-D__GNUC_GNU_INLINE__']

executed some commands :

export HOSTCFLAGS="-D__STDC_WANT_LIB_EXT1__=1"
export HOSTCPPFLAGS="-D__STDC_WANT_LIB_EXT1__=1"
rm -rf /root/tc-samba4x-netbsd4le/samba/bin/default
rm -rf /root/tc-samba4x-netbsd4le/samba/c4che
rm -f /root/tc-samba4x-netbsd4le/samba/.lock-wscript

recompiled same error.
in the file /root/tc-samba4x-netbsd4le/samba/lib/replace/replace.h
Near the very top, BEFORE any includes, added:

#ifndef __STDC_WANT_LIB_EXT1__
#define __STDC_WANT_LIB_EXT1__ 1
#endif

then

rm -rf /root/tc-samba4x-netbsd4le/samba/bin/default
rm -rf /root/tc-samba4x-netbsd4le/samba/c4che
rm -f /root/tc-samba4x-netbsd4le/samba/.lock-wscript

recompiled
result:

Samba 4.x build complete.
Log: /root/tc-earmv4-netbsd4/samba4xold.log
Regular binary: /root/tc-netbsd4le/sbin/smbd
Stripped binary: /root/tc-netbsd4le/sbin/smbd.stripped

[jamesyc]

Try running downloadsamba4xoldle.sh again. Or check your .env again. The build should be reproducible on a new VM with only copying .env to the folder, and then running

su
./downloadoldle.sh
./bootstrapoldle.sh
./downloadsamba4xoldle.sh
./samba4xoldle.sh

Note, I run the builds as root- not ideal but it works.

[tbex78]

Try running downloadsamba4xoldle.sh again. Or check your .env again. The build should be reproducible on a new VM with only copying .env to the folder, and then running

su ./downloadoldle.sh ./bootstrapoldle.sh ./downloadsamba4xoldle.sh ./samba4xoldle.sh

Note, I run the builds as root- not ideal but it works.

I started the process all over again from the very beginning
I can confirm you that without adding

#ifndef __STDC_WANT_LIB_EXT1__
#define __STDC_WANT_LIB_EXT1__ 1
#endif

Near the very top, BEFORE any includes in /root/tc-samba4x-netbsd4le/samba/lib/replace/replace.h

the build will fail.

Also those packages need to be installed to successfully compile the binaries:

pkgin install gcc7 gmake python27 python314 nettle pkgconf
ln -s /usr/pkg/bin/python2.7 /usr/pkg/bin/python
ln -s /usr/pkg/bin/python3.14 /usr/pkg/bin/python3

[jamesyc]

Ah, .env.example is stale

[tbex78]

Ah, .env.example is stale

I see you did a commit to update .env.example
There is an ip address hard coded in it:
TC_NETBSD7_HOST='root@192.168.1.217' shouldn't be TC_NETBSD7_HOST='root@timecapsule.local' instead like the two others?