Add a new DragonFlyBSD build host

[wilzbach]

Citing from @dkgroot's email:

Hi Guys,
I have been working on the DragonFly for a while, and finally got to
the point to submit the Pull Request (and already getting the first
couple approved). To make sure these change will keep working in the
future, it might be a good idea to get some Continues Integration (CI)
in place. I do know the PR is still open and under consideration.

I gave travis-CI a try, which does work, but times out during
unittesting (50+ minutes :-),
(https://travis-ci.org/dkgroot/dmd_dragonflybsd/builds/314543739)).
Without native support for BSD and very slow qemu-system, this look
like a dead end. It could be made to work, if i could save
intermediate results somewhere.

Which CI would you guys like to see first ? If travis is a must have,
i am sure i can make it work, but will require quite a bit of work.
Circle CI might be faster / easier (haven't tried). As FreeBSD and
DragonFlyBSD are pretty close (setup/library/packaging wise), it
should not be a big deal adapting the FreeBSD instances that you
already have (for example on auto-tester and/or Jenkins).

I currently use a very simple compile.sh script
(https://github.com/dkgroot/dragonflybsd_dmd_port) to bootstrap
dmd/druntime and phobos. and then build master/dub/tools/etc from
that.

If you guys can give me a nudge in the right direction, that would be great.

Regards,
Diederik de Groot

'Master' Pull requests:

• Add DragonFlyBSD support for dmd dlang/dmd#7463
• Add DragonFlyBSD support for druntime dlang/druntime#1999
• Add DragonFlyBSD support : src/core/sys/posix related changes dlang/druntime#2001
• Add DragonFlyBSD support : Split PR : src/core/sys/dragonflybsd related changes dlang/druntime#2002
• Add DragonFlyBSD support for phobos dlang/phobos#5941

Q: How difficult would it be to add a new machine to the build hosts? How much maintenance would it require?

CC @yshui @joakim-noah as they are working on Musl support for D

[braddr]

Adding machines, and even new platforms, to the build fleet is fairly easy, from a software standpoint.

However, there's a big open issue that's tied to every new platform and that's how well supported is it going to be? Right now everything is at the same tier of support. Is the release and project management team ready to support dragonflybsd at the same level as linux? Who's supplying the hardware necessary to make that viable (at least 2 full time build hosts)?

[braddr]

Also, the auto-tester doesn't really support multi-stage boot straps. A working (preferably 2.068) compiler needs to exist first to be the HOST_DC.

[joakim-noah]

DragonFly is such a niche platform that I'm not sure it's worth going to much trouble to add a CI for it. I don't have a CI set up for Android anywhere, though I was looking into Circle CI support again yesterday (it seems like such a PITA to run command-line tests that I set it aside again).

That said, I don't know how much trouble it is to add this, so it might be worth it if @dkgroot has the hardware and time to make sure this all goes through.

[dkgroot]

@braddr I do have a working v2.067.1 version of the compiler. Providing a 2.068 version should not be a problem (tiny bit of porting required). I could provide the same type of zip file that is normally used for a non boostrap build.

Providing 2 build hosts is going to be a little bit problematic for me personally, But there are some open source hosting projects out there that might be willing to help (for example osuosl.org). Another option would be running dragonfly in a FreeBSD jail (if you build environment can deal with that).

If we do contact this oss hosting providers, they might provide more hardware for other builds/projects as well (including power8 ;-))

[dkgroot]

Btw I already create these two sample ci's on semaphoreci as a sort of proof of concept: https://semaphoreci.com/dkgroot

Note: they run dragonflybsd inside qemu/kvm on top of the semaphoreci VM so this is not optimal. It does however document the whole os installation (from latest iso). And does include building the bootstrap compiler + latest master.

@joakim-noah providing hardware would be an issue. Time however is not really problematic.

Thanks for looking into this guys, much appreciated.

[dkgroot]

There is another way to conserve resources. Instead of emulating a full system we could use cross compilation and run the unittests via qemu user mode directly. This would allow a single x86_86 machine to run multiple compilations and unittests in parallel, without having the burden of running multiple vm's (CPU/memory bound). This is how chrome does most of its testing, see: https://www.chromium.org/chromium-os/testing/qemu-unittests. This also covers running tests for non-native architectures like arm/mips etc.

I am not sure if dmd can easily be cross-compiled, but ldc and gdc sure can.

[jacob-carlborg]

I am not sure if dmd can easily be cross-compiled

Unfortunately it cannot. Only 32bit <-> 64bit of the same platform.

[braddr]

Is anything going to happen here? This really isn't about the tester but about supported platforms for D. Get buy-in from the D leadership and release managers that this new platform should be officially supported, get at least 2 machines to support the build, and then re-open this issue. Without both, this can't go anywhere.