[WIP] Detect presence of flockfile

[mvastola]

@vitaut,

Wanted to continue the discussion from #4646 since that's a closed issue..

I think declaring functions without actually implementing them is really broken and should be fixed but I'm open to a PR to set FMT_USE_FLOCKFILE to 0 on newlib, provided that it's easy to detect.

@vitaut thanks for your understanding on this. I truly appreciate what you're saying, and it makes sense, but from what I've seen (and from the logistics I anticipate would be involved in not doing so), separating the declarations and implementations of low-level standard library functions in such a way that the former can't depend on the latter is SOP for bare-metal targets. (If you have any resources that suggest otherwise, I'm very open.)

A few clarifications:

How strongly wedded are you to using the presence of newlib specifically as the deciding factor in setting FMT_USE_FLOCKFILE?

IMHO, the ideal solution would be to do something along the lines of what I've done in this PR, which directly tests if flockfile can be linked, especially if you're now okay making a small addition to the CMake script. This involves as little fuss as possible and is 100% accurate, baring a mismatch in compile options between the compilation of this library and the software it is included in.

I'm realizing, however, that while your stated condition was that it be "easy to detect" and this implementation is certainly simple, it's possible you wanted me to just do this via a macro. I wanted to double check since you expressed reticence about doing this in CMake previously.

---

Unfortunately, if we restrict ourselves to macros and/or detecting newlib, the accuracy of testing if flockfile works/links drops significantly:

It seems there is an official macro to test the presence of newlib (aptly named __NEWLIB__), but as I mentioned previously, the use of newlib isn't dispositive of flockfile being unimplemented. Newlib is just one libc implementation popular with bare-metal targets (others include picolibc and LLVM libc). Further, some bare-metal platforms do, in fact, support filesystems (and, thus, flockfile).

If you're not convinced about using CMake to do the check, I'd be happy to address any concerns you have and figure out how to obviate them.

For sake of completeness, I've also included several macro-only solutions I've investigated here, with my assessment of each. As mentioned, they unfortunately all have (often major) downsides.

1. Testing for a bare-metal target with #if __STDC_HOSTED__ == 0.
   Downside: This depends on the user or platform toolchain passing -ffreestanding to the compiler, which isn't always done.
2. Testing for the absence of any common OSes. For example: #if !defined(_WIN32) && !defined(__linux__) && !defined(__APPLE__) && !defined(__unix__) ...
   Downside: this is brittle and would requires an exhaustive list of OSes with filesystems. Additionally, there are quite a few RTOS OSes where filesystem support is an optional feature.
3. Using macros make an educated guess as to if a target is bare-metal. For example: #if (defined(__ELF__) && !defined(__linux__) && !defined(__unix__)) || defined(__ARM_EABI__)
   Downside: While this would likely be more accurate than the previous two at identifying bare-metal targets, it still can't tell if its being used on a bare-metal target with a filesystem, and is likely to miss many bare-metal targets anyway.
4. Testing if #include <dirent.h> works. While newlib doesn't have any macros to indicate filesystem support, one thing it does have is a default sys/dirent.h containing #error directive. If <dirent.h> is included and sys/dirent.h is not masked by a file of the same name in the platform's headers containing a real implementation, a compile-time error occurs.
   Downside: this only works on newlib, plus there's no way to use this indicator without using CMake to assist, because the compiler will only know if it has an #error directive by #includeing it, at which point you've broken compilation.

[mvastola]

@vitaut, If it helps, here is a Dockerfile (like the one in #4646) showing this isn't limited to newllib. This demonstrates the identical problem with LLVM's libc implementation, based on code I am proposing we test-compile with this PR.

(Click to expand `Dockerfile` source)

# syntax=docker/dockerfile:1
FROM ubuntu:noble

# Lastest release of official "Arm Toolchain for Embedded"
ADD --unpack=true https://github.com/arm/arm-toolchain/releases/download/release-22.1.0-ATfE/ATfE-22.1.0-Linux-x86_64.tar.xz /usr/local
ENV TOOLCHAIN_DIR="/usr/local/ATfE-22.1.0-Linux-x86_64"
ENV TOOLCHAIN_BIN_DIR="${TOOLCHAIN_DIR}/bin"

ENV PATH="${TOOLCHAIN_BIN_DIR}:$PATH"
ENV SRC_DIR=/src

RUN mkdir -p "${SRC_DIR}"

ADD --chmod=0755 <<EOF "${SRC_DIR}/build-and-run.sh"
#!/bin/bash

set -euo pipefail

CFLAGS=(
  --std=c23
  --target=armv7-unknown-none-eabi -mfpu=none # generic defaults so the compiler can pick a set of runtime libraries
  --config=\${TOOLCHAIN_BIN_DIR}/llvmlibc.cfg -Wl,-Tllvmlibc.ld # use llvm libc\'s runtime libraries and linker script
  -static # no dynamic linking available because no dynamic linker present (nor fs to load shared libs from) on bare-metal
  -nostartfiles # by default, the llvm linker looks for (the non-existent) crt0.o; we need to specify our own because the real file is named like libcrt0.a
  -lcrt0-semihost -lsemihost # choose the libcrt0.a for semihosting, which provides support for stdin/out
)

set -x
cd "\${SRC_DIR}"

! test -f "test" || rm -f test

"${TOOLCHAIN_BIN_DIR}/clang" "\${CFLAGS[@]}" -o test test.c
exec ./test

EOF

ADD <<EOF "${SRC_DIR}/test.c"
#include <stdio.h>
#if __has_include(<newlib.h>)
#  include <newlib.h> // include __NEWLIB__ macro, if it exists
#endif

#if __NEWLIB__
#  warning "using newlib"
#endif
#if __LLVM_LIBC__
#  warning "using llvm libc"
#endif

int main() {
  flockfile(stdout);
  funlockfile(stdout);
  printf("Test complete!\n");
  return 0;
}
EOF

WORKDIR "${SRC_DIR}"
CMD ["./build-and-run.sh"]

To use, write the contents above to a file named Dockerfile in an empty directory and (within that directory) run:

docker build -t mvastola/fmt-test-flockfile-poc-llvm-libc . && docker run --rm mvastola/fmt-test-flockfile-poc-llvm-libc

Expected output is:

+ /usr/local/ATfE-22.1.0-Linux-x86_64/bin/clang --std=c23 --target=armv7-unknown-none-eabi -mfpu=none --config=/usr/local/ATfE-22.1.0-Linux-x86_64/bin/llvmlibc.cfg -Wl,-Tllvmlibc.ld -static -nostartfiles -lcrt0-semihost -lsemihost -o test test.c
test.c:10:4: warning: "using llvm libc" [-W#warnings]
10 | #  warning "using llvm libc"
|    ^
1 warning generated.
ld.lld: error: undefined symbol: flockfile
>>> referenced by test.c
>>>               /tmp/test-1e0caa.o:(main)

ld.lld: error: undefined symbol: funlockfile
>>> referenced by test.c
>>>               /tmp/test-1e0caa.o:(main)
clang: error: linker command failed with exit code 1 (use -v to see invocation)

[mvastola]

So, I think I was able to come up with the beginnings of a headers-only way to do this, if you feel strongly about approaching it that way.

Let me know if you prefer this approach. While this wouldn't be my preference, I wanted to put it out there as an alternative approach if you're not comfortable with the one in this PR. (If that's not an issue, no need to bother with this as it's a more complex and slightly more fraught solution.)

Basically this alternate approach turns f(un)lockfile info weak symbol references, thereby avoiding linker errors.

There are some downsides/implications, that I'll list here for your consideration:

• This solution is only coded for GCC and Clang, though it potentially can be tuned to work on any compiler/target combination supporting __attribute__((weak)). (We can potentially use __has_attribute but not sure what support is like across other compilers/versions or what is returned if the compiler generally supports the attribute but the binary file format doesn't.)
• This approach requires use of wrapper functions (which I've named fmt_f(un)lockfile) rather than using f(un)lockfile file directly, as seen in the diff I linked in this comment.
• This approach can't detect if f(un)lockfile exists at compile time (since obviously the linking step comes after compilation). This implementation adds fmt_f(un)lockfile functions that are no-ops if there is are linked-in f(un)lockfile methods for to call. For something like file_print_buffer there would need to be a runtime check to determine if the locking specialization of file_print_buffer should be used. (Fortunately both file_print_buffer specializations are subclasses of buffer with no other public functions, so such a change would be pretty seamless.)
• It seems you were making something of a habit of using a type parameter in lieu of FILE in format-inl.h. I'm not sure of the reasoning behind this (if it was purely stylistic, there's no issue), but this approach requires forward-declaring f(un)lockfile with extern "C" linkage, so a template isn't an option.
• At some point in the past, the f(un)lockfile methods defined in format-inl.h to wrap Windows' _(un)lock_file were behind a #ifdef _WIN32 (or similar) that was since removed. It's possible we need to place them behind such a conditional again to avoid a conflict for the two fallbacks.