Handle bindings for dynamically linked libraries

[SMFloris]

Working on a little something with raylib and c3 and the current bindings are great.

Bundling static libs and bindings together is pretty good in the matter of stable bindings, but not so great with dynamic libraries.
I think that, in order to improve both dynamic libraries and static libs we should probably consider improving the manifest file to be more descriptive. Thinking about it, here are my ideas:

• add a version string to the bundled libs: currently no idea what version the bundled static lib is
• a flag to whether to use or not to use the bundled static lib: if the flag is not set, then just search LD_LIBRARY_PATH. Maybe even the cli that pulls the bindings could use this flag in order to know whether or not to bother pulling in the static libs.

The suggestions above would improve packaging of the bindings in distros since it would be clear what the dependencies are.

[lerno]

Absolutely. Could you suggest some manifest.json properties, and maybe write up some examples?

[SMFloris]

So here's a basic example of what I think could work on every platform (with some tweaks).

The way I see it, we can either provide the static libs ourselves - which means we would need to compile each and every one of them for every platform, or we can provide the means for the user to statically compile them on their own.

I think it is quite ok to provide the static libs ourselves since it would improve the user experience of the package manager. It would be so easy to grab a dependency and use it!

{
  "provides": [
    {
      "name": "raylib",
      "version": "1.2.3",
      "source": {
        "url": "github.com/raylib/raylib/releases/release-1.2.3.tar.gz",
        "checksum": "sha256:abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234"
      },
      "bundled": {
        "static": true
      },
      "requires": {
        "c3c": "0.6.1"
      }
    }
  ],
  "targets": {
    "macos-x64": {
      ...
    },
    "macos-aarch64": {
      ...
    },
    "linux-x64": {
      ...
    },
    "windows-x64": {
      ...
    }
  }
}

Alas, it could prove quite challenging to provide static libs for every platform imaginable ... but it is not impossible.

I think that in this repo, we can leave only the bindings themselves. We can store the recipes and the compiled static libs somewhere else. In another repo, we can use Nix to provide a reproducible and easily upgradeable environment to write the recipes for compiling the static libs. The resulting static libs we can store in an archive/cdn/s3/etc.

Unfortunately, Nix only solves Linux (x64, aarch64) and MacOs(x64, aarch64); does not work for Windows. I haven't used Windows in years, but I presume we can use a separate build system for it.

So, to conclude, what I propose is:

1. Modify manifest.json format to better express versions/sources
2. Create a new repo (vendor-static-recipes), containing recipes of static libs compilation powered by Nix. This new repo would "listen" for changes in this one and grab the manifests file every time it changes, plug versions/sources in the nix file recipes, build the static libs and create some artefacts and store them somewhere
3. The "c3c package manager" cli when invoked for example with the --static flag (i.e. c3c pm import vendor/raylib --static) would grab the static lib from wherever we stored it. In a sense, this repo, becomes sort of like a release channel. We could even have tags based on the version of c3c, that way we sort of say that for this particular version of c3c --- these are the bindings that work out of the box.

And of course, I'm open to collaborate together on this in order to bring the world the great gift that is C3.

[lerno]

Hmm.. this can be done in several ways. I'm considering ways to declare both static and dynamic libraries in the same manifest.json, but it seems like one of those things that work nicely in theory but isn't great in practice – because you need to know all the switches. So what if we did something that didn't need anything:

For a library that has both static and dynamic variants, release something like raylib.c3l (static) and raylib-dynamic.c3l (dynamic). It's hard to get that wrong. For the dynamic and possibly also the static, we can provide the "source" tag you suggest but only as a suggestion. So something like c3c lib-url raylib-dynamic.c3l would show that URL but not download it. It's then up to the user to kind of manage the way they download their dynamic libraries. A general package manager could use the information to get the dependency too, if it chooses to. You could also do c3c lib-urls <target> on a project, to get a (recursively resolved) list of all libraries and their urls.

All of this in an easy parseable format, so say you use the dynamic raylib, and you use curl. Both of them list dynamic library dependencies:

Library: raylib.c3l
URL: github.com/raylib/raylib/releases/release-1.2.3.tar.gz
URL: github.com/some_other/library_needed.tar.gz
Library: curl.c3l
URL: NOT PROVIDED

You could then also possibly have some instructions to each .c3l: c3c lib-readme raylib-dynamic.c3l

The C3 curl wrapper provides access to curl. For most targets you need to already have 
curl installed.
On MacOS you can XXXXXXX, on Linux you can XXXXXX

What do you think?

[lerno]

This is a good start. I want to make it as seamless as possible and bundling the libraries have proven a challenge.