Manage all content negociation on API services

[srosset81]

Problem

Turtle handling is broken for a long time (see #995). It's complicated to deal with several formats on the LdpService and TripleStoreService. Plus it's annoying to have to declare a contentType or accept param for every calls (see #542)

Proposal

We should make it a rule that Moleculer services only handle JSON-LD data, because JSON is a first-class citizen on Moleculer, and more generally on javascript.

This means it is the responsibility of API services (we have a new, very functional LdpApiService) or actions to transform data from/to Turtle, before handing it to Moleculer services as JSON-LD. This means the TripleStoreService will not need to worry about content negociation anymore.

This way we can clean up the LdpService and TripeStoreService while fixing the Turtle handling.

We will need to add tests for the Turtle format, as none exist at the moment.

[srosset81]

The main issue will be to find a good library to transform from/to JSON-LD, where performances are good. If we could directly use the correct JSON-LD context, that would be great.

There are some answers here:
digitalbazaar/jsonld.js#255
https://forum.solidproject.org/t/is-there-a-converter-between-json-ld-and-turtle-n3/1817/2

The most stable libraries all use the same Sink interface: https://rdf.js.org/#sink-interface. One solution could be to use https://github.com/rdfjs-base/parser-jsonld to parse JSON-LD to RDF.js quads, and then https://github.com/rdfjs-base/serializer-turtle to serialize RDF.js quads to Turtle. Haven't found a Turtle parser though...

https://github.com/rdfjs/N3.js is not a good candidate since it doesn't support JSON-LD.

[simonLouvet]

I don't have any objections, but I'm afraid it will greatly reduce performance if an application wants to communicate with the server in turtle mode.

[srosset81]

Is there any app, in the past 4 years, which has been using Turtle instead of JSON-LD ?
I guess not, since the Turtle format hasn't been working for a long time.

[simonLouvet]

I agree with the statement about usage.
I think we need to clarify/document this on semapps.org and in our communications to avoid any misunderstandings.
Semapps can handle formats other than json-ld at the expense of a loss of performance (to be quantified). This is not a prerequisite for moving forward, just a desire for transparency in the future.

[srosset81]

We could maybe use the new JSONExt serializer to transform Turtle to JSON-LD
https://github.com/moleculerjs/moleculer/releases/tag/v0.15.0-beta1

[Laurin-W]

We could create middleware functions for the api.addRoute registrations.
This can give as the flexibility of supporting transparent content-negotiation for different rdf content types and accept types (where desired).
Since we are going to create shapes for ldo object creation and validation, we can create middlewares for that use cases too. This way we will be able to created custom routes and controlled container mixins that come with type safety

I agree that we should keep an eye on performance (a general thing that we can improve on!). I'm not too worried though, my intution says that multiple round-trip-times between db and backend are by far the greater slow-down (happy to be disproven though :) ).

[Laurin-W]

We might even be able to use this to support dereferncing resources linked with certain predicates / join it with the DereferenceMixin.

[srosset81]

The conversion from Turtle/N-Triples is done in the LdpApiService. It's a very simple code. If we want, we can later move it to some route middlewares so that it can be used by other endpoints.