FREME cloud progress

[jnehring]

I have implemented the FREME cloud basic services with these basic services:

• eureka-server starts the service discovery server. needs to be included in a single freme-package all over the cluster, right now in the broker.
• eureka-client adds the eureka service discovery client to a freme package. needs to be included in every freme package in the cluster, which is right now broker and worker.
• load-balancer is put in the broker and distributes requests to the worker nodes. The load balancer discovers the worker nodes via eureka.

Further I have created two freme packages

• cloud-worker will be have many instances that do the enrichment. It uses DKT NER for enrichment.
• cloud-broker performs the load balancing.

I wanted to test the new basic services on the Google Compute engine but it did not work out because it is quiet complicated to configure the Google Compute engine. So my next approach is to try it out locally using Docker images. When it works locally then I can use any cloud environment that runs docker.

A good side effect of this work is that FREME will be run on docker which maybe easens the installation of FREME.

I will use this issue to document the progress of this topic.

[jnehring]

I dockerized both freme-cloud-worker and freme-cloud-broker. I started the broker container once and the worker container twice. They workers use the e-capitalization service. When I do multiple API requests to e-capitalization service of the load balancer then I get the correct output and the log files indicate that the requests are served by both workers round-robin.

Next steps

• The configuration is very cumbersome because the full configuration is part of the docker image. Every configuration change requires a rebuild of the docker images. Therefore I want to integrate the config server in the setup so configuration changes are easier.
• With the config server the setup has three node types: The configuration / eureka server node, the broker and the worker. This requires an additional freme package and reconfiguration of existing broker package.

[jnehring]

It did not work out to have a server that contains both configuration server and eureka node. But I think it is not necessary to provide these as basic services. One can always use these servers independently from the others.

This activity is delayed until september.