Skip to content
This repository was archived by the owner on Jul 15, 2020. It is now read-only.
This repository was archived by the owner on Jul 15, 2020. It is now read-only.

Refactor Pzsvc-Exec to use CF Tasks for Auto-Scalability #205

@ssaniev

Description

@ssaniev

As a BF User, I need the BF application to scale so that I can reliably use the product even while many other users are using it; so that I may accomplish my mission in a timely manner when required.
Currently, Beachfront Pzsvc-ndwi-py (running Pzsvc-exec) scales linearly on a manual basis. That is, the application only scales when a developer or administrator updates the number of instances of the application. This is difficult to automate because the algorithm does not uniformly use system resources, so it can be hard to automatically detect when scaling is needed.
In order to implement proper auto-scalability, it is instead recommended to use Cloud Foundry Tasks. cf run-tasks. This can be used to create one-off containers in PCF, tied to the host application, that can run a single task before that container is reaped. Using this system, we can create individual tasks for each run of the detection algorithm, that will scale uniformly and linearly with available PCF resources. Using Tasks, we will not be required to manage any state, or perform any polling of resources. This makes it an incredibly desirable solution for BF Scaling.
Pzsvc-Exec needs to be reworked to use CF Tasks. Code refactoring should be kept to a minimum so as not to break any existing code. This is not intended to be a major refactor - just a rework to use CF Tasks.
The design will have the Taskworker (now renamed to Dispatcher) be the single process that runs on the Pzsvc-ndwi-py application. This process will poll Piazza for work and when found, will create a task that runs the Pzsvc-exec application (now renamed to Worker). This task will run, isolated, and send results to Piazza; at which point CF will clean it up. This allows us to have a single application for the algorithm that will scale with tasks. 

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions