Aedev simple

[asia281]

Towards highly configurable AE [UNFINISHED]

The major goal is to build maintainable and readable standard for model development throught the project with hight configurability and readable data processing pipeline. The pull includes dense model oriented modules and provides (hopefully generalizable) standard of model decomposition into composable components.

In utils add:

• utils.common_types for handling tailored datatypes and strustures used throught different project files including:

• • Batch - defining batch loaded and processed by dense model components. Aligned with the dataloader batch output.
• • StructuredForwardOutput - output of the dense model components containing batch and losses returned by the model component
• • ConfigStructure - specification of the config structure bound with class taking config Namespace variable as an input. Used to validate the input config structure to filter out config errors at the initialization stage. Each class should have corresponding variable containing ConfigStructure object.

• utils.config for managing configuration files and variables including:

• • validate_config_structure - used for validating config variable passed to the model component constructor (related with ConfigStructure)
• • functions for loading and parsing the yaml configuration files

In models add:

• models.blocks - a module with main component StackedBlocks input, output as well as intermediate dimensions might be specified to define a dense block (e.g dense encoder). Composed of lower level Blocks for which activation, normalization and dropout might be specified in any combination and with additional kwargs all in configuration file. The code should be eaisily extendible to incorporate other modules composing a Block. It should allow to eaisiliy define dense encoder or decoder of typical structure with decent level of configurability based solely on the passed config file.
• models.latent - a module for latent processing including:
• • creation of random variables: gaussian and gaussian mixture with differentiable sampling for gm upsent in torch.distributions in the desired format
• • GaussianPosterior - a module adding gaussian posterior with entropy loss calculation on top of inputted batch
• • _GM - abstract parent class for handling gaussian mixture with trainable parameters
• • GaussianMixturePriorNLL(_GM) - a module calculating nll in the trainable GM prior in a conditionable manner with a possibility of partly labeled data (-1 denotes lack of component label)
• • 'FuzzyClustering(_GM)' - sort of simplification of GaussianMixturePriorNLL(_GM). No conditioning and additional constraint on spread of compnents is added.
• • LatentConstraint - explicit constraint on latent as in RAE
• • base code for WAE like mmd calculation - not implemented as it might be hard to work with it in a conditional case e.g. when each sample comes from different prior, differentiable sampling from gm was added
• models.loss - utils for managing losses in display and calculating constaints and loss aggregation -> vae vs iwae
• models.misc - data aggregation in the data processing with 'AggregateDataAdapter' added to the chain model as a model component it also allows to rename some batch elements - it should allow option "none" for renaming or sthg

Add base for models.chain.ChainModel a highly configurable nn.Module childred allowing for specifying chains of batch processing operations e.g. dense blocks + vae posterior + vae prior + dense block + latent fuzzy clustering + dense block + vae prior + dense block + wae constraint etc. (exaggerated) meaning a hierarchical model (in the simplest sense) might be build seamlessy) as well as any dense encoder decoder model with vae, iwae with gaussian posteriors and gaussian mixture parameterised posterior might be build and multiple extensions might be added to the codebase in a readable, maintainable (hopefully) and standardised format.

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[asia281]

Split this test into 3 tests.

[asia281]

Split into 2 tests.