richard-churchman
5f6bbe45f7
Major bug fixes relating to the Redis implementation in Reprocessing and Search Key Cache were identified and fixed as part of the rekeying development. Included functionality to stop the Exhaustive training. Fixed the Unsupervised learning functionality in Exhaustive Training. Embedded the Mock Data CSV data into a class to make it compatible with Docker without needing to modify the Dockerfile (i.e, build). Included a new table that stores training data to enhance support capabilities for Exhaustive. Reintroduced and validated the Multitenancy functionality. Postgres is now fully deprecated in place of Redis for cache, and all implementations of the Interfaces for Postgres have been removed. No migrations exist to support this deprecation, and Postgres as a cache is now wholly unsupported. Removed the Indexing background thread, as Redis does not require it, given that it is a key-value pair database. Created a RedisCommandFlag Environment Variable to support Fire and Forget write to Redis in Invocation. Implemented functionality to Export, Peek and Download configuration environments using a secure, encrypted scheme. Created a permission to access the functionality via the user interface. Extensive database-based logging is included to provide for a comprehensive audit. Created functionality to Import the encrypted Export file inside a database transaction to ensure consistency. As Docker is now the primary deployment model, log4net instantiation has been moved to an approach that primarily uses programmatic instantiation of the ConsoleLogger and RollingFileAppender given Environment Variables. Implemented functionality to support remote \ mounted Log4Net XML Serialisation files for more advanced log4net configurations. Reintroduced access to counters via the user interface and created a permission. Took the opportunity to implement ReSharper linter suggestions. Disabled the Lock button across the whole application, with that functionality being reserved for future use. It is intended that locking happens programmatically on the basis of dependencies in future commits. Fixed several significant end-use bugs which became apparent during testing of the refactored code relating to case management and visualisations. Fixed certain other minor bugs that came up during testing. Brought documentation up to date for this commit. Brought the Docker Compose file up to date for this commit. Removed Redundant Environment Variables for this commit. Made the README file less verbose and unnecessary.
5.1 KiB
layout, title, nav_order, parent, grand_parent
| layout | title | nav_order | parent | grand_parent |
|---|---|---|---|---|
| default | Exhaustive Adaptation Training Promoted Model Performance | 4 | Exhaustive Adaptation | Configuration |
🚀Speed up implementation with hands-on, face-to-face training from the developer.
Exhaustive Adaptation Training Promoted Model Performance
Model performance is expressed as a score. The score is the mean of percentage classification accuracy and the Spearman Rank Correlation. There are other means to assess overall model performance however. Model promotion statistics are computed to make available more comprehensive performance statistics via the Promoted Model Performance tab:
Clicking on the Promoted Performance Model tab will expose several performance measures:
The performance measures are as follows:
| Performance Measure | Description |
|---|---|
| Errors Histogram | During the testing phase of the training process, the errors made between the predicated value and actual value plotted as a histogram. |
| Errors Histogram | During the testing phase of the training process, the errors made between the predicated value and actual value plotted as a histogram. |
| Errors Histogram | During the testing phase of the training process, the errors made between the predicated value and actual value plotted as a scatter plot. |
| Confusion Matrix | During the testing phase of the training process, the classification predicted vs actual classification organised into a table with all possible states being reflected (true positive, true negative, false positive and false negative). |
| Receiver Optimisation Curve | A curve that compares false positive rates to true positive rates given different thresholds of classification being applied incrementally, stepped low to high thresholds increasing by a value specified in the Environment Variables. |
The Promoted Models tab keeps a journal of all promoted models:
Clicking on the tab exposes more details about the models that have been promoted, expressing the same as that in the Learning Curve:
The table is ordered from the best performing model to the weakest, then on topology complexity (the number of weights in the topology) on the very unlikely total deadlock in model performance.
It can be noted that the grid is hierarchical and can be expanded upon:
Expanding upon the grid exposes the selected input variables variables that have been included in the model:
The variables are presented in the order of most sensitive to least sensitive. The nature of exhaustive exploration is such that all variables will provide for sensitivity, as poorly generalised models will have been discarded by the training algorithm.
The variable statistics are included and contrast to prescription statistics. The prescription statistics are summary statistics having been performed as a consequence of Monte Carlo Model Simulation yielding a positive classification:
In the above example, it can be observed that the sampled data suggests 0.8 Abstraction.CountTransactionsDecline (this is of course 1 in practice), whereas positive classified by the model suggests a Abstraction.CountTransactionsDecline of closer to 6, where the inference is that given positive, Abstraction.CountTransactionsDecline tends to be higher. Prescriptions are to provide some explanatory value to models, given that Neural Networks are often felt to be less explainable.
It is possible to reject or deactivate a model via the switch in each row of the grid:
Selecting the switch to false will mean that this model is no longer considered as promoted:
Upon deactivation, and upon clicking the refresh button, the model performance will no longer be available in the Promoted Model Performance tab nor Promoted Model Testing tab, instead, the next highest performing model will be presented:
