richard-churchman
5f6bbe45f7
Major bug fixes relating to the Redis implementation in Reprocessing and Search Key Cache were identified and fixed as part of the rekeying development. Included functionality to stop the Exhaustive training. Fixed the Unsupervised learning functionality in Exhaustive Training. Embedded the Mock Data CSV data into a class to make it compatible with Docker without needing to modify the Dockerfile (i.e, build). Included a new table that stores training data to enhance support capabilities for Exhaustive. Reintroduced and validated the Multitenancy functionality. Postgres is now fully deprecated in place of Redis for cache, and all implementations of the Interfaces for Postgres have been removed. No migrations exist to support this deprecation, and Postgres as a cache is now wholly unsupported. Removed the Indexing background thread, as Redis does not require it, given that it is a key-value pair database. Created a RedisCommandFlag Environment Variable to support Fire and Forget write to Redis in Invocation. Implemented functionality to Export, Peek and Download configuration environments using a secure, encrypted scheme. Created a permission to access the functionality via the user interface. Extensive database-based logging is included to provide for a comprehensive audit. Created functionality to Import the encrypted Export file inside a database transaction to ensure consistency. As Docker is now the primary deployment model, log4net instantiation has been moved to an approach that primarily uses programmatic instantiation of the ConsoleLogger and RollingFileAppender given Environment Variables. Implemented functionality to support remote \ mounted Log4Net XML Serialisation files for more advanced log4net configurations. Reintroduced access to counters via the user interface and created a permission. Took the opportunity to implement ReSharper linter suggestions. Disabled the Lock button across the whole application, with that functionality being reserved for future use. It is intended that locking happens programmatically on the basis of dependencies in future commits. Fixed several significant end-use bugs which became apparent during testing of the refactored code relating to case management and visualisations. Fixed certain other minor bugs that came up during testing. Brought documentation up to date for this commit. Brought the Docker Compose file up to date for this commit. Removed Redundant Environment Variables for this commit. Made the README file less verbose and unnecessary.
3.8 KiB
layout, title, nav_order, parent, grand_parent
| layout | title | nav_order | parent | grand_parent |
|---|---|---|---|---|
| default | Reprocessing Instance | 3 | Reprocessing | Configuration |
🚀Speed up implementation with hands-on, face-to-face training from the developer.
Reprocessing Instance
To create a reprocessing request, ensure that the Reprocessing Filter created beforehand is available:
Scroll down and locate the Reprocess button:
By clicking the Reprocess button, an entry will be made in the database requesting the engine perform reprocessing as per the filter specification.
Only one reprocessing request may exist at any given point in time, and a new request cannot be created until any unprocessed requests move to a completed state.
Click the Reprocess button:
An entry will be created showing the reprocessing job request:
It is possible to Delete - which will also stop - a reprocessing request by clicking the Delete button:
The reprocessing job has the following processing steps:
| Name | Description |
|---|---|
| Not Allocated | The request has not been picked up by the reprocessing engine. |
| Allocated | The request has been picked up by the reprocessing engine. |
| Initial Count | The initial count has been completed, to establish created date range and universe of records. |
| Processing | The request has begun processing data. |
| Completed | The request has completed. |
The reprocessing instance table is available and updated as reprocessing takes place:
| Name | Description |
|---|---|
| Status | The current status of the reprocessing job. |
| Created Date | The date and time the reprocessing request was created. |
| Reference Date | The current date and time being processed in that moment. This is the reference date specified in the model. |
| Available Count | The total number of records in the universe as brought back during the initial counts. |
| Sampled Count | The total number of records that have been sampled for the purposes of reprocessing. |
| Matched Count | The total number of records that have been matched (after being sampled) for the purposes of reprocessing. |
| Processed Count | The total number of records reprocessed so far. |
| Completed Date | The date and time that the job completed. |
| Error Count | The number of records in error observed during reprocessing. |
Refreshing the grid by navigating again to the Reprocessing Filter (Models >> Reprocessing, clicking on the Reprocessing Filter and scrolling down to the instances gird):
It can be seen that the reprocessing job has completed (which will allow new reprocessing jobs to be created for this reprocessing filter by simply clicking the Reprocess button).