richard-churchman
5f6bbe45f7
Major bug fixes relating to the Redis implementation in Reprocessing and Search Key Cache were identified and fixed as part of the rekeying development. Included functionality to stop the Exhaustive training. Fixed the Unsupervised learning functionality in Exhaustive Training. Embedded the Mock Data CSV data into a class to make it compatible with Docker without needing to modify the Dockerfile (i.e, build). Included a new table that stores training data to enhance support capabilities for Exhaustive. Reintroduced and validated the Multitenancy functionality. Postgres is now fully deprecated in place of Redis for cache, and all implementations of the Interfaces for Postgres have been removed. No migrations exist to support this deprecation, and Postgres as a cache is now wholly unsupported. Removed the Indexing background thread, as Redis does not require it, given that it is a key-value pair database. Created a RedisCommandFlag Environment Variable to support Fire and Forget write to Redis in Invocation. Implemented functionality to Export, Peek and Download configuration environments using a secure, encrypted scheme. Created a permission to access the functionality via the user interface. Extensive database-based logging is included to provide for a comprehensive audit. Created functionality to Import the encrypted Export file inside a database transaction to ensure consistency. As Docker is now the primary deployment model, log4net instantiation has been moved to an approach that primarily uses programmatic instantiation of the ConsoleLogger and RollingFileAppender given Environment Variables. Implemented functionality to support remote \ mounted Log4Net XML Serialisation files for more advanced log4net configurations. Reintroduced access to counters via the user interface and created a permission. Took the opportunity to implement ReSharper linter suggestions. Disabled the Lock button across the whole application, with that functionality being reserved for future use. It is intended that locking happens programmatically on the basis of dependencies in future commits. Fixed several significant end-use bugs which became apparent during testing of the refactored code relating to case management and visualisations. Fixed certain other minor bugs that came up during testing. Brought documentation up to date for this commit. Brought the Docker Compose file up to date for this commit. Removed Redundant Environment Variables for this commit. Made the README file less verbose and unnecessary.
4.4 KiB
layout, title, nav_order, parent
| layout | title | nav_order | parent |
|---|---|---|---|
| default | Prerequisites | 1 | Getting Started |
🚀Speed up implementation with hands-on, face-to-face training from the developer.
Prerequisites
The README file in the head of the project provides for a Quickstart that assumes some ability to provision servers, databases and operating system dependencies:
git clone https://github.com/jube-home/aml-fraud-transaction-monitoring
cd aml-fraud-transaction-monitoring/Jube.App
export ConnectionString="Host=<host>;Port=<port>;Database=<defaultdb>;Username=<username>;Password=<password>;Pooling=true;Minimum Pool Size=0;Maximum Pool Size=100;SSL Mode=Require;Trust Server Certificate=true;"
export RedisConnectionString="<host>:<port>,password=<password>"
export ASPNETCORE_URLS="https://localhost:5001"
export JWTKey="IMPORTANT:_ChangeThisKey_~%pvif3KRo!3Mkm1oMC50TvAPi%{mUt<9sBm>DPjGZyfYYWssseVrNUqLQE}mz{L_UsingThisKeyIsDangerous"
dotnet run
This section of the documentation makes no assumption about technical ability and provides a step by step approach to getting started with Jube on DigitalOcean servers. DigitalOcean is an enterprise hosting provide much like AWS, Azure or Google Cloud Platform. Digital DigitalOcean has a much simpler and focussed offer, specifically very well priced high performance Virtual Machines running commodity Linux and Managed PostgreSQL, and when taken together greatly simplify the infrastructure and reduce cost. DigitalOcean also offer a free $200 credit on being referred from Jube documentation, which is more than enough to complete testing and proof of concept.
This documentation is intended to be introductory, and while Redis is introduced, it is introdduced for completeness, it is highly elective for the purpose of this section of the documentation (insofar as it serves as a cloud introduction).
Jube has the following prerequisites:
- .Net 9 Runtime.
- Postgres database version 13 onwards (tested on 15.4 but no significant database development to cause a breaking change).
- Optional: Redis version 6 or above (it probably works fine on earlier versions, as the command used are basic. RESP wire compatible implies that it is possible to use KeyDB, DragonflyDB, Garnet or any RESP compliant wire protocol database). In DigitalOcean Managed Redis is not without some integration complexity given TLS, and is presented here for completeness. Managed Redis at DigitalOcean is subject to being comfortable with some advanced administrative concepts described as follows: https://www.digitalocean.com/community/tutorials/how-to-connect-to-managed-redis-over-tls-with-stunnel-and-redis-cli
Create an account on DigitalOcean. Once the account is created and navigate to the page as follows:
For security, there is no means to pass configuration values via anything other than Environment Variables, and the contents of those Environment Variables are never - ever - stored by Jube (which is something the CodeQL security scanner tests for).
There are sensitive cryptographic values that need to be included at startup. At a minimum the JWTKey value is required:
export JWTKey="IMPORTANT:_ChangeThisKey_~%pvif3KRo!3Mkm1oMC50TvAPi%{mUt<9sBm>DPjGZyfYYWssseVrNUqLQE}mz{L_UsingThisKeyIsDangerous"
The JWTKey value is used to encrypt access tokens providing for API authentication, and therefore user interface authentication.
While outside of the scope of this installation documentation, other sensitive variables, while optional, are strongly suggested:
export PasswordHashingKey="IMPORTANT:_ChangeThisKey_~%pvif3KRo!3Mkm1oMC50TvAPi%{mUt<9sBm>DPjGZyfYYWssseVrNUqLQE}mz{L_UsingThisKeyIsDangerous"
It is imperative that the keys be changed from their defaults and kept safe in appropriate storage. Jube will not start if the keys above are used.
Creating a new project if necessary:
The installation assumes three servers as follows:
- General Purpose Droplet (Virtual Machine) 8GB RAM / 2vCPU.
- Managed PostgreSQL Database 4GB RAM / 2vCPU.
- Optional: Managed Redis Database 4GB RAM / 2vCPU.
Putting aside high availability and storage, this configuration will be more than adequate for the purpose of testing and proof of concept.