richard-churchman
9bad89cd29
This commit consolidates extensive refactoring across the engine, orchestrators, background tasks, JSON pipelines, caching layers, messaging, and infrastructure. Major improvements include async correctness, cancellation propagation, architectural restructuring, performance optimizations, parser and engine enhancements, UI updates, deployment improvements, and critical bug fixes. Asynchronous, Cancellation, and Concurrency Improvements: * Converted all repository and controller methods to fully asynchronous implementations; removed remaining synchronous database calls. * Introduced centralized cancellation via a `CancellationTokenProvider` in DI and propagated tokens through database, engine, cache, and background processes. * Implemented graceful SIGTERM handling with coordinated shutdown. Most tasks are subject to immediate cancellation, while: - Draining AMQP consumers - Draining Kestrel in-flight requests - Draining async Archive writes - Draining async Case Creation processing * Added middleware to track active requests and emit `Connection: close` during shutdown. * Enhanced exception handling: - Cancellation exceptions logged as INFO - Improved logging for async call failures * Added `/ready` endpoint: - Returns 200 if the engine is ready, 503 otherwise - Integrated readiness signaling from TaskController and ActivationWatcher * Fixed concurrency/timing issues, including LRU cache updates using a semaphore instead of locks. * Ensured proper propagation of cancellation tokens across async call stacks, except where drainage is required. Engine, Orchestrator, and Pipeline Refactoring: * Rebuilt Invoke Orchestrator using a Context–Strategy–Pipeline hybrid pattern. * Extracted JSON parsing and database record parsing into dedicated helper classes. * Standardized instance object state management using a centralized Context object; removed private mutable state from orchestrators and engine. * Reorganized namespaces and moved background tasks to dedicated classes. * Removed god-class, organic-growth code and streamlined dependency flow; dependencies are now passed via Context only. * Added reference date integrity checks: - Prevent processing of future `ReferenceDate` values to prevent purge of records through bad data. - Updating of reference date ignored for records older than the latest entry to prevent invalid purges. * Introduced nested classes for large models and properties to improve readability. Moved very large property sets to nested classes for better context in engine pipelines. * Implemented rich extension functions library for parser and rule coder functionality: - Parser Scans new extensions assembly to add parser security tokens, enabling centralized development of extension functions to enhance rule capabilities. - Documented new tokens under Rule Compilation Tokens and Extensions. * Simplified `Invoke` data extraction logic via helper extension methods. * Centralized Context creation for JSON settings and contract resolvers. * Dropped legacy Response Payload approach, pending improved serialization to support pruning response payload. * Renamed `Report Keys` to `Archive Keys` for consistency with POCOs and database entities. * Reorganized engine files and namespaces to reflect new software architecture. * Implemented a bytes limit for payloads received over the invoke controller, controlled via the `MaxInvokeControllerRequestBytes` environment variable. Cache, Redis, TTL, and Background Task Enhancements: * Replaced Postgres `NOTIFY` in callbacks with Redis-based signaling given Postgres should not be interacted with during real-time flows — only Redis. * Improved callback timeout/removal logic and reduced initial invocation payload sizes. * Added environment variables for SearchKey cache job interval control: `SearchKeyCacheServerIntervalType` and `SearchKeyCacheServerIntervalValue`. * Refactored cache purge with observability tables and supporting migrations to enhance confidence in high-risk processing. * Converted TTL counters to `long` type with corresponding storage updates in the `ArchiveKeys` table. * Ensured only one TTL increment per invocation, irrespective of activation rules. * Updated Redis background scans from `HGETALL` to incremental `HSCAN` to prevent deadlocks and improve performance. - Removed obsolete payload-first cache tracking. * Improved initial LRU cache fill visibility via TaskController integration. * Refactored repository access in the Cache Service and audited DbContext disposal to prevent connection leaks. * Fixed counter metrics in the LRU cache. * Batched SortedSetRemoveAsync and HashDeleteAsync as it was causing long operations in Redis. AMQP, Threading, and Background Workflow Improvements: * Refactored AMQP messaging to resolve deadlocks in async consumers. * Moved RabbitMQ channel creation to consumer tasks and improved heartbeat handling to avoid deadlocks given `IModel` is not thread-safe. * Offloaded expensive case-creation processing to scalable background threads: - Reduces pressure on main transaction threads - Eliminates blocking in Archive storage and reduces memory leak risk (backlogged case creation has a much smaller footprint than backlogged Archive payload) - Threads controlled via the `CaseCreationThreads` environment variable * Introduced `TaskController` to manage all background (non-transactional) tasks. TaskController allows visibility of all background tasks. * Centralized cancellation and graceful shutdown across engine, background tasks, and web host. * On SIGTERM, if a task is not terminated within two seconds — perhaps due to drainage — it is written to console to give feedback on blocking tasks. * Implemented heartbeats in the connection to handle brittle connections and unforgiving .NET RabbitMQ clients, controlled by the `AMQPHeartbeatSeconds` environment variable. JSON, Parser, and Serialization Fixes: * Updated JSON settings, contract resolvers, and DI wiring. * Reduced payload response size on async HTTP invocation to only return the EntityAnalysisModelInstanceEntry GUID for callback, notwithstanding head check and parse still taking place. * Added `DBNull` checks for database led reprocessing deserialization. * Ensured the DictionaryNoBoxing converter is properly added to serialization settings to parse dates using the round-trip format. Performance, Stability, and General Cleanup: * Updated PooledDictionary to fix hash collisions and improve initial sizing. * Return null from PooledDictionary.cs instead of exception. * Ensured DB connections are disposed properly during model sync to prevent pool starvation in the new patterns. * Refactored SMTP client usage and added authentication/connection validations. * Removed unnecessary AutoMapper list mappings and streamlined repository configurations. * Renamed illogical methods and variables for clarity, as came across them. * Applied `ConfigureAwait` consistently across all projects. * Added `Async` suffix to async method names. * Replaced async unsupported `out` variables with tuples UI, Infrastructure, and Deployment Updates: * Updated UI to show case-creation, missing tagging and notification queue balance counters. * Removed obsolete test Blazor files (this project will go Angular). * Added production GC environment variables to `docker-compose.yml`. * Added `Jube.HealthCheck.csproj` for lightweight HTTP health checks in container (avoids installing `curl` as root). * Updated `docker-compose.yml` to bind port `5001` for job process readiness checks, as was already the case with the other process. * Implemented frame-busting on the login page to prevent the authentication page from being loaded in a frame after expiration. Critical Bug Fixes: * Fixed transaction buffer flushes on commit. * Fixed bulk write attempts on empty buffers. Miscellaneous Improvements: * Moved engine and model manager into a unified hybrid context with strategy & pipeline patterns. * Shifted thread pool configuration to startup pipeline. * Reorganized files and namespaces to align with the new architecture. * Improved observability and versioning for `Archive` and `ArchiveKey` tables to support reprocessing. * Dropped unused/complex payload-first cache tracking. * Many additional small bug fixes and refinements discovered during the grand refactor. * The Sandpit had some telemetry to a Chatbot and Google Analytics. Although it was disabled by default — and the Sandpit was intended for Jube use only — it should not have been in the public build. * Updated UI to display case-creation counters and missing tagging/notification fields. * Added a filter to remove instances from the synchronization page that have not sent a heartbeat. * Fixed invoke counter and removed the condition that counters are only written on invoke for more consistent system observability. * Updated the parser to support dates in inline functions, which was previously broken, which meant that dates not available. * Improved thread pool startup parameters to be a factor of two times logical cores for worker threads and four time logical cores for io threads. * Set command timout to zero for archive persistence given sensitivity. In time will create a dead letter archive for total failure, as on failure the records dropped. **Notes:** Functionality remains substantially the same where intended; most changes focus on stability, maintainability, async correctness, and operational robustness.
5.0 KiB
layout, title, nav_order, parent, grand_parent
| layout | title | nav_order | parent | grand_parent |
|---|---|---|---|---|
| default | HTTP API Asynchronous Model Invocation | 1 | API | Concepts |
🚀Speed up implementation with hands-on, face-to-face training from the developer.
HTTP API Asynchronous Model Invocation
Most of the procedures in this documentation makes use of synchronous HTTP requests in which the request is blocked for the client unitil absolute conclusion of the processing, notwithstanding a significant amount of asynchronous processing taking place in the processing to ensure reasonable response time by eliminating blocking waits on IO.
As the HTTP requests are handled by the Kestrel web server embedded into Jube, this translates to a request being made of a Thread from the .Net managed Thread Pool.
The .Net Thread Pool is perfect at sizing itself, however in periods of burst, thread load or saturation can occur, which may lead to inconsistent - yet not necessarily unreasonable - response times. The manageability and maintainability of the application becomes difficult should unmarshalled requests be made of the thread pool, especially in burst. Henceforth, as a philosophy, it is a preferred approach to queue work on in the instances internal concurrent queues and manually size thread counts, having detailed knowledge of the underlying hardware infrastructure. A further benefit of the manual thread sizing approach is to allow for asynchronicity in the client integration, without having to relying on asynchronous methods and thread pools client side, which may bring about many of the same challenges that this architecture sets out to solve.
There exists a switch in the model invocation endpoint which will instruct the asynchronous processing of a transaction or event:
https://localhost:5001/api/invoke/EntityAnalysisModel/90c425fd-101a-420b-91d1-cb7a24a969cc
By adding the Async switch to the endpoint url the request payload is only processed up to the conclusion of Request XPath, after which the remainder of the processing is instructed via internal asynchronous queue, with a partial response payload being returned by HTTP. There are a predefined number of threads available for the remaining model invocation, specified as an Environment Variable called ModelInvokeAsynchronousThreads:
ModelInvokeAsynchronousThreads=1
In the event of burst, or incorrectly sized thread instantiations given available compute and load, then the queue will backup. The queue balance can be monitored via select:
select * from "EntityAnalysisModelAsynchronousQueueBalance"
Returning the following data:
Queue balances are captured roughly evey minute. In burst, queues backing up are to be expected, and will clear assuming that the threads are sized slightly ahead of demand.
The response payload can be called back via the invocation of the GET HTTP method Callback HTTP endpoint using the entityAnalysisModelInstanceEntryGuid element value taken from the initial HTTP response as key:
https://localhost:5001/api/invoke/EntityAnalysisModel/Callback/de1119dc-dda0-4916-a716-bbd39c69a17e
The response payload will be identical to that returned synchronously having been stored in an instance buffer once it is complete, and communicated via notification to other running instances. The default timeout is three seconds which implies that should background processing not have concluded, and made available within thirty seconds, a HTTP status 408 will be returned. The presence of timeouts would imply that that the threads are not handling burst adequately and not able to reach requests within tolerance.
A callback may only be made once, as the callback is deleted upon matching EntityAnalysisModelInstanceEntryGuid value.
To enable HTTP callbacks, it is necessary to set the EnableCallback and CallbackTimeout Jube Environment Variables:
EnableCallback=True
CallbackTimeout=3000
There exists cluster wide publishing of callback Create and Delete events to facilitate effective continued load balancing. The full response payload is stored in the instance memory, duplicated by however many instances are configured for callback, which is the main rational for both Create and Delete events, allowing for the releasing of the memory across all instances just as soon as the Callback has been collected.
Given the potential for memory saturation on orphaned callbacks, in the event that a callback has not been collected, it will be removed upon the timout interval having been realised, configured in milliseconds in the CallbackTimeout Environment Variable. Monitoring for timeouts is achieved by polling the "EntityAnalysisModelAsynchronousQueueBalance" table as above, as a timout counter is maintained for each Callback.