mirror of
https://github.com/hanzoai/team.git
synced 2026-09-30 01:57:35 +00:00
d96a1cb4ea
* MM-68179: Run sendLoop workers on all HA nodes
In HA clusters, sendLoop worker goroutines only ran on the leader node.
When an API request to send a channel invite landed on a non-leader node,
SendMsg enqueued the task to a local in-memory channel but no goroutine
consumed it, silently losing the message. Fix by starting sendLoop workers
in Start() on all nodes, independent of the leader-only ping lifecycle.
- Separate sendLoop lifecycle (Start/Shutdown) from ping lifecycle
(pingStart/pingStop on leader change)
- Rename resume/pause to pingStart/pingStop for clarity
- Change Active() to mean "service started" via atomic.Bool
- Remove SetActive (no longer needed; tests use Start())
* address review comment
* Added idempotency guard to Start()
* Start() and Shutdown(): CompareAndSwap instead of Load/Store — eliminates races where concurrent calls could both proceed. Only the winner of the CAS executes; the loser returns nil
immediately.
Ping test: replaced time.Sleep with assert.Never/assert.Eventually — no more brittle fixed sleeps. Uses assert.Never to verify no pings fire on non-leader, and assert.Eventually to
verify pings stop after losing leadership (snapshot-then-compare pattern).
* make unit tests parallel capable
---------
Co-authored-by: Mattermost Build <[email protected]>
Remote Cluster Service
Package remotecluster implements Mattermost's "Secured Connections" feature, which enables communication between different Mattermost clusters. Specifically, this package provides:
Service Management:
- Manages inter-cluster communication via topic-based messages
- Handles connection state (active/inactive) based on cluster leadership
- Maintains concurrent send channels (MaxConcurrentSends = 10) for parallel message processing
- Implements periodic health checks (pings) to monitor remote cluster connectivity
Message Handling:
- Sends messages using a pool of goroutines to handle concurrent sends while preserving message order per remote
- Uses hash-based routing to ensure messages for the same remote ID go to the same channel
- Supports different types of sends: messages, files, and profile images
- Implements topic-based message routing with listener callbacks
Connection Management:
- Handles invitation confirmations between clusters
- Maintains HTTP client connections with proper timeouts and transport settings
- Supports connection state listeners for monitoring remote cluster availability
- Implements ping mechanism to verify remote cluster health
Core Features:
- Topic-based message routing
- File transfer capabilities
- Profile image synchronization
- Invitation system for establishing connections
- Health monitoring via pings
- Concurrent message processing
- Connection state management
This package is designed to be thread-safe and handles leadership changes in clustered environments, only running active operations on the leader node.