* Adds consistent hashing with bound loads sharding algorithm
Signed-off-by: Akram Ben Aissi <[email protected]>
* Make the assignement consistent accross all clusters
- The assignment or running of the algorithm has to be consistent across all the clusters. Changed the function to return a map where the consistent hash will be used to build the map
- Modifications to the createConsistentHashsingWithBoundLoads function. This will create the map for cluster to shard. Note that the list must be consistent across all shards so that is why the cluster list must be sorted before going through the consistent hash algorithm
Signed-off-by: Akram Ben Aissi <[email protected]>
* Extracting constant and simplifying boolean expression
Signed-off-by: Akram Ben Aissi <[email protected]>
* Update docs: consistent-hashing sharding algorithm
Signed-off-by: Akram Ben Aissi <[email protected]>
---------
Signed-off-by: Akram Ben Aissi <[email protected]>
* Adding app list to sharding cache
Signed-off-by: Andrew Lee <[email protected]>
* Add shard by apps test
Signed-off-by: Andrew Lee <[email protected]>
* Fix lint
Signed-off-by: Andrew Lee <[email protected]>
* Add coverage to test
Signed-off-by: Andrew Lee <[email protected]>
* Fix lint
Signed-off-by: Andrew Lee <[email protected]>
* Converted cluster/app accesors to private, add apps-in-any-namespace suport in shardingcache init, added read lock to GetAppDistribution
Signed-off-by: Andrew Lee <[email protected]>
* Fix tests
Signed-off-by: Andrew Lee <[email protected]>
---------
Signed-off-by: Andrew Lee <[email protected]>
* feat(sharding): use a cache
Signed-off-by: Alexandre Gaudreault <[email protected]>
* cluster cmd
Signed-off-by: Alexandre Gaudreault <[email protected]>
* - Assign shard 0 to in-cluster cluster and nil check updates
- Caching clusters while sharding: Fixing unit tests
- Update generated docs
- Debug e2e tests
- Default the shardNumber to the number of replicas if it is calculated to a higher value
- defered Unlock only when a lock is set
- Disabling temporarly other versions of k3s to check if e2e passes
- Do not fail if hostname format is not abc-n
- Fix unit test and skip some e2e
- Skip TestGitSubmoduleHTTPSSupport test
- Remove breaking defer c.lock.Unlock()
- Reverting testing all k3s version
- Default sharding fix
Signed-off-by: Akram Ben Aissi <[email protected]>
Signed-off-by: Akram Ben Aissi <[email protected]>
* fixes related to code review: renaming structure param, moving db initialisation
Signed-off-by: Akram Ben Aissi <[email protected]>
* Code review
Signed-off-by: Akram Ben Aissi <[email protected]>
* Set default shard to 0
Signed-off-by: Akram Ben Aissi <[email protected]>
* Set different default value for Sts and Deployment mode
Signed-off-by: Akram Ben Aissi <[email protected]>
* Expose ClusterShardingCache
Signed-off-by: Akram Ben Aissi <[email protected]>
* Removing use of argoDB.db for DistributionFunction
Signed-off-by: Akram Ben Aissi <[email protected]>
* Update generated documentation
Signed-off-by: Akram Ben Aissi <[email protected]>
* Fix comment about NoShardingDistributionFunction and NoShardingAlgorithm
Signed-off-by: Akram Ben Aissi <[email protected]>
---------
Signed-off-by: Alexandre Gaudreault <[email protected]>
Signed-off-by: Akram Ben Aissi <[email protected]>
Co-authored-by: Alexandre Gaudreault <[email protected]>
This commit replaces `os.Setenv` with `t.Setenv` in tests. The
environment variable is automatically restored to its original value
when the test and all its subtests complete.
Reference: https://pkg.go.dev/testing#T.Setenv
Signed-off-by: Eng Zer Jun <[email protected]>
* fix: Extraction of DistributionFunction to allow passing different type of functions to filter clusters by shard
- Adding unit tests for sharding
- Refresh clusters list on DistributionFunction call
Signed-off-by: Akram Ben Aissi <[email protected]>
Signed-off-by: ishitasequeira <[email protected]>
* fix: Incorrect conversion of an integer with architecture-dependent bit size from [strconv.Atoi](1) to a lower bit size type uint32 without an upper bound check.
Signed-off-by: Akram Ben Aissi <[email protected]>
* Added config to switch to round-robin sharding
Signed-off-by: Raghavi Shirur <[email protected]>
Signed-off-by: Akram Ben Aissi <[email protected]>
* Documenting sharding more, adding shuffling tests (skipped), re-enable sharding algo env var
Signed-off-by: Akram Ben Aissi <[email protected]>
* Allow configuration through argocd-cmd-params-cm configMap and key: controller.sharding.algorithm
Signed-off-by: Akram Ben Aissi <[email protected]>
* De-duplicate code, remove reflection for default case, shorten distribution methods name, ran codegen on manifests
Signed-off-by: Akram Ben Aissi <[email protected]>
Signed-off-by: Akram Ben Aissi <[email protected]>
---------
Signed-off-by: Akram Ben Aissi <[email protected]>
Signed-off-by: ishitasequeira <[email protected]>
Signed-off-by: Raghavi Shirur <[email protected]>
Co-authored-by: Raghavi Shirur <[email protected]>
