Commit Graph
81 Commits
Author SHA1 Message Date
Rodrigo ZhouandGitHub b2a733823e Support CountWorkflowExecutions for SQL DB (#3955) 2023-02-14 16:59:12 -08:00
Alex ShtinandGitHub 743be2c562 Fix ExecutionTime rounding when using PostgreSQL (#3952) 2023-02-13 18:50:13 -08:00
Rodrigo ZhouandGitHub d7f4606485 Advanced visibility for PostgreSQL (#3896) 2023-02-02 22:16:37 -08:00
Rodrigo ZhouandGitHub d49786debf [Advanced Visibility with SQL] Adding PostgreSQL 12 schema and interface (#3844)
* Create base schema for PostgreSQL 12

* Changes to PostgreSQL 12 visibility schame to support advanced visibility

* Create PostgreSQL 12 db interface and configs
2023-01-31 14:35:51 -08:00
Rodrigo ZhouandGitHub 7e1ddb70dc Add GetIndexName method to visibility manager (#3820) 2023-01-27 00:44:39 -08:00
Rodrigo ZhouandGitHub 77f0af8625 Fix time fields of rows from GetClusterMembers (#3840) 2023-01-25 17:18:50 -08:00
Rodrigo ZhouandGitHub f5c2446dc3 Fix GetWorkflowExecution in PostgreSQL (#3816) 2023-01-24 12:20:45 -08:00
Yichao YangandGitHub b5ff033db0 Support arbitrary history task category in SQL persistence (#3489) 2022-10-19 16:08:01 -07:00
Rodrigo ZhouandGitHub 4213f95d0f Create visibility GetWorkflowExecution API (#3488) 2022-10-17 19:26:16 -07:00
Michael SnowdenandGitHub 73f94d9300 Implement GetAllHistoryTreeBranches for SQL persistence backends (#3438) 2022-09-29 17:00:06 -07:00
Alexander MaysandGitHub 74c74e0a45 Remove rds iam auth plugin (#3079)
* Revert "Fix auth plugin configuration to match the new SQLAuthPlugin config block (#2844)"

This reverts commit 37ab2b5664.

* Revert rds-iam-auth - current drivers dont support authentication callbacks

Signed-off-by: Alexander Mays <[email protected]>
2022-07-10 14:31:02 -07:00
Alexander MaysandGitHub b8f3f89555 Add RDS IAM auth plugin for SQL drivers (#2830)
Signed-off-by: Alexander Mays <[email protected]>
Added:
A new environment variable to the CLI: SQL_AUTH_PLUGIN=rds-iam-auth
A new flag arg to the CLI: --sql-auth-plugin rds-iam-auth
2 new docker template variables: authPlugin: {{ default .Env.SQL_AUTH_PLUGIN "" }} and authPlugin: {{ default .Env.SQL_VIS_AUTH_PLUGIN "" }}
A new SQL configuration attribute authPlugin
2022-05-12 16:28:26 -07:00
Jeremy BreidingandGitHub 5d0296a421 Add sqlite persistence tests (#2681)
* Add sqlite persistence tests

* fix up a test

* Remove limit on prune operation for sql plugin
2022-03-30 13:05:33 -07:00
Yichao YangandGitHub ae23339fca Make matching persistence range queries [inclusive, exclusive) (#2599)
- Range for matching GetTasks is now [inclusive, exclusive), instead of (exclusive, inclusive]
- Range for matching CompleteTasksLessThan is now < max TaskID, instead of <=
- Range for admin ListTaskQueueTasks is now [inclusive, exclusive)
2022-03-15 10:42:12 -07:00
Yichao YangandGitHub be2e4a2d56 Support pagination for SQL GetTransfer/VisibilityTasks (#2564)
- support pagination for sql GetTransfer/VisibilityTasks
- improve pagination for sql GetTimer/ReplicationTasks
- new ExecutionMutableStateTaskSuite suite
2022-03-04 12:29:31 -08:00
Yichao YangandGitHub 3896610a3d Unify history task persistence range query behavior (#2547)
- Make range based on taskID and task fire time both [inclusive, exclusive)
2022-02-28 12:08:18 -08:00
Jeremy BreidingandGitHub a5ddc3ca73 Sql task queue visibility (#2537)
* Adding task_queue to read/write paths
for postgresql executions_visibility

* Adding task_queue for mysql and sqlite
2022-02-24 18:22:46 -08:00
Mikhail GryzykhinandGitHub ef2063c9e5 Add support for GetWorkflowHistory in reverse order (#2442)
Add API to fetch workflow history in reverse order
2022-02-14 12:49:02 -08:00
Yichao YangandGitHub 13b102065e Remove unused queue processor (#2473) 2022-02-08 10:39:11 -08:00
wxing1292andGitHub 7258e62519 Separate SQL session management into a dedicated file (#2391)
* Create reusable MySQL session util
* Create reusable PostgreSQL session util
2022-01-19 13:33:07 -08:00
Yu XiaandGitHub c3a5c35916 Remove cluster metadata dual write (#2358)
* Remove cluster metadata dual write

* Update sample cluster config
2022-01-11 14:09:09 -08:00
wxing1292andGitHub fa2db04ca3 Split lease task queue API into 2 (#2269)
* Split matching persistence LeaseTaskQueue API into
  * CreateTaskQueue
  * GetTaskQueue
* Add new task queue UT
* Add new task queue task UT
* Deprecate legacy matching task tests
* Modify logic accordingly
2021-12-10 21:39:05 -08:00
wxing1292andGitHub 0aa486930d Remove unnecessary start version from current execution table (#2236) 2021-11-29 14:46:56 -08:00
dbbbb27dd6 Add List Cluster Metadata API (#2206)
* Add List Cluster Metdata API

Co-authored-by: yux0 <[email protected]>
2021-11-19 14:31:25 -08:00
VitalyandGitHub a12c16d612 Add tiered storage queue processor (#2189) 2021-11-18 21:34:13 -08:00
5abf350905 Make cluster metadata schema backward compatible (#2177)
* Make cluster metadata schema backward compatible

Co-authored-by: yux0 <[email protected]>
2021-11-15 13:48:07 -08:00
c38d135021 Migrate cluster metadata store to use cluster metadata info table (#2155)
* Migrate cluster metadata store to use cluster metadata info table

Co-authored-by: yux0 <[email protected]>
2021-11-11 15:51:43 -08:00
Yimin ChenandGitHub 1ddc3c397f Promote local namespace to global namespace (#2105) 2021-11-01 15:05:26 -07:00
Michael NussbaumandGitHub 923d9ca555 Make SQL CLI tool more idempotent (#2104)
* Make PostgreSQL CLI tool DB create and schema create idempotent
* Don't error if a target schema version is specified and schema is up-to-date
2021-10-28 23:02:56 -07:00
Harshita PoojaryandGitHub 997d0211e1 Escape password in connection url (#2015)
* Escape password in connection url

* Run make goimports in root
2021-10-03 11:18:15 -07:00
wxing1292andGitHub 0ecaee5672 Use internal error to indicate code bug / contract breach (#1958)
* Update sql persistence layer
* Update cassandra persistence layer
* Update visibility persistence layer
2021-09-21 17:59:56 -07:00
wxing1292andGitHub b7a94537c0 Use serviceerror.Unavailable error type instead of serviceerror.Internal (#1916)
* Use serviceerror.Unavailable error type instead of serviceerror.Internal
  * serviceerror.Unavailable indicates errors like service's dependency is down, retryable
  * serviceerror.Internal indicates errors like NPE in service logic, non-retryable
2021-09-13 18:31:14 -07:00
Yimin ChenandGitHub 5b336f4883 Merge AppendHistory with CreateWorkflow (#1853) 2021-08-26 17:12:34 -07:00
Yimin ChenandGitHub c0f5486fd6 Refactor Queue for namespace replication (#1625)
* Refactor Queue for namespace replication

* fix test
2021-06-15 14:00:44 -07:00
wxing1292andGitHub a865a21973 Show connection error details for PostgreSQL layer (#1581)
* Return errors encountered when trying to connect to PostgreSQL compatible DBs
2021-05-27 17:42:44 -07:00
wxing1292andGitHub a8efffb0e7 Implement history node deletion logic (#1443)
* SQL: Rename SelectFromHistoryNode to RangeSelectFromHistoryNode
* SQL: Rename DeleteFromHistoryNode to RangeDeleteFromHistoryNode
* SQL: New DeleteFromHistoryNode aiming to delete one record from history node table
2021-04-13 16:52:53 -07:00
wxing1292andGitHub 65cadaed5c Unify ReadHistoryBranch logic (#1441)
* New InternalHistoryNode struct containing history node info
* Modify SQL persistence layer to use key set pagination, instead of existing hacky solution
* ReadHistoryBranch in addition will allow returning history node metadata only
* Move history node filtering logic from persistence layer to upper history store layer
2021-04-13 16:14:40 -07:00
wxing1292andGitHub 1a4d21bc14 Add prev transaction ID attribute to history node table for events chaining (#1435)
* New last_history_node_txn_id within execution info tracking the last valid history node
* New prev_txn_id within history node table tracking previous valid history node record
2021-04-06 15:38:08 -07:00
wxing1292andGitHub cc0c337790 Begin the migration of executions table operation using atomic counter (#1420)
* Add db version attribute to executions table for better CAS guarantee
* 1.9.x will by default use existing next event ID for CAS, but will also support the new DB version CAS
* 1.10.x will by default use DB version CAS

Last event ID is not enough for CAS operation protection: e.g. activity heartbeat / buffer events.
The newly introduced DB version will be increased by 1 each time execution table is updated (update to mutable state record).
2021-04-05 15:03:57 -07:00
wxing1292andGitHub 048bbe8832 Improve compatibility with Aiven DB (#1413)
* Try using a set of default database names when initializing admin connection

Aiven DB is a PostgreSQL compatible DB, but comes with a different default DB name
2021-03-26 16:59:52 -07:00
Alex ShtinandGitHub 996824e53c Move config and dynamicconfig packages to the common (#1372) 2021-03-15 10:45:11 -07:00
wxing1292andGitHub a4e4b37136 Unify db version check (#1299)
* Add version check method to SQL interface
* Remove tools version check method in favor of functions defined in persistence package
* Refactor tests to not depend on tools package
2021-02-18 10:41:44 -08:00
Alex ShtinandGitHub c4d40f4db9 Bugfix: NoopResolver was always used for version compatibility check (#1233) 2021-01-28 07:47:41 -08:00
wxing1292andGitHub d3d96148fb Cleanup DB schema (#1130) 2021-01-27 16:07:17 -08:00
wxing1292andGitHub 066448b5cf Improve PostgreSQL config (#1214)
* Include connectAttributes as part of connection string
2021-01-22 11:27:44 -08:00
Alex ShtinandGitHub b530bb0f5f Add ServiceResolver interface and corresponding ServerOption (#1188) 2021-01-14 21:29:04 -08:00
wxing1292andGitHub a8430b6561 Cleanup debug logs (#1184) 2021-01-13 12:22:55 -08:00
Alex ShtinandGitHub 46fe71cf04 Drop database/keyspace if exists (#1181) 2021-01-12 22:07:05 -08:00
wxing1292andGitHub 3eeeb8d967 Bugfix schema migration (#1129)
* Since 1.5.x is still using the to be deprecated column names, postpone the deprecation
2020-12-23 12:58:13 -08:00
wxing1292andGitHub d38faebead Cleanup cluster metadata table (#1102) 2020-12-22 22:54:42 -08:00