Commit Graph
99 Commits
Author SHA1 Message Date
wxing1292andGitHub f330e30bdd Fix PostgreSQL EnableHostVerification behavior (#1048) 2020-12-02 18:11:47 -08:00
wxing1292andGitHub d26f78a13d Remove forced namespace cache refresh (#1014) (#1016)
* Remove forced namespace cache refresh
* Make SQL insert into history tree idempotent

NOTE: Previously, if a namespace is newly created and not loaded
in cache, the namespace cache can do forced cache refresh reducing
the wait time for caller. This forced cache refreshment may able
to trigger a new callback for failover. Now, if a task with deleted
namespace is processed, namespace cache will trigger the above
cache refreshment and causing a deadlock. The only solution now is
to remove the namespace cache forced refreshment logic.
2020-11-20 14:09:12 -08:00
wxing1292andGitHub 6b04aa3997 Optimize SQL layer supporting batch delete (#982)
* Optimize SQL layer supporting batch delete
* Split DeleteFromActivityInfoMaps into DeleteFromActivityInfoMaps and DeleteAllFromActivityInfoMaps
* Split DeleteFromTimerInfoMaps into DeleteFromTimerInfoMaps and DeleteAllFromTimerInfoMaps
* Split DeleteFromChildExecutionInfoMaps into DeleteFromChildExecutionInfoMaps and DeleteAllFromChildExecutionInfoMaps
* Split DeleteFromRequestCancelInfoMaps into DeleteFromRequestCancelInfoMaps and DeleteAllFromRequestCancelInfoMaps
* Split DeleteFromSignalInfoMaps into DeleteFromSignalInfoMaps and DeleteAllFromSignalInfoMaps
* Split DeleteFromSignalsRequestedSets into DeleteFromSignalsRequestedSets and DeleteAllFromSignalsRequestedSets
2020-11-16 16:50:30 -08:00
wxing1292andGitHub 24b80b2f91 Add context to all SQL interfaces (#980)
* Add context to all SQL interfaces
* Wire context with SQLx
* Set default execution DB timeout to 5s
* Set default visibility DB timeout to 10s
2020-11-11 16:30:36 -08:00
wxing1292andGitHub b1532fda64 Use latest sqlx and dependencies & update SQL max conn config (#951)
* Revert "Revert "Use sqlx master branch (#924)" (#947)"

This reverts commit 392b8217fc.

* Use latest sqlx and dependencies

* Revert "Update SQL max conn lifetime according to recommendations"

This reverts commit 680ab493f3618ec4f18b010e08a5b67eb51fbf29.
2020-11-11 16:07:18 -08:00
wxing1292andGitHub 7f86935d1f NoSQL / SQL specific changes for messages, part 4 (#979)
* Unify queue message ID definition
2020-11-11 15:25:09 -08:00
wxing1292andGitHub ef7860ce5d Add SQL specific tests for messages, part 2 (#914)
* Rewrite queue metadata APIs similar to transfer tasks, timer tasks
* Add queue metadata tests
* Add missing encoding type
2020-11-09 11:57:38 -08:00
wxing1292andGitHub 392b8217fc Revert "Use sqlx master branch (#924)" (#947)
This reverts commit 586480fd8a.
2020-11-03 11:08:57 -08:00
wxing1292andGitHub a8e5c2fa03 Bugfix: matching persistence layer update sticky task queue (#935)
* Sticky task queue update should be done within transaction lock or conditional update
2020-11-02 11:54:52 -08:00
wxing1292andGitHub 5942de0661 Use filter struct instead of pointer for matching SQL layer (#934) 2020-10-31 11:10:14 -07:00
wxing1292andGitHub 586480fd8a Use sqlx master branch (#924) 2020-10-30 12:07:01 -07:00
wxing1292andGitHub a13e753cd7 Add SQL specific tests for messages, part 1 (#892)
* Rewrite APIs similar to transfer tasks, timer tasks
* Add queue message tests
2020-10-26 19:13:45 -07:00
wxing1292andGitHub 0dc3f9d611 Split remaining SQL persistence interfaces into dedicated files (#872)
* Move ClusterMetadata into dedicated file
* Move HistoryExecutionBuffer into dedicated file
* Move HistoryShard into dedicated file
* Move Namespace into dedicated file
* Move MatchingTask into dedicated file
* Move MatchingTaskQueue into dedicated file
* Use filter struct directly instead of pointer to struct
2020-10-21 10:01:11 -07:00
wxing1292andGitHub 3f4c9d0742 Add SQL specific tests for replication tasks (#859)
* Some minor code structure change
* Add replication tasks tests
* Add replication DLQ tasks tests
2020-10-15 15:55:28 -07:00
wxing1292andGitHub 75a0122292 Update PostgreSQL TLS config (#853)
* Allow CA or key / cert to be separately set for PostgreSQL
* Add support of host name verification & fix existing issue
2020-10-14 09:28:39 -07:00
wxing1292andGitHub 4d8d473fd2 Add TLS configuration for PostgreSQL (#849) 2020-10-13 19:00:26 -07:00
Alex ShtinandGitHub 3f02bf677e Update license header in all go files (#837) 2020-10-13 15:23:23 -07:00
VitalyandGitHub f6a48e2b30 Adding dual read/write for immutable metadata (#840) 2020-10-12 21:22:25 -07:00
wxing1292andGitHub 08eefd0741 Split overloaded SQL filters (#834)
* Split ActivityInfoMapsFilter into ActivityInfoMapsSelectFilter and ActivityInfoMapsDeleteFilter
* Split TimerInfoMapsFilter into TimerInfoMapsSelectFilter and TimerInfoMapsDeleteFilter
* Split ChildExecutionInfoMapsFilter into ChildExecutionInfoMapsSelectFilter and ChildExecutionInfoMapsDeleteFilter
* Split RequestCancelInfoMapsFilter into RequestCancelInfoMapsSelectFilter and RequestCancelInfoMapsDeleteFilter
* Split SignalInfoMapsFilter into SignalInfoMapsSelectFilter and SignalInfoMapsDeleteFilter
* Split SignalsRequestedSetsFilter into SignalsRequestedSetsSelectFilter and SignalsRequestedSetsDeleteFilter
2020-10-12 18:05:17 -07:00
wxing1292andGitHub 9ff4ba4d3c Reduce redundant history tree creation for workflow reset (#825)
* When performing workflow reset & terminating current workflow, one unnecessary create history tree API is called, this PR remove the redundant call to improve performance
2020-10-12 17:24:43 -07:00
Sergey BykovandGitHub 36faccb366 Enforce minimum TLS version of 1.2 (#831)
* Enforce minimum TLS version of 1.2
2020-10-12 11:04:11 -07:00
wxing1292andGitHub 7b086939f1 Enable PostgreSQL tests on github (#821)
* Enable PostgreSQL tests on Github
* Move some tests around to avoid import cycle
2020-10-09 17:16:40 -07:00
wxing1292andGitHub 33e374bbea Add SQL specific tests for history events (#820)
* Add history node tests
* Add history tree tests
2020-10-09 16:55:22 -07:00
wxing1292andGitHub a1369eeca6 Use timestamp instead of convert for time.Time pointer conversion (#819) 2020-10-08 15:40:59 -07:00
wxing1292andGitHub 9aee69ab46 Add SQL specific tests for executions (#814)
* Add history current execution tests
* Add history execution tests
2020-10-08 15:23:04 -07:00
wxing1292andGitHub e844e65f4b Change shard ID type from int to int32 (#811)
* Unify the shard ID type to int32
* Remove unused admin Kafka resend command
2020-10-07 23:46:48 -07:00
wxing1292andGitHub b802e5e67a Add SQL specific tests for visibility, part 2 (#813)
* Add visibility tests
* Add visibility pagination tests
2020-10-07 21:17:56 -07:00
wxing1292andGitHub 47ddc73ab1 Add SQL specific tests for visibility, part 1 (#809)
* Add visibility tests
* Unify the behavior on MySQL & PostgreSQL
2020-10-07 19:53:44 -07:00
VitalyandGitHub 25cde0971b Adding mutable metadata and cluster id (#803)
Adding mutable metadata and cluster id
2020-10-07 19:28:22 -07:00
wxing1292andGitHub bb46fa2649 Split overloaded transfer / timer task APIs (#806)
* Split TransferTasksFilter into TransferTasksFilter and TransferTasksRangeFilter
* Split SelectFromTransferTasks into SelectFromTransferTasks and RangeSelectFromTransferTasks
* Split DeleteFromTransferTasks into DeleteFromTransferTasks and RangeDeleteFromTransferTasks
* Split TimerTasksFilter into TimerTasksFilter and TimerTasksRangeFilter
* Split SelectFromTimerTasks into SelectFromTimerTasks and RangeSelectFromTimerTasks
* Split DeleteFromTimerTasks into DeleteFromTimerTasks and RangeDeleteFromTimerTasks
2020-10-07 15:47:31 -07:00
wxing1292andGitHub 8f7e868b56 Add SQL specific tests for transfer / timer tasks (#802)
* Add history transfer task tests
* Add history timer task tests
* Add missing functionalities
2020-10-06 12:51:55 -07:00
wxing1292andGitHub 0682f421f8 Add SQL specific tests for mutable state, part 2 (#800)
* Add history execution request cancel tests
* Add history execution signal tests
* Add history execution signal request tests
2020-10-06 12:08:05 -07:00
wxing1292andGitHub b28183f9b0 Add SQL specific tests for mutable state, part 1 (#799)
* Add history execution buffer tests
* Add history execution activity tests
* Add history execution child workflow tests
* Add history execution timer tests
2020-10-05 15:37:10 -07:00
wxing1292andGitHub c3f861e075 Add SQL specific tests (#796)
* Add history shard tests
2020-10-05 14:59:24 -07:00
wxing1292andGitHub 1f56c0492a Add SQL specific tests for namespace (#793)
* Add namespace tests
* Remove redundant select during creation of namespace
2020-10-05 14:34:04 -07:00
wxing1292andGitHub 845940ab38 Add SQL persistence specific tests (#790)
* Add matching task queue tests
2020-10-02 21:40:50 -07:00
wxing1292andGitHub c3d126b59a Add SQL persistence specific tests (#789)
* Add matching task tests
2020-10-02 17:58:59 -07:00
wxing1292andGitHub 35272a29e1 Split SQL storage interfaces & definitions (#788) 2020-10-02 11:47:09 -07:00
Quanzheng LongandAlex Shtin f5f781cbf7 [SQL]Fix upsert SQL template for Postgres plugin (#3498) 2020-09-25 18:54:08 -07:00
Samar AbbasandGitHub 6aeba03656 Fixes MySQL persistence for workflow reset (#713)
After 2dc removal we trigger a slightly different codepath on mysql
persistence when resetting workflows.
AppendHistoryNodes when called with IsNewBranch add both
history_tree metadata and history_node.  This caused the problem
because history_node is already added when fork was called so
adding it again failed with duplicate error.  Updated
AppendHistoryNodes api to upsert instead.

Another issue was with create workflow execution never expected
to be called with `CreateWorkflowModeContinueAsNew` mode.  With
NDC now workflow reset is implemented in similar fashion as
doing continue as new.  So removed the check to disallow
`CreateWorkflowModeContinueAsNew` on create execution and added
appropriate validation.
2020-09-05 18:53:19 -07:00
Shawn HathawayandGitHub 2db621446c Support time.Time in MutableState#ActivityWithTimerHeartbeat methods (#666) 2020-08-13 12:31:26 -07:00
Shawn HathawayandGitHub 02b201e55a Replace Insert Ignore in Execution (#658) 2020-08-09 00:35:22 -07:00
Alex ShtinandGitHub 97abc65ebc Replace time.Now() with time.Now().UTC() and time.Unix() with time.Unix().UTC() (#648) 2020-08-04 13:02:16 -07:00
Shawn HathawayandGitHub 55508de3f7 Convert ActivityInfo Persistence POGOs to Proto (#615) 2020-07-27 20:42:28 -07:00
Shawn HathawayandGitHub cbe7cab9ff Convert REPLACE INTO -> INSERT ON DUPLICATE KEY UPDATE (#601) 2020-07-24 03:10:22 -07:00
Shawn HathawayandGitHub 815cd8b09f Add pinned version_partition column to schema version tables (SQL) (#602) 2020-07-23 19:16:30 -07:00
Shawn HathawayandGitHub 7ae7ab205e ClusterMembership Partition Pin/AutoIncrement Removal (#583) 2020-07-22 10:08:41 -07:00
mastermanuandGitHub 3e4936d9c7 Changes ListClosedWorkflow result ordering for SQL-based visibility stores to be based off of workflow close time (#578)
This PR fixes the following issues:

SQL stores were using the Workflow Open Time instead of the Close Time for pagination, which is not aligned with Cassandra and is not the intention.

The SQL store pagination logic had a bug where the tie-breaker logic of using run_id for two rows with the same close_time was not implemented properly.

The SQL store pagination logic had a bug where it was using the MinStart time instead of the MaxStart time for pagination purposes (to be confirmed as to whether this is truly a bug or not, but it does appear that way)

Fixes Makefile so that schema installation for MySQL / Postgres functions properly.

Fixes Postgres username / password in postgres development environment.

Adds appropriate MySQL/Postgres indexes for querying by close_time
2020-07-21 12:23:10 -07:00
Shawn HathawayandGitHub 467131c5f6 Add Hash to Task Schema Key, compound TaskQueueId (#557) 2020-07-20 11:12:12 -07:00
Shawn HathawayandGitHub 888f942274 TaskQueue Schema Collapse (#500) 2020-07-16 20:35:41 -07:00