Michael Snowdenandrodrigozhou
73e6c8d570
<!-- Describe what has changed in this PR --> **What changed?** We now only return one record per workflow execution when listing archived workflows with the S3 storage backend. <!-- Tell your future self why have you made these changes --> **Why?** To fix https://github.com/temporalio/temporal/issues/4432#issuecomment-1661078704. There's more details in the comments in the code as to why this bug happened and how it is fixed. <!-- How have you verified this change? Tested locally? Added a unit test? Checked in staging env? --> **How did you test it?** I modified our unit test to verify that we don't return any duplicates. The previous unit test was actually too permissive and allowed duplicates. I also manually verified this by restarting the server. Before: <img width="947" alt="image" src="https://github.com/temporalio/temporal/assets/5942963/3a616521-f26b-45f6-b7b4-063f6c3b9321"> After: <img width="966" alt="image" src="https://github.com/temporalio/temporal/assets/5942963/bd650c30-dc56-4260-8439-237c5ac75ddb"> <!-- Assuming the worst case, what can be broken when deploying this change to production? --> **Potential risks** The ability to list archived workflows when using S3 is already very new, as it was completely unsupported before, and the changes here are just isolated to that query path, so there is little to lose even if this just crashes. <!-- Is this PR a hotfix candidate or require that a notification be sent to the broader community? (Yes/No) --> **Is hotfix candidate?** Should update the issue when this lands.
Amazon S3 blobstore
Configuration
See https://docs.aws.amazon.com/sdk-for-go/v1/developer-guide/configuring-sdk.html#specifying-credentials on how to set up authentication against s3
Enabling archival is done by using the configuration below. Region and bucket URI are required
archival:
history:
state: "enabled"
enableRead: true
provider:
s3store:
region: "us-east-1"
visibility:
state: "enabled"
enableRead: true
provider:
s3store:
region: "us-east-1"
namespaceDefaults:
archival:
history:
state: "enabled"
URI: "s3://<bucket-name>"
visibility:
state: "enabled"
URI: "s3://<bucket-name>"
Visibility query syntax
You can query the visibility store by using the tctl workflow listarchived command
The syntax for the query is based on SQL
Supported column names are
- WorkflowId String
- WorkflowTypeName String
- StartTime Date
- CloseTime Date
- SearchPrecision String - Day, Hour, Minute, Second
WorkflowId or WorkflowTypeName is required. If filtering on date use StartTime or CloseTime in combination with SearchPrecision.
Searching for a record will be done in times in the UTC timezone
SearchPrecision specifies what range you want to search for records. If you use SearchPrecision = 'Day'
it will search all records starting from 2020-01-21T00:00:00Z to 2020-01-21T59:59:59Z
Limitations
- The only operator supported is
=due to how records are stored in s3.
Example
Searches for all records done in day 2020-01-21 with the specified workflow id
./tctl --ns samples-namespace workflow listarchived -q "StartTime = '2020-01-21T00:00:00Z' AND WorkflowId='workflow-id' AND SearchPrecision='Day'"
Storage in S3
Workflow runs are stored in s3 using the following structure
s3://<bucket-name>/<namespace-id>/
history/<workflow-id>/<run-id>
visibility/
workflowTypeName/<workflow-type-name>/
startTimeout/2020-01-21T16:16:11Z/<run-id>
closeTimeout/2020-01-21T16:16:11Z/<run-id>
workflowID/<workflow-id>/
startTimeout/2020-01-21T16:16:11Z/<run-id>
closeTimeout/2020-01-21T16:16:11Z/<run-id>