## What changed?
Add SQL support for matching task fairness.
Based on #8043 with some changes to reduce code duplication (can be
applied to cassandra later).
## Why?
Support all persistence implementations.
## How did you test it?
- [x] covered by existing tests
- [x] added new unit test(s)
---------
Co-authored-by: sivagirish81 <[email protected]>
## What changed?
<!-- Describe what has changed in this PR -->
Remove license header from every file. Because it is really hard to
follow in this PR here is the summary:
1. License header is removed from all `.go` and `.proto` files
:fireworks::fireworks:🎆.
2. `LICENSE` file in the root directory has only Temporal and Uber
copyrights.
3. 5 other `LICENSE` files added to the packages which have copyrights
different from Temporal and Uber: Datadog, Xargin, "Mat Ryer, Tyler
Bunnell and contributors".
4. `license_file` flag is removed from all code generation tools.
5. `copyright_file` flag is removed from `go:generate mockgen`
directive.
6. All copyright related targets are removed from `Makefile`.
7. Updated Temporal copyright year to 2025 everywhere.
## Why?
<!-- Tell your future self why have you made these changes -->
I double checked with legal department that it is not needed to have
license header in every file. One file per repo is enough. I put all
copyrights to the root `LICENSE` file and removed header from all other
files. Also updated tools and `Makefile`.
## What changed?
<!-- Describe what has changed in this PR -->
Remove duplicate code: three implementations of `CreateAdminDB` were
exactly identical to the respective `CreateDB`.
## Why?
<!-- Tell your future self why have you made these changes -->
DRY.
## How did you test it?
<!-- How have you verified this change? Tested locally? Added a unit
test? Checked in staging env? -->
Existing tests.
## Potential risks
<!-- Assuming the worst case, what can be broken when deploying this
change to production? -->
## Documentation
<!-- Have you made sure this change doesn't falsify anything currently
stated in `docs/`? If significant
new behavior is added, have you described that in `docs/`? -->
## Is hotfix candidate?
<!-- Is this PR a hotfix candidate or does it require a notification to
be sent to the broader community? (Yes/No) -->
## What changed?
SQL persistence support has been added for CHASM nodes.
- A new `chasm_node_maps` table has been added to SQL schemas
- SQL test suites updated
- SQL execution manager/util updated
- SQL plugins for `chasm_node_map` table
## Why?
- We've landed Cassandra persistence support for CHASM nodes, this
brings SQL support up to parity.
## How did you test it?
- New tests ran on all SQL plugins
- `go test -v ./...`
## Potential risks
- No application paths write chasm_node_maps yet, but change does
involve a schema change
---------
Co-authored-by: Yichao Yang <[email protected]>
## What changed?
Make import formatting a part of the build, and test validity with
"ensure-no-changes" target
## Why?
So we never discuss import order again (hopefully)
## How did you test it?
its a build change, so the build will test it!
## Potential risks
As long as build works, the only thing I risk is the ire of coworker who
want imports to be different ;)
## Documentation
N/A
## Is hotfix candidate?
No
## What changed?
Both our PostgreSQL and MySQL database backends will now automatically
reconnect to the database when certain errors occur: all errors chosen
have been experienced when testing this behavior through an AWS Aurora
RDS failover of either MySQL or PostgreSQL.
For both backends we will reconnect when we see:
- `ECONNRESET`
- `ECONNABORTED`
- `ECONNREFUSED`
- `io.EOF`
- `io.ErrUnexpectedEOF`
- `database/sql/driver.ErrBadConn`
for postgres we will also reconnect on the following SQLStates:
- `25006` read-only transaction
- `57P03` cannot connect now
- `0A000` feature not supported, but ONLY when the message is `cannot
set transaction read-write mode during recovery`
for mysql we will also reconnect when we see the following error codes:
- `1040` too many connections
- `1792` read-only transaction (SQLstate `25006`)
- `1836` running in read-only mode
This logic is easily extensible should we discover more failure modes
over time
## Why?
We've had multiple community reports of Temporal problems during RDS
failover. One part of this is the fact that we wouldn't necessarily
reconnect; we were at the whims of our chosen SQL abstraction's
connection pooling logic.
## How did you test it?
I manually tested this functionality in the presence of repeated RDS
failovers:
- [x] postgres12 plugin with pq driver
- [x] postgres12 plugin with pgx driver
- [x] mysql plugin
Automated testing will be added to our regular testing pipelines once
our infrastructure friends have added the support I need (it's in
progress)
## Potential risks
We're concerned there's a correctness issue in our PostgreSQL backend
that's related to our behavior during an RDS failover. If we merge this
before I figure out what's going on, we could hide the issue and make it
harder to reproduce.
## Documentation
N/A
## Is hotfix candidate?
Yes?
## What changed?
- Refactored Nexus Registry code to merge incoming and outgoing into one
registry
- Added separate concept of "endpoints" - which is roughly analogous to
what was previously called "services"
- Upgraded the Nexus Go SDK and API dependencies accordingly
- Command handler interface was modified to accept a context given new
requirements
- Note that invoking a worker endpoint is not yet supported, there will
be some minor restructuring required to enable that
- Requires this unmerged API PR
https://github.com/temporalio/api/pull/403. The two PRs should be merged
together.
## Why?
Part of late redesign. See also https://github.com/nexus-rpc/api/pull/5
## How did you test it?
Modified existing and added new tests.
## What changed?
Attempting to configure a MySQL 8 Visibility database with
go-sql-driver/mysql's `interpolateParams` option will now fail.
## Why?
There are a number of reasons why this doesn't work right now so we're
going to prevent it from happening while we discuss how much we should
invest in fixing it. The current work to fix this is in
https://github.com/temporalio/temporal/pull/5428
## How did you test it?
Added a new test
## Potential risks
There should be none. Any user who has currently set up their database
this way will already have encountered exciting errors like `Cannot
create a JSON value from a string with CHARACTER SET 'binary'` and
`Unable to parse ExecutionStatus value from DB`.
## Is hotfix candidate?
That's up for discussion
This PR is intended to merge 4 months of work in the `nexus` feature
branch into `main`.
The functionality it brings is:
- Dispatching Nexus Tasks by namespace and task queue
- Internal implementation of the Incoming Service Registry - not yet
exposed
- Attaching workflow close callbacks on `StartWorkflowExecutionRequest`
and processing of those callbacks
There's more to come and some of the callback code will be refactored
into a new plugin architecture
(https://github.com/temporalio/temporal/pull/5446) soon.
---------
Co-authored-by: PJ Doerner <[email protected]>
<!-- Describe what has changed in this PR -->
**What changed?**
Adding EnqueueMessage() and ReadMessage() methods for SQL version of
persistence.QueueV2, which supersedes the persistence.Queue interface.
The implementations for delete and merge are left out for now and will
be implemented later.
<!-- Tell your future self why have you made these changes -->
**Why?**
We plan on using this for the upcoming history task DLQ project.
<!-- How have you verified this change? Tested locally? Added a unit
test? Checked in staging env? -->
**How did you test it?**
Unit tests.
<!-- Assuming the worst case, what can be broken when deploying this
change to production? -->
**Potential risks**
<!-- Is this PR a hotfix candidate or require that a notification be
sent to the broader community? (Yes/No) -->
**Is hotfix candidate?**
* Revert "Capture UpdateInfos in MutableState mutation (#4372)"
This reverts commit c78ee02eee.
* Revert "Adds update state indexing to MutableState (#4224)"
This reverts commit f8303d8e7e.
* Add UpdateInfo map to WorkflowExecutionInfo
This inlines the UpdateInfo index into the pre-existing execution blob
so that downstream persistence implementations won't see any change.
Add UpdateInfo records to MutableState
A mechanism for storing pointers into the workflow history representing
update accepted or completed events. Storage added for Cassandra,
PosgtreSQL, MySQL, and SQLite.
* Add version check method to SQL interface
* Remove tools version check method in favor of functions defined in persistence package
* Refactor tests to not depend on tools package
* Optimize SQL layer supporting batch delete
* Split DeleteFromActivityInfoMaps into DeleteFromActivityInfoMaps and DeleteAllFromActivityInfoMaps
* Split DeleteFromTimerInfoMaps into DeleteFromTimerInfoMaps and DeleteAllFromTimerInfoMaps
* Split DeleteFromChildExecutionInfoMaps into DeleteFromChildExecutionInfoMaps and DeleteAllFromChildExecutionInfoMaps
* Split DeleteFromRequestCancelInfoMaps into DeleteFromRequestCancelInfoMaps and DeleteAllFromRequestCancelInfoMaps
* Split DeleteFromSignalInfoMaps into DeleteFromSignalInfoMaps and DeleteAllFromSignalInfoMaps
* Split DeleteFromSignalsRequestedSets into DeleteFromSignalsRequestedSets and DeleteAllFromSignalsRequestedSets
* Restructure NamespaceInfo Protos
* System namespace registration is moved to the immutableClusterMetadata path, rather than registering in schema
* Add NewUUID(), Fix Namespace test
* Remove unused import
* Serialization moved to Manager
