tasks

mirror of https://github.com/hanzoai/tasks.git synced 2026-10-04 02:04:03 +00:00

Files

T

History

Carly de FrondevilleandGitHub 68fe1f306a support embedded schema for cassandra and sql update-schema and setup-schema (#5195 )

## What changed?


This change adds a flag to `temporal-sql-tool` and
`temporal-cassandra-tool`'s `update-schema` and `setup-schema` to
specify an embedded schema instead of a path to the schema directory or
schema file.

```
NAME:
   temporal-cassandra-tool update-schema - update cassandra schema to a specific version

USAGE:
   temporal-cassandra-tool update-schema [command options] [arguments...]

OPTIONS:
   --version value, -v value      target version for the schema update, defaults to latest
   --schema-dir value, -d value   path to directory containing versioned schema
   --schema-name value, -s value  name of embedded versioned schema, one of: [cassandra/temporal cassandra/visibility]
```

```
NAME:
   temporal-sql-tool update-schema - update sql schema to a specific version

USAGE:
   temporal-sql-tool update-schema [command options] [arguments...]

OPTIONS:
   --version value, -v value      target version for the schema update, defaults to latest
   --schema-dir value, -d value   path to directory containing versioned schema
   --schema-name value, -s value  name of embedded versioned schema, one of: [mysql/v57/temporal mysql/v57/visibility mysql/v8/temporal mysql/v8/visibility postgresql/v96/temporal postgresql/v96/visibility postgresql/v12/temporal postgresql/v12/visibility]
```


## Why?
We want to embed the db schemas in the `temporal-sql-tool` and
`temporal-cassandra-tool` with `go:embed` so that customers can simply
specify which schema they want using a flag instead of having to copy
the schema files to wherever they're running the db tool and passing the
path to the schema.

https://github.com/temporalio/temporal/issues/2059

## How did you test it?
I wrote a new unit test for reading the embedded file system, and the
existing codepath of the changed functions are all tested by the
existing unit tests. I also updated the unit test for config validation
to reflect the new changes.

I tested this locally with a locally running Postgres. Since the more
complex changes are in `updatetask.go`, which is shared by the sql and
the cassandra tool, I think there is no need to also test with a real
cassandra db. I also built the cassandra command line tool and confirmed
that the flags and validation checks behaved as expected.

## Potential risks
Schema updates could fail. But the changes are pretty contained to the
new `-s` flag which isn't used anywhere in production yet, so the risk
is very low. The code for the `-d` flag remained basically the same.

## Is hotfix candidate?
No

2023-12-05 12:22:06 -08:00

cqlclient_tests.go

Separate integration tests and unit tests (#3760 )

2023-01-04 22:17:25 -08:00

cqlclient.go

support embedded schema for cassandra and sql update-schema and setup-schema (#5195 )

2023-12-05 12:22:06 -08:00

handler_test.go

[WIP] Fix error return bug and some minor code style (#4234 )

2023-04-26 14:00:32 -07:00

handler.go

support embedded schema for cassandra and sql update-schema and setup-schema (#5195 )

2023-12-05 12:22:06 -08:00

main.go

support embedded schema for cassandra and sql update-schema and setup-schema (#5195 )

2023-12-05 12:22:06 -08:00

README.md

schama -> schema (#3501 )

2022-10-17 10:11:26 -07:00

setup_task_tests.go

Make all Go source file names snake case (#4558 )

2023-07-03 14:23:56 -04:00

update_task_tests.go

Make all Go source file names snake case (#4558 )

2023-07-03 14:23:56 -04:00

version_tests.go

Separate integration tests and unit tests (#3760 )

2023-01-04 22:17:25 -08:00

README.md

Using the cassandra schema tool

This package contains the tooling for temporal cassandra operations.

For localhost development

For the very first time run:

make

then run:

make install-schema

to create schema in your cassandra instance.

For production

Create the binaries

Run make
You should see an executable temporal-cassandra-tool

Do one time database creation and schema setup for a new cluster

This uses Cassandra's SimpleStratagey for replication. For production, we recommend using a replication factor of 3 with NetworkTopologyStrategy.

temporal-cassandra-tool --ep $CASSANDRA_SEEDS create -k $KEYSPACE --rf $RF

See https://www.ecyrd.com/cassandracalculator for an easy way to determine how many nodes and what replication factor you will want to use. Note that Temporal by default uses LOCAL_QUORUM and LOCAL_SERIAL for read and write consistency.

./temporal-cassandra-tool -ep 127.0.0.1 -k temporal setup-schema -v 0.0 -- this sets up just the schema version tables with initial version of 0.0
./temporal-cassandra-tool -ep 127.0.0.1 -k temporal update-schema -d ./schema/cassandra/temporal/versioned -- upgrades your schema to the latest version

./temporal-cassandra-tool -ep 127.0.0.1 -k temporal_visibility setup-schema -v 0.0 -- this sets up just the schema version tables with initial version of 0.0 for visibility
./temporal-cassandra-tool -ep 127.0.0.1 -k temporal_visibility update-schema -d ./schema/cassandra/visibility/versioned -- upgrades your schema to the latest version for visibility

Update schema as part of a release

You can only upgrade to a new version after the initial setup done above.

./temporal-cassandra-tool -ep 127.0.0.1 -k temporal update-schema -d ./schema/cassandra/temporal/versioned -v x.x    -- executes the upgrade to version x.x

./temporal-cassandra-tool -ep 127.0.0.1 -k temporal_visibility update-schema -d ./schema/cassandra/visibility/versioned -v x.x    -- executes the upgrade to version x.x