mirror of
https://github.com/hanzoai/tasks.git
synced 2026-10-04 02:04:03 +00:00
sqlstore
359
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  zeekayandHanzo Dev
|
cdca774661 |
One derived key, no wrapping
tasks generated a DEK per shard, wrapped it under a KEK derived from the master, and stored the wrapped blob beside the database — so it also needed an upgrade path to rewrap, a principal vocabulary to derive against, and tests for all of it. hanzoai/cek derives the key from the master and the namespace instead. Nothing is stored, so there is nothing to wrap, rewrap, or upgrade. dek.go, principal.go and upgrade.go are gone with the tests that pinned them: they proved properties of a scheme that no longer exists, not properties the store still has to hold. That also unpins hanzoai/sqlite, which deleted the DEK API in v0.5.0. tasks now builds against it, which is what proves the migration is complete rather than merely compiling. Databases written under the old scheme will not open under the new derivation. There are no customers and no data to preserve. Co-authored-by: Hanzo Dev <[email protected]> |
||
|
zeekayandHanzo Dev
|
e3636baf8e |
tasks: the trusted issuer is a set, and an empty entry is not one
TASKSD_WHITELABEL_ISSUERS adds the brands this tasksd accepts tokens from beyond the primary, each of which signs its own. Adding one is configuration. The sharp edge is the empty entry: splitting an UNSET env var yields one empty string, and an empty trusted issuer would match a token that carries no `iss` at all. Entries are trimmed and dropped, pinned by TestIssuerAllowlistDropsEmptyEntries — which builds the verifier through the same code path production uses, so the guard is tested where it runs rather than in isolation. Co-authored-by: Hanzo Dev <[email protected]> |
||
|
zeekayandHanzo Dev
|
f927109617 |
tasks: retire the fourth reader of one contract
pkg/auth held 190 lines of JWKS cache and JWT verification — a fourth independent
reading of what an IAM token means — and it had two defects the shared reader does
not:
it verified against EVERY key in the published set and took the first that
worked, so a token naming kid A was accepted on a signature from key B. Naming a
key is what `kid` is for.
its issuer comparison was CONDITIONAL on the configured issuer being non-empty,
so an unset TASKSD_JWT_ISSUER did not fail closed — it accepted tokens from any
issuer.
It also stripped only the four names it reads. Every other minted header —
X-User-IsAdmin and X-User-Permissions (the platform and money signals),
X-User-Owner, X-Billing-Account-Id, X-Scope, X-Workspace-Id, and the retired names
— passed through from the client untouched. tasksd terminates identity itself (its
ingress routes straight to the Service), so nothing upstream was deleting them
either. Restoring the four-name strip turns
TestRequireIdentity_StripsTheWholeEstateSet red on seven forged headers.
And it had NO machine predicate at all, so an admin-org client_credentials token
arrived as a person. It now reads the membership set, like the rest of the estate.
What remains is the part that is genuinely tasksd's: which environment variables
the policy comes from. The audience stays optional and that is deliberate rather
than an oversight — IAM sets `aud` to the requesting CLIENT (RFC 8707), and tasksd
is not a client id, so it pins the issuer and the signing keys instead.
Both transports now answer "which org does this act in" with the same function,
Claims.EffectiveOrg(""), rather than the HTTP path reading `owner` and the ZAP path
reading it separately.
Cost, measured: 342 linked packages and an 18.3 MB binary, DOWN from 345 and
18.6 MB — dropping go-jose more than paid for the leaf, and tasks links ZERO zip
packages because the edge names no transport. go-jose is gone from the module
entirely; the tests now mint with golang-jwt, the library IAM signs with.
Rebased onto 12 upstream commits: kept their ServeGated(addr string) signature,
their engine.As(Org(...)) rename, their dependency versions, and the LLM.md fact
about explicit project/user isolation. Their only change to the deleted jwt.go was
a comment naming hanzo.id, which the replacement already says.
Co-authored-by: Hanzo Dev <[email protected]>
|
||
|
zeekayandHanzo Dev
|
432713bae3 |
feat(store): tasksd upgrade — move a pre-principal store into this layout
An older binary addressed a shard by org alone (<data>/<org>/<ns>.db). This one reads three levels down, so the first boot after the upgrade finds nothing, creates empty shards beside the full ones, enumerates zero tenants and never fires a schedule again — silently, with every byte still on disk one directory up. store.Upgrade inserts the two sentinel levels uniformly, including the old empty-org dir (_ -> _/_/_). A rename within one filesystem: no page rewritten, no re-encryption. Journals move before their database, so a run that stops half way never leaves a database without the WAL holding its last commits. Idempotent — a moved shard leaves nothing at its old path — so a deploy can run it unconditionally ahead of the daemon. Two refusals, both loud and both non-destructive: A master key is set. Pre-upgrade shards are plaintext and moving one does not encrypt it; the next boot would wrap a fresh DEK around a file it then failed to decrypt. Move first, encrypt second. A destination is occupied (ErrOccupied). The only way there is booting this binary before the move ran, which creates an EMPTY shard where the full one belongs — and anything started after that boot is in the empty one. Both files are kept, the other tenants still move, and the operator picks which is authoritative. upgrade and migrate stay separate axes: upgrade moves EVERY shard forward in time (layout), migrate moves ONE shard across space (nodes). Also: refuse a socket path over sockaddr_un.sun_path by name. The kernel answers "bind: invalid argument" and names neither the path nor the limit, and a socket under a deployment's data directory sits close to it. Co-authored-by: Hanzo Dev <[email protected]> |
||
|
zeekayandHanzo Dev
|
679ae61163 |
feat(transport): address the engine and the SDK, so both can use a socket
The engine took a PORT and the SDK an Options.HostPort, so neither end
could use a unix socket for traffic that never leaves the host. The
field name was not the only obstacle: luxfi/zap bound net.Listen("tcp",
":port") and reached a peer with net.DialTimeout("tcp", addr), so a path
was refused at the dialer with "missing port in address" and could not
be bound at all. That is fixed upstream (luxfi/zap Address + Network);
this pins the commit until it is tagged.
EmbedConfig.Address, Embedded.Address, ServeGated(addr) and
client.Options.Address each take ONE address, because a port is a detail
of one transport rather than a thing every listener has. A path binds or
dials a unix socket; anything else is host:port.
TestDialRequiresHostPort became TestDialRequiresAddress: Dial needs an
address, and asserts a socket path is refused for being absent rather
than for being portless. TestE2E_UnixSocket runs a real workflow over a
socket end to end, and TestE2E_UnixSocketRebind proves the socket is
unlinked on stop so a restart binds cleanly.
Co-authored-by: Hanzo Dev <[email protected]>
|
||
|
zeekayandHanzo Dev
|
b4529923c1 |
feat(store): principal-addressed shards, created on demand, encrypted at rest
The store addressed a shard by (org, namespace) and only the root org's `default` namespace existed, so anything else had to be declared before it could be used and there was no way to isolate a project or a user. Shards also had no way to be keyed, so tasks was the one Hanzo store writing tenant state in the clear. Both are the same missing value, so both are one change. Principal is the tenant that owns a shard: an org, optionally narrowed to a project and a user. It serves the two places tenancy is decided, so they cannot disagree — the shard's directory (<data>/<org>/<project>/<user>/<ns>.db, each unset leg written "_") and the key its DEK is wrapped under. The fixed-depth encoding is injective, so a project and a user of the same name are different tenants, and the KEK chain master -> org -> project -> user gives each of them a key contained in the one it narrows. Namespaces come into existence on first use (engine.namespace), the way their shards already did. ValidName is the one rule every name that becomes a path segment passes through, so a namespace can no longer escape or alias a directory. EmbedConfig.MasterKey opens every shard encrypted: each file carries a DEK of its own, wrapped for its principal and bound to a random file id in a sidecar beside it, so rotating the master rewraps sidecars and never rewrites ciphertext, and moving the data directory is harmless. A keyed shard is ciphertext on every build — hanzoai/sqlite v0.4.0 uses the live libsqlcipher codec when one is linked and its pure-Go SQLCipher envelope when one is not — so there is nothing here to branch on. On the envelope a shard is sealed at checkpoint/close rather than per commit, which is what Shard.Checkpoint and the manager's sweep bound. replication.Frame and migration.Job carry the canonical principal string instead of an org, so a frame lands in the directory its principal names. Co-authored-by: Hanzo Dev <[email protected]> |
||
|
zeekay
|
706aeb7411 |
refactor(replication): move quasar driver to its own package (base sheds consensus)
hanzoai/base transitively pulled github.com/luxfi/consensus solely through pkg/tasks → replication (quasar.go, the only consensus importer). That made base — and every base consumer — vulnerable to the recurring luxfi/consensus force-moved-tag / vanished-geth supply-chain rot, for a driver base never uses. Make Quasar replication optional by isolating it: - Add Kind() + a unified Stats(accepted,rejected,timeouts) to the Replicator interface so embed.go introspects drivers through the interface, never a concrete type. - Move quasar.go (+ its test) into a new pkg/tasks/replication/quasar package that imports the interface package; the consensus dependency now lives ONLY there. Only cmd/tasksd imports it. - pkg/tasks → replication now has ZERO consensus in its dep graph (`go list -deps ./pkg/tasks | grep consensus` = 0), so base sheds it via Go 1.17+ module-graph pruning. Also unbreak tasks' own consensus: v1.22.85 → v1.36.3 (geth v1.16.73 was a vanished revision; v1.36.3 floors the valid unified luxfi stack). Non-quasar packages build + test green CGO-off; the quasar package builds in CI (its consensus→secp256k1 CGO can't compile in this sandbox — environmental). |
||
|
|
6f6a6460ab |
chore(tasks): rip all gRPC + protobuf — ZAP-native only (HIP-0114) (#12)
Complete the documented 2026-04-26 native-ZAP migration whose deletion
was left incomplete. The engine (cmd/tasksd), SDK (pkg/sdk) and
in-process server (pkg/tasks) are already 100% ZAP-native with zero
real go.temporal.io / google.golang.org/grpc / protobuf imports; the
only residual gRPC+Temporal footprint was dead porting scaffolding:
- rm proto/ — 55 vendored Temporal .proto (4 gRPC service defs +
message shapes), zero generated .pb.go, zero Go references.
- rm cmd/tools/ — 4 Temporal-porting tools (genrpcwrappers,
genroutingkeyextractor, getproto, check-dependencies); the only
files importing google.golang.org/grpc + go.temporal.io/{api,server}.
- go mod tidy — drops go.temporal.io/api, go.temporal.io/server,
google.golang.org/grpc, grpc-gateway entirely (zero grpc/protobuf
direct or transitive in go.mod).
- CI: remove orphaned proto jobs (linters.yml lint-protos/lint-api,
check-release-dependencies.yml) that ran deleted tools / missing
make targets.
- LLM.md: scrub stale gRPC transport claims -> ZAP :9999 / HTTP :7243.
Transport is luxfi/zap on :9999. go build ./... , go vet, go test all green.
Co-authored-by: Hanzo Dev <[email protected]>
|
||
|
zeekay
|
6b89892286 |
merge: upstream/main (preserve Hanzo brand, 494 commits)
# Conflicts: # .github/workflows/build-and-publish.yml # AGENTS.md # AGENTS.md~upstream_main # api/adminservice/v1/request_response.go-helpers.pb.go # api/adminservice/v1/request_response.pb.go # api/adminservice/v1/service.pb.go # api/adminservice/v1/service_grpc.pb.go # api/adminservicemock/v1/service_grpc.pb.mock.go # api/archiver/v1/message.pb.go # api/batch/v1/request_response.pb.go # api/checksum/v1/message.pb.go # api/deployment/v1/message.go-helpers.pb.go # api/deployment/v1/message.pb.go # api/enums/v1/replication.go-helpers.pb.go # api/enums/v1/replication.pb.go # api/enums/v1/task.go-helpers.pb.go # api/enums/v1/task.pb.go # api/errordetails/v1/message.go-helpers.pb.go # api/errordetails/v1/message.pb.go # api/historyservice/v1/request_response.go-helpers.pb.go # api/historyservice/v1/request_response.pb.go # api/historyservice/v1/service.pb.go # api/historyservice/v1/service_grpc.pb.go # api/historyservicemock/v1/service_grpc.pb.mock.go # api/matchingservice/v1/request_response.pb.go # api/matchingservice/v1/service.pb.go # api/persistence/v1/chasm.pb.go # api/persistence/v1/executions.go-helpers.pb.go # api/persistence/v1/executions.pb.go # api/persistence/v1/workflow_mutable_state.pb.go # api/replication/v1/message.pb.go # api/routing/v1/extension.pb.go # api/schedule/v1/message.pb.go # api/taskqueue/v1/message.go-helpers.pb.go # api/taskqueue/v1/message.pb.go # chasm/component.go # chasm/component_mock.go # chasm/context.go # chasm/context_mock.go # chasm/engine.go # chasm/engine_mock.go # chasm/field.go # chasm/field_test.go # chasm/lib/activity/activity.go # chasm/lib/activity/activity_tasks.go # chasm/lib/activity/activity_test.go # chasm/lib/activity/config.go # chasm/lib/activity/frontend.go # chasm/lib/activity/fx.go # chasm/lib/activity/gen/activitypb/v1/activity_state.pb.go # chasm/lib/activity/gen/activitypb/v1/request_response.go-helpers.pb.go # chasm/lib/activity/gen/activitypb/v1/request_response.pb.go # chasm/lib/activity/gen/activitypb/v1/service.pb.go # chasm/lib/activity/gen/activitypb/v1/service_client.pb.go # chasm/lib/activity/gen/activitypb/v1/service_grpc.pb.go # chasm/lib/activity/handler.go # chasm/lib/activity/library.go # chasm/lib/activity/proto/v1/activity_state.proto # chasm/lib/activity/proto/v1/request_response.proto # chasm/lib/activity/proto/v1/service.proto # chasm/lib/activity/proto/v1/tasks.proto # chasm/lib/activity/statemachine.go # chasm/lib/activity/statemachine_test.go # chasm/lib/activity/validator.go # chasm/lib/activity/validator_test.go # chasm/lib/buf.yaml # chasm/lib/callback/component.go # chasm/lib/callback/config.go # chasm/lib/callback/fx.go # chasm/lib/callback/invocable_internal.go # chasm/lib/callback/invocable_outbound.go # chasm/lib/callback/library.go # chasm/lib/callback/proto/v1/message.proto # chasm/lib/callback/proto/v1/tasks.proto # chasm/lib/callback/request.go # chasm/lib/callback/request_test.go # chasm/lib/callback/tasks.go # chasm/lib/callback/tasks_test.go # chasm/lib/nexusoperation/cancellation.go # chasm/lib/nexusoperation/cancellation_statemachine.go # chasm/lib/nexusoperation/config.go # chasm/lib/nexusoperation/fx.go # chasm/lib/nexusoperation/gen/nexusoperationpb/v1/operation.go-helpers.pb.go # chasm/lib/nexusoperation/gen/nexusoperationpb/v1/operation.pb.go # chasm/lib/nexusoperation/gen/nexusoperationpb/v1/tasks.go-helpers.pb.go # chasm/lib/nexusoperation/gen/nexusoperationpb/v1/tasks.pb.go # chasm/lib/nexusoperation/library.go # chasm/lib/nexusoperation/metrics.go # chasm/lib/nexusoperation/operation.go # chasm/lib/nexusoperation/operation_statemachine.go # chasm/lib/nexusoperation/proto/v1/operation.proto # chasm/lib/nexusoperation/proto/v1/tasks.proto # chasm/lib/scheduler/backfiller.go # chasm/lib/scheduler/backfiller_tasks.go # chasm/lib/scheduler/backfiller_tasks_test.go # chasm/lib/scheduler/config.go # chasm/lib/scheduler/export_test.go # chasm/lib/scheduler/fx.go # chasm/lib/scheduler/gen/schedulerpb/v1/message.go-helpers.pb.go # chasm/lib/scheduler/gen/schedulerpb/v1/message.pb.go # chasm/lib/scheduler/gen/schedulerpb/v1/request_response.go-helpers.pb.go # chasm/lib/scheduler/gen/schedulerpb/v1/request_response.pb.go # chasm/lib/scheduler/gen/schedulerpb/v1/service.pb.go # chasm/lib/scheduler/gen/schedulerpb/v1/service_client.pb.go # chasm/lib/scheduler/gen/schedulerpb/v1/service_grpc.pb.go # chasm/lib/scheduler/gen/schedulerpb/v1/tasks.go-helpers.pb.go # chasm/lib/scheduler/gen/schedulerpb/v1/tasks.pb.go # chasm/lib/scheduler/generator.go # chasm/lib/scheduler/generator_tasks.go # chasm/lib/scheduler/generator_tasks_test.go # chasm/lib/scheduler/handler.go # chasm/lib/scheduler/handler_test.go # chasm/lib/scheduler/helper_test.go # chasm/lib/scheduler/invoker.go # chasm/lib/scheduler/invoker_execute_task_test.go # chasm/lib/scheduler/invoker_process_buffer_task_test.go # chasm/lib/scheduler/invoker_tasks.go # chasm/lib/scheduler/library.go # chasm/lib/scheduler/migration/migration.go # chasm/lib/scheduler/migration/migration_test.go # chasm/lib/scheduler/proto/v1/message.proto # chasm/lib/scheduler/proto/v1/request_response.proto # chasm/lib/scheduler/proto/v1/service.proto # chasm/lib/scheduler/proto/v1/tasks.proto # chasm/lib/scheduler/scheduler.go # chasm/lib/scheduler/scheduler_idle_tasks_test.go # chasm/lib/scheduler/scheduler_tasks.go # chasm/lib/scheduler/scheduler_test.go # chasm/lib/scheduler/spec_processor.go # chasm/lib/tests/gen/testspb/v1/service.pb.go # chasm/lib/tests/handler.go # chasm/lib/tests/library.go # chasm/lib/tests/payload.go # chasm/lib/tests/proto/v1/message.proto # chasm/lib/tests/proto/v1/request_response.proto # chasm/lib/tests/proto/v1/service.proto # chasm/lib/tests/tasks.go # chasm/lib/workflow/fx.go # chasm/lib/workflow/library.go # chasm/lib/workflow/workflow.go # chasm/library_core.go # chasm/ms_pointer.go # chasm/nexus_operation_processor.go # chasm/node_backend_mock.go # chasm/ref.go # chasm/ref_test.go # chasm/registrable_component.go # chasm/registrable_task.go # chasm/registry.go # chasm/registry_test.go # chasm/search_attribute.go # chasm/search_attribute_test.go # chasm/statemachine.go # chasm/task.go # chasm/task_mock.go # chasm/test_component_test.go # chasm/test_library_test.go # chasm/test_task_test.go # chasm/test_var_test.go # chasm/tree.go # chasm/tree_test.go # chasm/visibility.go # chasm/visibility_manager.go # chasm/visibility_manager_mock.go # chasm/visibility_test.go # chasm/visibility_value.go # chasm/visibility_value_test.go # chasm/workflow.go # client/admin/client_gen.go # client/admin/metric_client.go # client/admin/metric_client_gen.go # client/admin/retryable_client_gen.go # client/frontend/client_gen.go # client/frontend/metric_client.go # client/frontend/metric_client_gen.go # client/frontend/retryable_client_gen.go # client/history/client_gen.go # client/history/metric_client.go # client/history/metric_client_gen.go # client/history/retryable_client_gen.go # client/matching/README.md # client/matching/client.go # client/matching/client_gen.go # client/matching/loadbalancer.go # client/matching/metric_client.go # client/matching/metric_client_gen.go # cmd/tools/genrpcwrappers/main.go # cmd/tools/getproto/files.go # cmd/tools/getproto/main.go # common/api/metadata.go # common/archiver/README.md # common/archiver/gcloud/connector/client.go # common/archiver/provider/provider.go # common/archiver/provider/provider_mock.go # common/archiver/s3store/history_archiver.go # common/archiver/s3store/history_archiver_test.go # common/archiver/s3store/mocks/generate.go # common/archiver/s3store/mocks/s3api.go # common/archiver/s3store/util.go # common/archiver/s3store/visibility_archiver.go # common/archiver/s3store/visibility_archiver_test.go # common/authorization/authorizer.go # common/authorization/claim_mapper.go # common/authorization/default_authorizer.go # common/authorization/default_jwt_claim_mapper.go # common/authorization/default_token_key_provider.go # common/authorization/interceptor.go # common/authorization/interceptor_test.go # common/authorization/roles.go # common/cluster/metadata.go # common/cluster/metadata_test.go # common/config/config.go # common/config/config_template_embedded.yaml # common/config/persistence.go # common/config/persistence_test.go # common/constants.go # common/contextutil/metadata.go # common/contextutil/metadata_test.go # common/dynamicconfig/constants.go # common/dynamicconfig/shared_constants.go # common/enums/defaults.go # common/headers/caller_info.go # common/headers/headers.go # common/headers/version_checker.go # common/log/tag/values.go # common/log/zap_logger.go # common/membership/ringpop/factory.go # common/membership/ringpop/monitor.go # common/membership/ringpop/service_resolver.go # common/membership/ringpop/test_cluster.go # common/metrics/config.go # common/metrics/metric_defs.go # common/metrics/tags.go # common/metrics/tally_metrics_handler.go # common/metrics/tally_metrics_handler_test.go # common/metrics/task_queues.go # common/namespace/namespace.go # common/namespace/nsregistry/registry.go # common/namespace/nsregistry/registry_test.go # common/namespace/nsregistry/registry_watch_test.go # common/namespace/nsreplication/replication_task_executor.go # common/namespace/nsreplication/replication_task_executor_test.go # common/namespace/nsreplication/transmission_task_handler.go # common/namespace/nsreplication/transmission_task_handler_test.go # common/namespace/registry.go # common/namespace/registry_mock.go # common/namespace/replication_resolver.go # common/namespace/replication_resolver_test.go # common/nexus/callback_token.go # common/nexus/endpoint_registry.go # common/nexus/endpoint_registry_test.go # common/nexus/failure.go # common/nexus/failure_test.go # common/nexus/link_converter.go # common/nexus/link_converter_test.go # common/nexus/nexustest/server.go # common/nexus/payload_serializer.go # common/nexus/payload_serializer_test.go # common/payload/payload.go # common/payload/payload_test.go # common/payloads/payloads.go # common/persistence/cassandra/matching_task_store_queue.go # common/persistence/cassandra/matching_task_store_user_data.go # common/persistence/cassandra/matching_task_store_v1.go # common/persistence/cassandra/matching_task_store_v2.go # common/persistence/cassandra/queue_v2_store.go # common/persistence/data_interfaces.go # common/persistence/faultinjection/cluster_metadata_store_gen.go # common/persistence/faultinjection/data_store_factory.go # common/persistence/faultinjection/execution_store_gen.go # common/persistence/faultinjection/fault_generator.go # common/persistence/faultinjection/gowrap_template # common/persistence/faultinjection/metadata_store_gen.go # common/persistence/faultinjection/method_fault_generator.go # common/persistence/faultinjection/nexus_endpoint_store_gen.go # common/persistence/faultinjection/queue_gen.go # common/persistence/faultinjection/queue_v2_gen.go # common/persistence/faultinjection/shard_store_gen.go # common/persistence/faultinjection/store_fault_generator.go # common/persistence/faultinjection/task_store_gen.go # common/persistence/persistence_interface.go # common/persistence/persistencetest/queues.go # common/persistence/persistencetest/queues_test.go # common/persistence/serialization/codec.go # common/persistence/serialization/codec_test.go # common/persistence/serialization/serializer.go # common/persistence/serialization/task_serializers.go # common/persistence/serialization/task_serializers_test.go # common/persistence/sql/sqlplugin/mysql/session/session.go # common/persistence/sql/sqlplugin/mysql/visibility.go # common/persistence/sql/sqlplugin/postgresql/driver/interface.go # common/persistence/sql/sqlplugin/postgresql/driver/pgx.go # common/persistence/sql/sqlplugin/postgresql/driver/pq.go # common/persistence/sql/sqlplugin/postgresql/plugin.go # common/persistence/sql/sqlplugin/postgresql/session/session.go # common/persistence/sql/sqlplugin/sqlite/driver.go # common/persistence/sql/sqlplugin/sqlite/plugin.go # common/persistence/task_manager.go # common/persistence/tests/postgresql_test.go # common/persistence/tests/postgresql_test_util.go # common/persistence/tests/util.go # common/persistence/tests/visibility_persistence_suite.go # common/persistence/visibility/chasm_visibility_manager.go # common/persistence/visibility/chasm_visibility_manager_test.go # common/persistence/visibility/manager/visibility_manager.go # common/persistence/visibility/manager/visibility_manager_mock.go # common/persistence/visibility/store/elasticsearch/client/aws.go # common/persistence/visibility/store/elasticsearch/query_interceptors.go # common/persistence/visibility/store/elasticsearch/query_interceptors_test.go # common/persistence/visibility/store/elasticsearch/visibility_store.go # common/persistence/visibility/store/elasticsearch/visibility_store_read_test.go # common/persistence/visibility/store/query/converter.go # common/persistence/visibility/store/query/converter_test.go # common/persistence/visibility/store/sql/query_converter_legacy.go # common/persistence/visibility/store/sql/query_converter_legacy_test.go # common/persistence/visibility/store/sql/visibility_store.go # common/persistence/visibility/store/util.go # common/persistence/visibility/store/visibility_store.go # common/persistence/visibility/store/visibility_store_mock.go # common/persistence/visibility/visibility_manager_dual.go # common/persistence/visibility/visibility_manager_impl.go # common/persistence/visibility/visibility_manager_rate_limited.go # common/persistence/visibility/visiblity_manager_metrics.go # common/primitives/task_queues.go # common/quotas/clocked_rate_limiter.go # common/quotas/dynamic_rate_limiter_impl.go # common/quotas/priority_rate_limiter_impl.go # common/quotas/rate_burst.go # common/resource/fx.go # common/rpc/encryption/fixedTLSConfigProvider.go # common/rpc/encryption/local_store_tls_provider.go # common/rpc/encryption/tls_config_test.go # common/rpc/interceptor/dc_redirection_policy.go # common/rpc/interceptor/health_check.go # common/rpc/interceptor/logtags/admin_service_server_gen.go # common/rpc/interceptor/logtags/history_service_server_gen.go # common/rpc/interceptor/logtags/workflow_service_server_gen.go # common/rpc/interceptor/namespace_handover.go # common/rpc/interceptor/namespace_rate_limit.go # common/rpc/interceptor/namespace_validator.go # common/rpc/interceptor/namespace_validator_test.go # common/rpc/interceptor/rate_limit.go # common/rpc/interceptor/redirection.go # common/rpc/interceptor/redirection_test.go # common/rpc/interceptor/request_error_handler.go # common/rpc/interceptor/routing_key_interceptor_test.go # common/rpc/interceptor/service_error_interceptor.go # common/rpc/interceptor/service_error_interceptor_test.go # common/rpc/rpc.go # common/rpc/test/http_test.go # common/rpc/test/rpc_localstore_tls_test.go # common/searchattribute/encode.go # common/searchattribute/encode_test.go # common/searchattribute/manager.go # common/searchattribute/mapper.go # common/searchattribute/mapper_test.go # common/searchattribute/name_type_map.go # common/searchattribute/name_type_map_test.go # common/searchattribute/sadefs/encode_value.go # common/searchattribute/sadefs/encode_value_test.go # common/searchattribute/sadefs/util.go # common/searchattribute/search_attirbute.go # common/searchattribute/stringify.go # common/searchattribute/stringify_test.go # common/searchattribute/system_provider.go # common/searchattribute/test_provider.go # common/searchattribute/validator.go # common/serviceerror/convert.go # common/tasks/interleaved_weighted_round_robin_test.go # common/testing/historyrequire/history_require.go # common/testing/mockapi/workflowservicemock/v1/service_grpc.pb.mock.go # common/testing/mocksdk/client_mock.go # common/testing/protorequire/require.go # common/testing/testhooks/hooks.go # common/testing/testhooks/test_impl.go # common/testing/testlogger/testlogger.go # common/testing/testvars/test_vars.go # common/tqid/task_queue_id.go # common/tqid/task_queue_id_test.go # common/tqid/task_queue_validator.go # common/util/util.go # common/util/wildcard.go # common/worker_versioning/version_membership_cache.go # common/worker_versioning/worker_versioning.go # common/worker_versioning/worker_versioning_test.go # components/callbacks/config.go # components/callbacks/config_test.go # components/callbacks/request.go # components/callbacks/request_test.go # components/nexusoperations/config.go # components/nexusoperations/executors_test.go # components/nexusoperations/fx.go # components/nexusoperations/workflow/commands.go # components/nexusoperations/workflow/commands_test.go # config/development-cluster-a.yaml # config/development-cluster-b.yaml # config/development-cluster-c.yaml # config/dynamicconfig/development-cass.yaml # config/dynamicconfig/development-sql.yaml # config/dynamicconfig/development-xdc.yaml # develop/docker-compose/docker-compose.yml # develop/docker-compose/grafana/provisioning/tempo/tempo.yaml # develop/github/docker-compose.yml # docker/docker-bake.hcl # docker/targets/admin-tools.Dockerfile # docker/targets/server.Dockerfile # docs/Makefile # docs/_assets/matching-context.d2 # docs/_assets/retries.d2 # docs/architecture/chasm.md # docs/architecture/nexus.md # docs/development/macos/cassandra.md # docs/development/testing.md # proto/internal/buf.yaml # proto/internal/temporal/server/api/adminservice/v1/request_response.proto # proto/internal/temporal/server/api/adminservice/v1/service.proto # proto/internal/temporal/server/api/archiver/v1/message.proto # proto/internal/temporal/server/api/batch/v1/request_response.proto # proto/internal/temporal/server/api/checksum/v1/message.proto # proto/internal/temporal/server/api/cli/v1/message.proto # proto/internal/temporal/server/api/clock/v1/message.proto # proto/internal/temporal/server/api/cluster/v1/message.proto # proto/internal/temporal/server/api/common/v1/api_category.proto # proto/internal/temporal/server/api/common/v1/dlq.proto # proto/internal/temporal/server/api/deployment/v1/message.proto # proto/internal/temporal/server/api/enums/v1/cluster.proto # proto/internal/temporal/server/api/enums/v1/common.proto # proto/internal/temporal/server/api/enums/v1/fairness_state.proto # proto/internal/temporal/server/api/enums/v1/nexus.proto # proto/internal/temporal/server/api/enums/v1/predicate.proto # proto/internal/temporal/server/api/enums/v1/replication.proto # proto/internal/temporal/server/api/enums/v1/task.proto # proto/internal/temporal/server/api/enums/v1/workflow.proto # proto/internal/temporal/server/api/enums/v1/workflow_task_type.proto # proto/internal/temporal/server/api/errordetails/v1/message.proto # proto/internal/temporal/server/api/health/v1/message.proto # proto/internal/temporal/server/api/history/v1/message.proto # proto/internal/temporal/server/api/historyservice/v1/request_response.proto # proto/internal/temporal/server/api/historyservice/v1/service.proto # proto/internal/temporal/server/api/matchingservice/v1/request_response.proto # proto/internal/temporal/server/api/matchingservice/v1/service.proto # proto/internal/temporal/server/api/metrics/v1/message.proto # proto/internal/temporal/server/api/namespace/v1/message.proto # proto/internal/temporal/server/api/persistence/v1/chasm.proto # proto/internal/temporal/server/api/persistence/v1/chasm_visibility.proto # proto/internal/temporal/server/api/persistence/v1/cluster_metadata.proto # proto/internal/temporal/server/api/persistence/v1/executions.proto # proto/internal/temporal/server/api/persistence/v1/history_tree.proto # proto/internal/temporal/server/api/persistence/v1/hsm.proto # proto/internal/temporal/server/api/persistence/v1/namespaces.proto # proto/internal/temporal/server/api/persistence/v1/nexus.proto # proto/internal/temporal/server/api/persistence/v1/predicates.proto # proto/internal/temporal/server/api/persistence/v1/queue_metadata.proto # proto/internal/temporal/server/api/persistence/v1/queues.proto # proto/internal/temporal/server/api/persistence/v1/task_queues.proto # proto/internal/temporal/server/api/persistence/v1/tasks.proto # proto/internal/temporal/server/api/persistence/v1/update.proto # proto/internal/temporal/server/api/persistence/v1/workflow_mutable_state.proto # proto/internal/temporal/server/api/replication/v1/message.proto # proto/internal/temporal/server/api/routing/v1/extension.proto # proto/internal/temporal/server/api/schedule/v1/message.proto # proto/internal/temporal/server/api/taskqueue/v1/message.proto # proto/internal/temporal/server/api/testservice/v1/request_response.proto # proto/internal/temporal/server/api/testservice/v1/service.proto # proto/internal/temporal/server/api/token/v1/message.proto # proto/internal/temporal/server/api/workflow/v1/message.proto # service/frontend/admin_handler.go # service/frontend/admin_handler_test.go # service/frontend/configs/quotas.go # service/frontend/configs/quotas_test.go # service/frontend/errors.go # service/frontend/fx.go # service/frontend/fx_test.go # service/frontend/health_check_test.go # service/frontend/namespace_handler.go # service/frontend/namespace_handler_test.go # service/frontend/nexus_completion_http_handler.go # service/frontend/nexus_handler.go # service/frontend/nexus_handler_test.go # service/frontend/nexus_operation_http_handler.go # service/frontend/nexus_operation_http_handler_test.go # service/frontend/operator_handler.go # service/frontend/operator_handler_test.go # service/frontend/service.go # service/frontend/workflow_handler.go # service/frontend/workflow_handler_second_test.go # service/frontend/workflow_handler_test.go # service/fx.go # service/history/api/command_attr_validator.go # service/history/api/command_attr_validator_test.go # service/history/api/create_workflow_util.go # service/history/api/describeworkflow/api.go # service/history/api/get_workflow_util.go # service/history/api/multioperation/api.go # service/history/api/namespace.go # service/history/api/pauseactivity/api.go # service/history/api/pollupdate/api_test.go # service/history/api/queryworkflow/api.go # service/history/api/recordactivitytaskstarted/api.go # service/history/api/recordactivitytaskstarted/api_test.go # service/history/api/recordworkflowtaskstarted/api.go # service/history/api/removesignalmutablestate/api.go # service/history/api/resetactivity/api.go # service/history/api/resetworkflow/api.go # service/history/api/respondactivitytaskcanceled/api.go # service/history/api/respondactivitytaskcompleted/api.go # service/history/api/respondactivitytaskfailed/api.go # service/history/api/respondactivitytaskfailed/api_test.go # service/history/api/respondworkflowtaskcompleted/api.go # service/history/api/respondworkflowtaskcompleted/api_test.go # service/history/api/respondworkflowtaskcompleted/workflow_task_completed_handler.go # service/history/api/respondworkflowtaskcompleted/workflow_task_completed_handler_test.go # service/history/api/respondworkflowtaskfailed/api.go # service/history/api/signalwithstartworkflow/api.go # service/history/api/signalwithstartworkflow/convert.go # service/history/api/signalwithstartworkflow/signal_with_start_workflow.go # service/history/api/signalwithstartworkflow/signal_with_start_workflow_test.go # service/history/api/signalworkflow/api.go # service/history/api/startworkflow/api.go # service/history/api/unpauseactivity/api.go # service/history/api/updateactivityoptions/api.go # service/history/api/updateactivityoptions/api_test.go # service/history/api/updateworkflow/api.go # service/history/api/updateworkflowoptions/api.go # service/history/api/updateworkflowoptions/api_test.go # service/history/api/verifychildworkflowcompletionrecorded/api.go # service/history/api/verifyfirstworkflowtaskscheduled/api_test.go # service/history/api/worker_versioning_util.go # service/history/api/workflow_id_dedup.go # service/history/api/workflow_id_dedup_test.go # service/history/archival_queue_factory_test.go # service/history/chasm_engine.go # service/history/chasm_engine_test.go # service/history/chasm_task_util.go # service/history/configs/config.go # service/history/configs/quotas.go # service/history/consts/const.go # service/history/deletemanager/delete_manager.go # service/history/deletemanager/delete_manager_test.go # service/history/fx.go # service/history/handler.go # service/history/history_engine.go # service/history/history_engine2_test.go # service/history/history_engine_factory.go # service/history/history_engine_test.go # service/history/history_test.go # service/history/historybuilder/event_factory.go # service/history/historybuilder/event_store.go # service/history/historybuilder/history_builder.go # service/history/historybuilder/history_builder_categorization_test.go # service/history/historybuilder/history_builder_test.go # service/history/interfaces/chasm_tree.go # service/history/interfaces/chasm_tree_mock.go # service/history/interfaces/mutable_state.go # service/history/interfaces/mutable_state_mock.go # service/history/interfaces/shard_context.go # service/history/interfaces/shard_context_mock.go # service/history/interfaces/workflow_context.go # service/history/interfaces/workflow_context_mock.go # service/history/ndc/events_reapplier_test.go # service/history/ndc/transaction_manager.go # service/history/ndc/transaction_manager_existing_workflow.go # service/history/ndc/transaction_manager_existing_workflow_test.go # service/history/ndc/transaction_manager_new_workflow.go # service/history/ndc/transaction_manager_new_workflow_test.go # service/history/ndc/workflow.go # service/history/ndc/workflow_mock.go # service/history/ndc/workflow_resetter.go # service/history/ndc/workflow_resetter_test.go # service/history/ndc/workflow_state_replicator_test.go # service/history/ndc_standby_task_util.go # service/history/ndc_task_util.go # service/history/outbound_queue_active_task_executor.go # service/history/outbound_queue_active_task_executor_test.go # service/history/outbound_queue_factory.go # service/history/outbound_queue_standby_task_executor.go # service/history/outbound_queue_standby_task_executor_test.go # service/history/queue_factory_base.go # service/history/queue_factory_base_test.go # service/history/queues/active_standby_executor.go # service/history/queues/active_standby_executor_test.go # service/history/queues/dlq_writer.go # service/history/queues/dlq_writer_test.go # service/history/queues/executable.go # service/history/queues/executable_factory.go # service/history/queues/executable_mock.go # service/history/queues/executable_test.go # service/history/queues/grouper.go # service/history/queues/metrics.go # service/history/queues/metrics_test.go # service/history/queues/priority_assigner.go # service/history/queues/queue_base.go # service/history/queues/queue_immediate.go # service/history/queues/queue_immediate_test.go # service/history/queues/reader.go # service/history/queues/scheduler.go # service/history/queues/speculative_workflow_task_timeout_queue.go # service/history/replication/ack_manager.go # service/history/replication/dlq_writer.go # service/history/replication/dlq_writer_test.go # service/history/replication/executable_sync_versioned_transition_task.go # service/history/replication/executable_task.go # service/history/replication/executable_task_converter.go # service/history/replication/executable_task_mock.go # service/history/replication/executable_task_tool_box.go # service/history/replication/executable_verify_versioned_transition_task.go # service/history/replication/fx.go # service/history/replication/metrics.go # service/history/replication/progress_cache.go # service/history/replication/progress_cache_test.go # service/history/replication/raw_task_converter.go # service/history/replication/raw_task_converter_test.go # service/history/replication/stream_receiver.go # service/history/replication/stream_receiver_monitor.go # service/history/replication/stream_receiver_monitor_test.go # service/history/replication/stream_sender.go # service/history/replication/task_processor.go # service/history/replication/task_processor_manager.go # service/history/replication/task_processor_manager_test.go # service/history/replication/task_processor_test.go # service/history/service.go # service/history/shard/context_factory.go # service/history/shard/context_impl.go # service/history/shard/context_test.go # service/history/shard/context_testutil.go # service/history/shard/controller_test.go # service/history/shard/fx.go # service/history/tasks/chasm_task.go # service/history/tasks/delete_workflow_execution_stage.go # service/history/tasks/predicates.go # service/history/tasks/state_machine_task.go # service/history/tasks/task.go # service/history/tasks/task_mock.go # service/history/tasks/utils.go # service/history/timer_queue_active_task_executor.go # service/history/timer_queue_active_task_executor_test.go # service/history/timer_queue_standby_task_executor.go # service/history/timer_queue_standby_task_executor_test.go # service/history/timer_queue_task_executor_base.go # service/history/timer_queue_task_executor_base_test.go # service/history/transfer_queue_active_task_executor.go # service/history/transfer_queue_active_task_executor_test.go # service/history/transfer_queue_factory.go # service/history/transfer_queue_standby_task_executor.go # service/history/transfer_queue_standby_task_executor_test.go # service/history/visibility_queue_factory.go # service/history/visibility_queue_task_executor.go # service/history/visibility_queue_task_executor_test.go # service/history/worker_versioning_util.go # service/history/workflow/activity.go # service/history/workflow/cache/cache.go # service/history/workflow/command_handler.go # service/history/workflow/context.go # service/history/workflow/fx.go # service/history/workflow/metrics.go # service/history/workflow/metrics_test.go # service/history/workflow/mutable_state_impl.go # service/history/workflow/mutable_state_impl_restart_activity_test.go # service/history/workflow/mutable_state_impl_test.go # service/history/workflow/mutable_state_rebuilder.go # service/history/workflow/mutable_state_rebuilder_test.go # service/history/workflow/noop_chasm_tree.go # service/history/workflow/retry.go # service/history/workflow/task_generator.go # service/history/workflow/task_generator_mock.go # service/history/workflow/task_generator_provider.go # service/history/workflow/task_generator_test.go # service/history/workflow/task_refresher.go # service/history/workflow/timer_sequence.go # service/history/workflow/update/abort_reason.go # service/history/workflow/update/errors_failures.go # service/history/workflow/update/export_test.go # service/history/workflow/update/store.go # service/history/workflow/update/store_mock_test.go # service/history/workflow/update/update.go # service/history/workflow/update/update_test.go # service/history/workflow/update/validation.go # service/history/workflow/util.go # service/history/workflow/workflow_task_state_machine.go # service/history/workflow/workflow_test/mutable_state_impl_test.go # service/matching/backlog_manager_test.go # service/matching/config.go # service/matching/configs/quotas.go # service/matching/counter/cmsketch.go # service/matching/counter/cmsketch_test.go # service/matching/counter/hybrid.go # service/matching/db.go # service/matching/fair_backlog_manager.go # service/matching/fair_task_reader.go # service/matching/forwarder.go # service/matching/fx.go # service/matching/handler.go # service/matching/handler_test.go # service/matching/matcher.go # service/matching/matcher_data.go # service/matching/matcher_data_test.go # service/matching/matching_engine.go # service/matching/matching_engine_test.go # service/matching/physical_task_queue_key.go # service/matching/physical_task_queue_key_test.go # service/matching/physical_task_queue_manager.go # service/matching/physical_task_queue_manager_interface.go # service/matching/physical_task_queue_manager_mock.go # service/matching/physical_task_queue_manager_test.go # service/matching/pri_backlog_manager.go # service/matching/pri_forwarder.go # service/matching/pri_matcher.go # service/matching/pri_matcher_test.go # service/matching/pri_task_reader.go # service/matching/task.go # service/matching/task_queue_partition_manager.go # service/matching/task_queue_partition_manager_test.go # service/matching/task_tracker.go # service/matching/task_tracker_test.go # service/matching/task_validation.go # service/matching/user_data_manager.go # service/matching/user_data_manager_test.go # service/matching/workers/registry.go # service/matching/workers/registry_impl.go # service/matching/workers/registry_impl_test.go # service/matching/workers/registry_test.go # service/matching/workers/worker_query_engine.go # service/matching/workers/worker_query_engine_test.go # service/worker/batcher/activities.go # service/worker/batcher/activities_test.go # service/worker/deletenamespace/deleteexecutions/activities.go # service/worker/deletenamespace/deleteexecutions/workflow_test.go # service/worker/deletenamespace/fx.go # service/worker/dummy/workflow.go # service/worker/fx.go # service/worker/migration/activities.go # service/worker/migration/activities_test.go # service/worker/migration/force_replication_workflow.go # service/worker/migration/force_replication_workflow_test.go # service/worker/migration/fx.go # service/worker/migration/handover_workflow.go # service/worker/migration/handover_workflow_test.go # service/worker/scheduler/activities.go # service/worker/scheduler/activities_test.go # service/worker/scheduler/workflow.go # service/worker/workerdeployment/activities.go # service/worker/workerdeployment/client.go # service/worker/workerdeployment/fx.go # service/worker/workerdeployment/util.go # service/worker/workerdeployment/util_test.go # service/worker/workerdeployment/version_activities.go # service/worker/workerdeployment/version_workflow.go # service/worker/workerdeployment/version_workflow_test.go # service/worker/workerdeployment/workflow.go # service/worker/workerdeployment/workflow_test.go # temporal/fx.go # temporal/server_option.go # temporal/server_options.go # tests/acquire_shard_test.go # tests/activity_api_batch_reset_test.go # tests/activity_api_batch_unpause_test.go # tests/activity_api_batch_update_options_test.go # tests/activity_api_pause_test.go # tests/activity_api_reset_test.go # tests/activity_api_rules_test.go # tests/activity_api_update_test.go # tests/activity_test.go # tests/add_tasks_test.go # tests/admin_test.go # tests/advanced_visibility_test.go # tests/archival_test.go # tests/callbacks_migration_test.go # tests/callbacks_test.go # tests/cancel_workflow_test.go # tests/chasm_test.go # tests/child_workflow_test.go # tests/client_data_converter_test.go # tests/client_misc_test.go # tests/continue_as_new_test.go # tests/describe_test.go # tests/dlq_test.go # tests/links_test.go # tests/max_buffered_event_test.go # tests/namespace_interceptor_test.go # tests/namespace_test.go # tests/nexus_api_test.go # tests/nexus_api_validation_test.go # tests/nexus_endpoint_test.go # tests/nexus_test_base.go # tests/nexus_workflow_test.go # tests/pause_workflow_execution_test.go # tests/premature_eos_test.go # tests/priority_fairness_test.go # tests/purge_dlq_tasks_api_test.go # tests/query_workflow_test.go # tests/relay_task_test.go # tests/schedule_migration_test.go # tests/schedule_test.go # tests/signal_workflow_test.go # tests/sizelimit_test.go # tests/stickytq_test.go # tests/task_queue_stats_test.go # tests/task_queue_test.go # tests/testcore/context.go # tests/testcore/dynamic_config_overrides.go # tests/testcore/functional_test_base.go # tests/testcore/matching_behavior.go # tests/testcore/onebox.go # tests/testcore/shard_salt.txt # tests/testcore/test_cluster.go # tests/testcore/test_cluster_pool.go # tests/testcore/test_env.go # tests/transient_task_test.go # tests/update_workflow_sdk_test.go # tests/update_workflow_test.go # tests/user_metadata_test.go # tests/user_timers_test.go # tests/versioning_3_test.go # tests/versioning_test.go # tests/worker_deployment_test.go # tests/worker_deployment_version_test.go # tests/workflow_alias_search_attribute_test.go # tests/workflow_buffered_events_test.go # tests/workflow_delete_execution_test.go # tests/workflow_failures_test.go # tests/workflow_memo_test.go # tests/workflow_task_reported_problems_test.go # tests/workflow_test.go # tests/workflow_timer_test.go # tests/workflow_visibility_test.go # tests/xdc/base.go # tests/xdc/chasm_test.go # tests/xdc/failover_test.go # tests/xdc/history_replication_dlq_test.go # tests/xdc/history_replication_signals_and_updates_test.go # tests/xdc/nexus_request_forwarding_test.go # tests/xdc/nexus_state_replication_test.go # tests/xdc/stream_based_replication_test.go # tests/xdc/visibility_test.go # tools/elasticsearch/handler.go # tools/elasticsearch/main.go # tools/elasticsearch/tasks.go # tools/flakereport/flakereport.go # tools/flakereport/github.go # tools/flakereport/slack.go # tools/flakereport/types.go # tools/flakereport/writer.go # tools/tdbg/chasm_registry.go # tools/tdbg/commands.go # tools/tdbg/flags.go # tools/tdbg/task_queue_commands.go # tools/tdbg/task_queue_commands_test.go # tools/tdbg/tdbg_commands.go # tools/tdbg/util.go # tools/testrunner/junit.go # tools/testrunner/junit_test.go # tools/testrunner/log.go # tools/testrunner/log_test.go # tools/testrunner/testdata/junit-alerts-output.xml # tools/testrunner/testdata/junit-crash-output.xml # tools/testrunner/testdata/junit-empty.xml # tools/testrunner/testdata/junit-timeout-output.xml # tools/testrunner/testrunner.go # tools/testrunner/testrunner_test.go |
||
|
|
94b3a93216 |
Implement SignalWithStart as a system nexus endpoint (#9833)
## What changed? This PR adds `SignalWithStartWorkflowExecution` as a synchronous Nexus operation exposed via `__temporal_system endpoint`, allowing workflows to signal-with-start other workflows through the CHASM Nexus operation framework. Key changes: - `chasm/lib/workflow/nexus_service.go`: `workflowServiceNexusHandler` implements the `SignalWithStartWorkflowExecution` Nexus sync operation by resolving the namespace and delegating to the History service. A `SignalWithStartOperationProcessor` handles input enrichment (namespace, request ID, links) and routing via CHASM's `NexusOperationProcessorResult`. - `chasm/lib/workflow/library.go` — library now holds the `workflowServiceNexusHandler`, the workflow `Config`, the SA mapper provider, and the SA validator. `newLibrary` (used by fx) takes those dependencies, while the public `NewLibrary` keeps its old signature for external callers. Adds `NexusServices()` so the library registers its Nexus service via CHASM. - `chasm/lib/workflow/validator.go`: `RequestValidator` consolidates the `SignalWithStartWorkflowExecution` validation logic (previously inlined in `WorkflowHandler`) into a reusable, injectable struct. This same validator is used by both the frontend handler and the new CHASM processor. - common/dynamicconfig/constants.go — adds `EnableSignalWithStartFromWorkflow` (namespace-scoped, default false). - `service/frontend/workflow_handler.go`: Removed the `SignalWithStart` validation block - `service/history/fx.go`: Provides a `HistoryServiceServerProvider` so the CHASM workflow library can call the history handler directly. - `temporal/fx.go`: Removes the now-redundant `ChasmLibraryOptions` grouping; each service module registers its own CHASM libraries. - `components/nexusoperations/workflow/commands.go`: `NotFound` and `InvalidArgument` errors during Nexus command handling are now surfaced as workflow task failures instead of being treated as transient handler errors. - `common/payloads`: Adds `EncodeSingle`, `MustEncodeSingle`, and `MustEncode` helpers used in tests. - `cmd/tools/getproto`: Adds support for nexus-proto-annotations proto imports. - `tests/signal_with_start_from_workflow_test.go`: Functional test suite covering the happy path, duplicate detection, conflict policies, and validation rejection for the new Nexus operation. ## Why? This functionality is one of our most requested GitHub issues. ## How did you test it? - [X] built - [X] run locally and tested manually - [ ] covered by existing tests - [X] added new unit test(s) - [X] added new functional test(s) ## Potential risks The history service now directly exposes a `HistoryServiceServer` interface via `fx` for injection into the CHASM workflow library. This tight coupling between the CHASM workflow library and the history handler could complicate future layering — callers outside the history service should not adopt this pattern. The feature is gated by `history.enableSignalWithStartFromWorkflow` for rollout. --------- Co-authored-by: Roey Berman <[email protected]> |
||
|
Yichao YangandGitHub
|
0c815e410e |
Codegen for routingKey extractor (#9836)
## What changed? - Codegen to automatically generate code for extracting businessID for workflowservice methods based on "temporal-resouce-id" proto option ## Why? - With codegen, any new API will automatically have the logic for businessID extraction and avoids the case where businessID extractor logic forget to be updated. ## How did you test it? - [x] built - [ ] run locally and tested manually - [x] covered by existing tests - [x] added new unit test(s) - [ ] added new functional test(s) |
||
|
zeekay
|
45bd80a26d |
feat(tasks): v3.9.0 — distributed engine: per-NS SQLite shards + PQ consensus + sticky routing
Architecture
One SQLite shard per (org, namespace) under <DataDir>/<org>/<ns>.db
with WAL mode, single writer, 5s busy timeout. Replaces the prior
shared memdb/zapdb store. Shard manager opens/caches/idle-evicts;
prefix scans, namespace isolation, and cross-restart durability all
proven by tests.
Replication (post-quantum, native)
New pkg/tasks/replication/ exposes a Replicator interface with two
drivers:
local — single-node passthrough (default for embedded/dev)
quasar — wraps github.com/luxfi/consensus NewPQ (BLS + Corona
dual-threshold). On every WAL-affecting op the shard
builds a Frame, calls Propose, awaits Decision, applies
on Accept. Followers receive accepted frames via the
same Subscribe path so every node converges.
Sticky task-queue routing
New pkg/tasks/routing/ — rendezvous-style smallest-hash leader
election keyed by SHA-256(shard|taskQueue|seed). OnChange fan-out
re-hashes on cluster membership change. Dispatcher consults Router
before accepting subscriptions and forwards non-leader requests.
Namespace migration
New pkg/tasks/migration/ — lock → checkpoint → copy → unlock with
a job lifecycle (pending|locking|copying|replaying|released|failed).
CLI: `tasksd migrate --org <id> --namespace <name> --to <node>`.
HTTP: POST /v1/tasks/namespaces/{ns}/migrate.
New env / flags
--data / TASKSD_DATA_DIR shard root (./tasks-data)
--replicator / TASKSD_REPLICATOR local | quasar (default local)
--node-id / TASKSD_NODE_ID stable id (required for quasar)
--validators / TASKSD_VALIDATORS CSV host:port
--validator-key / TASKSD_VALIDATOR_KEY (reserved for PQ key path)
TASKSD_STORE sqlite | memory (default sqlite)
New routes
GET /v1/tasks/cluster
GET /v1/tasks/cluster/health
POST /v1/tasks/namespaces/{ns}/migrate
UI bundle (synced)
Cluster page with status / validators / shards / leader-map.
NamespaceDetail "Migrate to node" button gated by
canWriteNamespace + cluster.enabled. MigrationDialog enforces
typed-name confirmation. Sidebar Cluster nav hidden when running
single-node.
Tests
+33 backend tests (store: 11, replication: 10, routing: 5,
migration: 3, cluster routes: 2, store factory: 2). All packages
PASS with -race; go vet clean.
+13 UI tests (effectiveHealth: 6, canSubmitMigration: 4,
useCluster: 3). 169 total UI tests, 1 pre-existing skip.
Migration v3.8.x → v3.9.0
Fresh-data startup. zapdb backend dropped. Operators drain old
cluster, snapshot for archival, deploy v3.9.0 with new --data
directory. To enable HA, scale to 3+ replicas with --replicator
quasar and matching --node-id / --validators.
|
||
|
zeekay
|
056a57ba71 | fix(tasksd): /v1/tasks/settings unauthenticated for SPA bootstrap | ||
|
zeekay
|
1444d61c56 |
feat(tasks): v3.7.0 — multi-tenant hardening, durable store, full inverse-options, 13 new endpoints
Security (red-team pass)
- ZAP wire handlers were org-blind: every opcode bypassed WithOrg
and used the unscoped engine. Patched: zapHandlers takes a
JWTValidator; per-request engine is scoped to claims.Owner.
TASKSD_REQUIRE_IDENTITY enforces validation in production.
- /v1/tasks/mcp tools/call ran on the unscoped engine — patched
to use auth.OrgID(r.Context()).
- /v1/tasks/events SSE fan-out was global; events now stamp OrgID
via engine.emit(ev) and the SSE handler filters per-request org.
- AddSearchAttribute rejected reserved/system names (WorkflowId,
ExecutionStatus, TaskQueue, *StartTime, Temporal*).
- Path-traversal guards on every {ns}/{id} URL slot (validIdent +
validPathSegment).
- dispatcher.queries map is now bounded (4096 entries, 5m TTL,
drop-oldest on overflow) — closes a DoS via silent worker.
Durability
- newStoreFromEnv selects memdb (default) or zapdb via
TASKSD_STORE=zapdb. Survives restart; covered by
TestStore_ZapdbPersistsAcrossOpen.
Inverse-of-options
- RevokeIdentity, DeleteNexusEndpoint, RemoveSearchAttribute,
DeprecateNamespace, DeleteDeploymentVersion, UnsetCurrent.
- Schedule full update + trigger + matching-times.
13 new HTTP routes
GET /v1/tasks/settings
DELETE /v1/tasks/namespaces/{ns}
POST /v1/tasks/namespaces/{ns}/schedules/{id} (full update)
POST /v1/tasks/namespaces/{ns}/schedules/{id}/trigger
GET /v1/tasks/namespaces/{ns}/schedules/{id}/matching-times
POST /v1/tasks/namespaces/{ns}/deployments/{name}/set-current
DELETE /v1/tasks/namespaces/{ns}/deployments/{name}/versions/{buildId}
GET /v1/tasks/namespaces/{ns}/archival
GET /v1/tasks/namespaces/{ns}/workflows/{id}/executions
GET /v1/tasks/namespaces/{ns}/task-queues/{q}/partitions
GET /v1/tasks/namespaces/{ns}/workers/{identity}
DELETE /v1/tasks/namespaces/{ns}/nexus/{id}
DELETE /v1/tasks/namespaces/{ns}/identities/{email}
DELETE /v1/tasks/namespaces/{ns}/search-attributes/{name}
Tests
go test ./pkg/tasks/... -race -count=1 → PASS, +22 new tests
(cross-org keyspace isolation, query non-widening, reserved attrs,
pendingQuery bounds, TTL eviction, Identifier validation, persistence
cross-restart). go vet clean.
UI bundle (synced)
Canonical Hanzo H mark in HanzoMark.tsx replaces stylized H rectangle.
useSettings() hook gates write actions when namespaceWriteDisabled,
archival/heartbeats/advanced-visibility flags. New inverse buttons
(worker stop, identity revoke, search-attr delete, deployment unset,
batch terminate confirm).
|
||
|
zeekay
|
fdb4067b4c |
feat(auth): JWT validation against IAM JWKS, strip+mint identity
tasksd is now self-sufficient multi-tenant: no gateway dependency required. Every inbound request has client-supplied X-Org-Id / X-User-Id / X-User-Email stripped unconditionally; if Authorization: Bearer <jwt> is present, the token is validated against the configured JWKS (TASKSD_JWKS_URL), checked against issuer/audience, and identity headers are minted from validated claims. Closes the spoofable-identity hole on tasks-api.hanzo.ai: any request with a self-set X-Org-Id previously passed through and got that org's scoped engine. Env: TASKSD_JWKS_URL e.g. https://hanzo.id/.well-known/jwks TASKSD_JWT_ISSUER e.g. https://hanzo.id TASKSD_JWT_AUDIENCE optional TASKSD_REQUIRE_IDENTITY=true rejects unauthenticated requests |
||
|
zeekay
|
468107f0c9 |
feat(tasksd): wire RequireIdentity onto /v1/tasks/ surface
/healthz, /v1/tasks/health, /_/tasks/* stay public — probes and SPA shell run before any session exists. /v1/tasks/, /v1/tasks/mcp, /v1/tasks/events go through auth.RequireIdentity, gated by TASKSD_REQUIRE_IDENTITY (default false). |
||
|
zeekay
|
0b79daded5 |
feat: MCP server + SSE realtime event stream
Two new surfaces on top of the engine, both wire-compatible with the
canonical ZAP transport (their opcodes are reserved for native ZAP
adoption — HTTP today, ZAP tomorrow without breaking clients).
pkg/tasks/mcp.go — Model Context Protocol JSON-RPC 2.0 endpoint at
POST /v1/tasks/mcp. Methods: initialize, tools/list, tools/call, ping.
Tool catalogue:
list_namespaces / describe_namespace
list_workflows / describe_workflow / start_workflow /
signal_workflow / cancel_workflow / terminate_workflow
list_schedules / create_schedule / pause_schedule
Every tool maps onto an engine method with the SDK's snake_case wire
shape. AI agents can now drive Hanzo Tasks end-to-end via stdio MCP
clients (Claude Desktop, agent, Cursor, etc.) by pointing them
at /v1/tasks/mcp.
pkg/tasks/events.go — bounded fan-out broker + GET /v1/tasks/events
SSE stream. Every state-changing engine method publishes an Event:
namespace.registered
workflow.started / workflow.canceled / workflow.terminated /
workflow.signaled
schedule.created (more transitions in v1.41)
Slow subscribers drop the oldest event rather than backing up the
publisher. Hello-frame on connect, ctx.Done() unsubscribes on disconnect.
cmd/tasksd mounts both at fixed paths next to the existing /v1/tasks/*
shim. Binary 7.6 MB, integration tests still 4/4 PASS via ZAP.
|
||
|
zeekay
|
30e70000eb |
feat: native engine + @hanzo/ui shell (full Temporal Cloud parity)
Engine (pkg/tasks):
- store.go — KV layer over luxfi/database memdb (snapshot-and-iterate
list to avoid RWMutex upgrade deadlock)
- types.go — wire types: Namespace, WorkflowExecution, Schedule,
BatchOperation, Deployment, NexusEndpoint, Identity
- engine.go — domain ops: register/list/describe ns; start/list/
describe/cancel/terminate/signal workflow; create/list/
describe/delete/pause/unpause schedule; start/list/
describe batch; create/list deployment; create/list
nexus endpoint; grant/list identity; cron sweeper goroutine
- embed.go — every ZAP opcode (0x0060-0x0090) + every HTTP route
dispatches through the same model funcs; no drift
HTTP surface (browser-only JSON shim, ZAP is the canonical wire):
GET /v1/tasks/namespaces
POST /v1/tasks/namespaces
GET /v1/tasks/namespaces/{ns}
GET /v1/tasks/namespaces/{ns}/workflows
POST /v1/tasks/namespaces/{ns}/workflows
GET /v1/tasks/namespaces/{ns}/workflows/{id}
POST /v1/tasks/namespaces/{ns}/workflows/{id}/cancel
POST /v1/tasks/namespaces/{ns}/workflows/{id}/terminate
POST /v1/tasks/namespaces/{ns}/workflows/{id}/signal
GET /v1/tasks/namespaces/{ns}/schedules
POST /v1/tasks/namespaces/{ns}/schedules
GET /v1/tasks/namespaces/{ns}/schedules/{id}
DELETE /v1/tasks/namespaces/{ns}/schedules/{id}
POST /v1/tasks/namespaces/{ns}/schedules/{id}/pause
POST /v1/tasks/namespaces/{ns}/schedules/{id}/unpause
GET /v1/tasks/namespaces/{ns}/batches
POST /v1/tasks/namespaces/{ns}/batches
GET /v1/tasks/namespaces/{ns}/batches/{id}
GET /v1/tasks/namespaces/{ns}/deployments
POST /v1/tasks/namespaces/{ns}/deployments
GET /v1/tasks/namespaces/{ns}/nexus
POST /v1/tasks/namespaces/{ns}/nexus
GET /v1/tasks/namespaces/{ns}/identities
POST /v1/tasks/namespaces/{ns}/identities
UI (full Temporal Cloud parity):
- @hanzo/ui primitives in src/components/ui/{button,card,tabs,badge,
alert,dialog,input,label,skeleton,dropdown-menu}.tsx — same shadcn
pattern @hanzo/ui itself uses; CSS tokens from @hanzo/ui design system
- App.tsx: sidebar layout (Workflows/Schedules/Batches/Deployments/
Namespaces/Nexus/Support) with top bar (UTC clock, theme toggle,
account chip, ns indicator, embedded badge)
- pages/Namespaces.tsx + pages/NamespaceDetail.tsx — list + detail w/
Overview+Identities tabs, Region/Retention/APS cards, Connect
dropdown showing ZAP+HTTP endpoints with copy buttons, Active badge
- pages/Workflows.tsx — list + Start Workflow dialog (real POST)
- pages/WorkflowDetail.tsx — execution details
- pages/Schedules.tsx — list w/ cron / interval describe
- pages/Batches.tsx — list + Start Batch dialog (operation enum)
- pages/Deployments.tsx — list of worker version series
- pages/Nexus.tsx — list of cross-ns endpoints
- pages/Support.tsx — 4 cards linking to docs / GitHub / health
- ThemeToggle (light/dark, persisted), UTCClock (live ticker),
AccountChip (popover w/ identity + sign out)
Binary: 7.5 MB. Workflow execution semantics (deterministic replay of
user code) still pending — engine records state and tracks history len
on signals, but does NOT run user-supplied workflow goroutines yet.
Worker SDK runtime is the next phase.
|
||
|
zeekay
|
3aabbe5ef1 |
feat(http): wire /v1/tasks/* JSON shim, mirror ZAP opcodes
The browser can only speak HTTP/JSON; ZAP is binary. Both transports now dispatch to the same model functions in pkg/tasks so they cannot drift. - pkg/tasks/embed.go: extract per-opcode model() funcs; ZAP handlers and HTTPHandler() both call them. opcodes 0x0060/0x0065/0x0072/0x0081/ 0x0082 return real JSON (empty/default state); the rest still return 501 until the native engine lands. - cmd/tasksd: mount srv.HTTPHandler() at /v1/tasks/namespaces[/...] so the embedded React UI's useSWR fetches succeed. UI verified end-to-end with Playwright (headless Chromium): / → 302 → /_/tasks/namespaces (clean redirect) Namespaces tab: "default" registered, retention 720h Workflows tab: "No running or recent workflows" empty state Schedules tab: "No schedules defined" empty state console.error count: 0 failed requests: 0 Binary 6.99 MB (was 6.65; +340 KB for the shim and model funcs). |
||
|
zeekay
|
26a782c911 |
fix(http): redirect / to /_/tasks/
The Vite bundle is built with React Router basename="/_/tasks", so the bundle only renders under that prefix. Serving it at "/" loaded the HTML but the router refused to render — blank page in browser. GET / → 302 → /_/tasks/ GET /_/tasks/ → 200 (UI) GET /_/tasks/foo → 200 (SPA route) GET /healthz → 200 GET /v1/tasks/health → 200 |
||
|
zeekay
|
aa83a0a1d2 |
rip: delete temporal fork — native ZAP-only tasksd
Goes from 135 MB binary with 1153 files importing go.temporal.io and 224
importing google.golang.org/grpc to a 10 MB native binary with ZERO of
either.
Deleted:
- tasks/, tasktests/, service/, chasm/, client/, api/, proto/, tools/
- cmd/tools/, components/, docker/
- schema/{cassandra,elasticsearch,mysql,postgresql,sqlite}/
- 40 tainted common/ subdirs + 7 top-level common files
- tests/testutils (no longer referenced after surviving deps drop common)
New cmd/tasksd (~100 lines) boots tasks.Embed (zap.Node) + http.Server.
New pkg/tasks/embed.go registers ZAP handlers for the SDK opcode surface
(0x0060-0x00A5). Workflow handlers return a 501 "not yet implemented"
envelope; native engine lands behind this contract.
Build: go build ./cmd/tasksd → 10 MB
Vet: go vet ./... → clean
Boot: tasksd --zap-port 9999 --http :7243
Probes: /healthz, /v1/tasks/health → 200
UI: /_/tasks/, / → 200 (embedded Vite bundle)
ZAP: _tasks._tcp on :9999 (mDNS disabled; direct connect)
go.mod: tidy. No go.temporal.io. No google.golang.org/grpc.
|
||
|
zeekay
|
d5d67f8284 | tasksd: drop stdlib log in bootstrap CLI — use hanzoai/tasks/common/log | ||
|
zeekay
|
a10bde2005 |
fix(embedded-sqlite): bootstrap schema + expose HTTP API on first boot
The plain `start` command on tasksd does not run sqlite/v1 schema migrations, so a fresh /data PVC crashes with "no such table: cluster_metadata_info". Add a new `embedded-sqlite` command that wraps temporaltest/embedded.LiteServer, which calls sqlite.SetupSchema() before binding the frontend. Also expose the HTTP API by default on port 7234 (gRPC stays on 7233). Without HTTPPort the frontend is gRPC-only and clients that POST workflows/schedules over HTTP get connection refused. Image v2.0.2. |
||
|
Alex StanfieldandGitHub
|
6c3e5559bf |
Rewrite dependency version check as Go tool, extend to main branch (#9816)
Rewrites the release dependency check as a Go tool and extends it to cover the main branch. The original check was a shell script using `grep`/`awk` to extract versions from `go.mod`. Moving to Go lets us use `golang.org/x/mod/modfile` (proper AST parsing), `module.IsPseudoVersion`/`PseudoVersionRev` (pseudo-version decomposition), and `semver.IsValid` — none of which are feasible to replicate reliably in shell. Using a Go tool is also consistent with the pattern in `cmd/tools/`. The rewrite also extends validation: the original script only enforced tagged releases on `release/*` and `cloud/*` branches. The new tool adds a `main` branch policy: pseudo-versions are allowed on main, but the referenced commit must be on the dependency's default branch (not a feature branch or a fork). ## Policies enforced - `release/*` and `cloud/*`: must be tagged semver releases - `main`: tagged releases accepted; pseudo-versions must reference a commit on the dependency's default branch - other branches: skipped ## Why If an API or SDK references a commit that's not on the main branch or a tag, it creates problems when bumping the version later on. There was a recent occurrence of this. ## Running locally ``` go run ./cmd/tools/check-dependencies --base-branch main ``` Pass the branch you're targeting as `--base-branch`. For example, to simulate a PR against a release branch: ``` go run ./cmd/tools/check-dependencies --base-branch release/v1.31 ``` |
||
|
zeekay
|
760c29bb8d |
rebrand(env): rename all TEMPORAL_* env vars to TASKS_*
- common/config/loader.go: EnvKey* constants now TASKS_ROOT, TASKS_CONFIG_DIR, TASKS_ENVIRONMENT, etc. - cmd/tasksd/main.go: --services flag EnvVars → TASKS_SERVICES - tests/testutils: source root lookup uses TASKS_ROOT - config/docker.yaml + common/config/config_template_embedded.yaml: all TEMPORAL_JWT_*, TEMPORAL_AUTH_*, TEMPORAL_TLS_*, TEMPORAL_BROADCAST_ADDRESS → TASKS_* - k8s/deployment.yaml: env var names updated - LLM.md: env var table updated; dropped "TEMPORAL_* prefix preserved" note No backwards compatibility. Existing clusters must migrate env vars. go build ./cmd/tasksd/... clean. |
||
|
Fred TzengandGitHub
|
79c8e8d5c8 |
Add callback support for standalone activities (#9786)
## What changed? Added completion callback support to standalone activities: - Callback lifecycle: When a standalone activity reaches a terminal state (completed, failed, canceled, terminated, timed out), it now fires any registered Nexus completion callbacks — the same mechanism workflows already use. - Frontend validation: Callback URL length, header size, and endpoint allowlist validation is now shared between workflows and standalone activities via callbacks.ValidateCallbacks, refactored out of the workflow handler. - Describe response: DescribeActivityExecution now returns callback state (CallbackInfo) including trigger, status, attempt count, and failure details. - Start response: StartActivityExecutionResponse now includes a Link_Activity_ identifying the started (or reused) activity. - Config: Renamed MaxCHASMCallbacksPerWorkflow to MaxCallbacksPerExecution since it now applies to both workflows and standalone activities. ## Why? The v2 scheduler needs to start standalone activities on a schedule and be notified when they complete, so it can track action results, handle overlap policies, and support pause-on-failure. Workflows already have this via CompletionCallbacks + Nexus callback delivery. This PR gives standalone activities the same capability, reusing the existing callback library rather than reimplementing the delivery/retry/backoff logic. This is a prerequisite for the scheduler's Invoker to call StartActivityExecution with a callback pointing back to the Scheduler component. ## How did you test it? - [X] built - [X] run locally and tested manually - [X] covered by existing tests - [X] added new unit test(s) - [X] added new functional test(s) |
||
|
Shahab TajikandGitHub
|
4ef03935f8 |
Proper load balancing for Nexus tasks (#9397)
## What changed? Enhance matching clients to select random partition for Nexus poll and dispatch requests. ## Why? Before this change all Nexus matching request were going to the root partition only. ## How did you test it? - [ ] built - [ ] run locally and tested manually - [ ] covered by existing tests - [ ] added new unit test(s) - [x] added new functional test(s) ## Potential risks None |
||
|
|
f484359653 |
Serverless Feature Integration (#9779)
This PR merges the serverless feature branch into main. Individual PRs included in this branch: - #9380 - #9651 - #9412 - #9746 - #9759 --------- Co-authored-by: Stefan Richter <[email protected]> Co-authored-by: Hanzo Dev <[email protected]> |
||
|
zeekay
|
5a601d081b |
rebrand: go.temporal.io/server -> github.com/hanzoai/tasks
Full module path rebrand from Temporal to Hanzo Tasks: - go.mod module: github.com/hanzoai/tasks - All internal Go imports updated (1906 files) - cmd/server renamed to cmd/tasksd - Binary name: tasksd (was temporal-server) - Docker images: hanzoai/tasks (was temporalio/server) - README rewritten with Hanzo Tasks branding - LLM.md created with integration notes - Makefile, goreleaser, CI workflows updated - go.temporal.io/sdk and go.temporal.io/api imports preserved (separate repos) Fork of Temporal.io (MIT license). |
||
|
Stephan BehnkeandGitHub
|
4ea9105f9c |
OperationID, ChasmRunID and ActivityID log tags in interceptor (#9498)
## What changed? Extend genrpcserverinterceptors to support `ActivityID` and `OperationID` and `ChasmRunID` ## Why? Fix and improve logging. ## How did you test it? - [ ] built - [ ] run locally and tested manually - [x] covered by existing tests - [ ] added new unit test(s) - [ ] added new functional test(s) |
||
|
Alan WuandGitHub
|
39597a88de |
Add chasm task type dynamic config filter and standby task discard delay dynamic config flag (#9506)
## What changed? Add chasm logical task type dynamic config filter and standby task discard delay dynamic config flag. ## Why? Standby tasks have a configurable dynamic config (per task type) that specifies the timeout for discarding the task and running the post-discard function. Currently, this is not configurable for CHASM logical tasks. By default, CHASM tasks need a higher discard timeout since they are not regenerated from pending tasks in MS, which can lead to abandoned tasks. For tasks that can be safely dispatched to Matching, they can be configured with a lower value. (See Standalone activity tasks). ## How did you test it? - [X] built - [X] run locally and tested manually - [X] covered by existing tests - [ ] added new unit test(s) - [ ] added new functional test(s) |
||
|
feiyangandGitHub
|
3dc376d335 |
dynamic config: enhance update with metrics etc (#9366)
## What changed? 1) added metric(gauge) when update fails 2) change lastUpdatedTime to lastCheckedTime and not update it for possible transient errors ## Why? may be a root cause why pods run into crashloop adding this metric for alert make debugging easier ## How did you test it? - [x] built - [x] run locally and tested manually - [x] covered by existing tests - [x] added new unit test(s) - [ ] added new functional test(s) |
||
|
David ReissandGitHub
|
e0e021d144 |
Add matching fairness simulator (#8158)
## What changed? Add a new cli tool to run the matching fairness simulator, and some tests for basic fairness behavior. ## Why? So users can tell how it will behave for their workloads. |
||
|
Sean KaneandGitHub
|
42ba856e72 |
Convert flake report from Python to Go (#9334)
## What changed? * Convert the flake report we use to monitor functional test reliability from invoking the `tringa` cli tool and a Python script to only a Go script that fetches the artifacts, parses the JUnit, and creates the report. * This also fixes an issue where `tringa` only found the last 20 tests, the Go script will iterate through all failures in the last `N` days ## Why? Everything is in a single executable making it easier to test and extend. ## How did you test it? - [X] built - [X] run locally and tested manually - [ ] covered by existing tests - [X] added new unit test(s) - [ ] added new functional test(s) ## Potential risks Minimal, this is only a tooling change. |
||
|
Stephan BehnkeandGitHub
|
f496b52b64 |
Parallelize integration tests (#9292)
## What changed? Made integration tests run in parallel. ## Why? Before: ~8min [[run](https://github.com/temporalio/temporal/actions/runs/21930252400/job/63333789136#step:7:1)] 🐢 After: ~3m [[run](https://github.com/temporalio/temporal/actions/runs/22114061618/job/63917852614?pr=9292)] 🐰 ## How did you test it? - [ ] built - [ ] run locally and tested manually - [x] covered by existing tests - [ ] added new unit test(s) - [ ] added new functional test(s) ## Potential risks They are not known to be flaky; and anecdotally all passed on the first run. |
||
|
Stephan BehnkeandGitHub
|
c9e360a3fb |
go fix (#9337)
## What changed? Integrate Go 1.26's new `go fix` into workflow. NOTE that the changes caused our linter to fire; a [separate commit](https://github.com/temporalio/temporal/pull/9337/commits/1b23f787aef370098b35829c4d17dcc06230b4c1) addresses those. ## Why? Ensure Go code is standardized/modernized. ## How did you test it? - [ ] built - [ ] run locally and tested manually - [x] covered by existing tests - [ ] added new unit test(s) - [ ] added new functional test(s) |
||
|
Roey BermanandGitHub
|
b91c43893b |
System nexus endpoint (#9002)
## Overview
This feature introduces a special internal Nexus endpoint called
`__temporal_system` that enables adding functionality to workflows
**without requiring new workflow commands and events**. Operations on
this endpoint are routed internally within Temporal's history service
rather than via external HTTP calls.
## Key Components
### 1. System Endpoint Infrastructure
- **Endpoint Name**: `__temporal_system` (constant in
`common/nexus/constants.go:6`)
- **Callback URL**: `temporal://system` for internal routing
- **New History Service RPCs**
(`proto/internal/temporal/server/api/historyservice/v1/service.proto:433-437`):
- `StartNexusOperation` - Starts operations on the system endpoint
- `CancelNexusOperation` - Cancels operations on the system endpoint
### 2. Operation Processor Framework
A new processor pattern (`chasm/nexus_operation_processor.go`) that
allows CHASM libraries to:
- **Validate and transform input**: Processors can validate operation
inputs and set default values
- **Determine routing**: Each processor returns a routing key that
determines which history shard handles the operation
- **Re-serialize input**: Mutated inputs can be re-serialized to persist
default values
**Routing strategies**:
- `NexusOperationRoutingKeyExecution` - Routes to the shard owning a
specific workflow execution
- `NexusOperationRoutingKeyRandom` - Routes to a random shard
### 3. CHASM Library Integration
CHASM libraries can now provide (`chasm/library.go:16-19`):
- **`NexusServices()`**: Regular Nexus service handlers (implement the
actual operation logic)
- **`NexusServiceProcessors()`**: Input processors for validation and
routing
Example from test library (`chasm/lib/tests/nexus_service.go`):
```go
// Service handler - implements the actual operation
TestOperation = nexus.NewSyncOperation("TestOperation",
func(ctx context.Context, input string, options nexus.StartOperationOptions) (string, error) {
return "Hello, " + input, nil
})
// Processor - validates input and determines routing
func (o testOperationProcessor) ProcessInput(ctx chasm.NexusOperationProcessorContext, input string)
(*chasm.NexusOperationProcessorResult, error) {
return &chasm.NexusOperationProcessorResult{
RoutingKey: chasm.NexusOperationRoutingKeyExecution{
NamespaceID: ctx.Namespace.ID().String(),
BusinessID: input, // Route based on input
},
}, nil
}
```
### 4. Execution Flow
When a workflow schedules a Nexus operation on `__temporal_system`
(`components/nexusoperations/executors.go:233-238`):
1. **Input Processing**: The processor validates input and determines
routing
2. **Internal RPC**: Instead of HTTP, calls
`HistoryClient.StartNexusOperation` with the target shard ID
3. **Handler Execution**: The history service invokes the registered
Nexus handler (`service/history/handler.go:2707-2768`)
4. **Result Handling**: Supports both sync (immediate result) and async
(operation token) responses
5. **Workflow Completion**: Results flow back through the same
completion path as external Nexus operations
### 5. Benefits
✅ **No schema changes**: Add functionality without new commands/events
in workflow history
✅ **Consistent API**: Uses existing Nexus operation semantics
(sync/async, callbacks, links)
✅ **Proper routing**: Operations are intelligently routed to the correct
shard
✅ **Input validation**: Type-safe input validation and default value
handling
✅ **Future extensibility**: Foundation for direct client invocation (not
yet implemented)
### 6. Technical Details
- **Error handling** (`components/nexusoperations/executors.go:444`):
Non-retryable service errors are properly handled and fail operations
immediately
- **Metrics**: System operations are tracked separately with
`DestinationTag` set to the endpoint name
- **Link conversion**: Helper functions convert between Nexus SDK links
and protobuf links (`common/nexus/util.go:17-46`)
- **Operation token handling**: Moved link converters to common package
for reuse (`common/nexus/link_converter.go`)
### 7. Current Limitations
- Only accessible from workflows (via `ScheduleNexusOperation` command)
- Direct client invocation not yet implemented
- Headers not supported for system endpoint operations
## Test Coverage
New test (`tests/nexus_workflow_test.go:2763-2843`) demonstrates:
- Scheduling operation on `__temporal_system` endpoint
- Synchronous operation completion
- Result propagation back to workflow
## Architecture
This architecture provides a clean, extensible way to add internal
functionality while maintaining compatibility with Temporal's existing
workflow execution model. The system endpoint acts as a bridge between
workflows and internal CHASM components, enabling:
- **Extensibility**: New operations can be added by implementing CHASM
libraries
- **Type safety**: Input validation happens before operations are routed
- **Scalability**: Intelligent routing ensures operations land on the
correct shard
- **Consistency**: Same execution model as external Nexus operations
|
||
|
Stephan BehnkeandGitHub
|
565cb54173 |
Move sharding salt to file (#9265)
## What changed? Instead of storing the test sharing salt in a repo var, store it in a file. ## Why? https://github.com/temporalio/temporal/pull/9236 tried to use the repo's variable for storing the test sharding salt; but that is not accessible to PRs from forks. ## How did you test it? - [ ] built - [ ] run locally and tested manually - [x] covered by existing tests - [ ] added new unit test(s) - [ ] added new functional test(s) ## Potential risks PRs could fail due to flaky tests. But then the next run will delete the branch (thereby closing the PR) and open a new one. |
||
|
Stephan BehnkeandGitHub
|
b547beb342 |
Auto balance shards (#9236)
## What changed? Adds script and workflow to auto-update functional test sharding salt. It downloads the previous JUnit XMLs and finds a salt that balances them well. Also added a new variable to store the salt: <img width="812" height="64" alt="Screenshot 2026-02-05 at 5 59 42 PM" src="https://github.com/user-attachments/assets/27316baa-03d7-4d0f-8700-9ead9cf322e8" /> ## Why? Balanced test run shards lead to lower PR test time. ## How did you test it? - [ ] built - [x] run locally and tested manually - [ ] covered by existing tests - [ ] added new unit test(s) - [ ] added new functional test(s) AFAIK there's no way to run the new GitHub action before merging ... but I ran the script locally successfully. ## Potential risks Given that we have some flakiness, the PR to update it might fail; requiring human intervention. |
||
|
Tim Deeb-SwihartandGitHub
|
2f60e641e4 |
chore: update code to use new log aliases where applicable (#9177)
## What changed? This updates the server code to use the shorthand log tag constructors introduced in #9174. As part of this it _does_ make a breaking change to the `Bool` constructor: it now takes in the key as a string to be consistent. The only inconsistent one is now `Error`, but that's used so heavily that changing it is likely not worth the time. ## Why? Consistency! ## How did you test it? Existing tests ## Potential risks The only risk is that I _have_ introduced a breaking change to the Bool constructor. I'm happy to undo that if my reviewers desire: my goal is minimal breaking changes. I'd prefer none, but I made this change to stir up discussion |
||
|
|
21db945e7d |
Send raw history events from matching to frontend service (#8829)
## What changed? This PR extends the raw history optimization to pass raw history bytes from History Service → Matching Service → Frontend without deserialization in Matching Service. **Key changes:** 1. History Service: - When `SendRawHistoryBetweenInternalServices` is enabled, sets `RawHistoryBytes` (field 21) with raw proto-encoded history batches 2. Matching Service: - Passes raw history bytes through to frontend via `PollWorkflowTaskQueueResponseWithRawHistory` - Uses wire-compatible proto messages so gRPC auto-deserializes `[][]byte` → `History` on the client side 3. Frontend: - Receives raw history in `RawHistory` field (auto-deserialized by gRPC) - Processes search attributes for raw history since it bypasses history service's normal processing 4. Proto definitions: - Added `raw_history_bytes` (field 21) to `RecordWorkflowTaskStartedResponse` - Added `PollWorkflowTaskQueueResponseWithRawHistory` message with wire-compatible layout - Added `raw_history` (field 22) to `PollWorkflowTaskQueueResponse` ## Why? When `history.sendRawHistoryBetweenInternalServices` is enabled, the previous implementation only avoided deserialization from persistence → History Service. However, Matching Service was still deserializing history events (via gRPC auto-deserialization) and re-serializing them when forwarding to Frontend. This change eliminates that unnecessary serialization/deserialization cycle in Matching Service by: 1. Having History Service send raw bytes directly 2. Having Matching Service forward these raw bytes without parsing 3. Having Frontend receive the bytes which gRPC auto-deserializes This reduces CPU usage in Matching Service for workflows with large histories. ## How did you test it? - [x] built - [x] covered by existing tests - [x] added new unit test(s) - [x] added new functional test(s) (`tests/workflow_task_test.go`) ## Potential risks SendRawHistoryBetweenInternalServices must be disabled when rolling back from this version to an older version. --------- Co-authored-by: Hanzo Dev <[email protected]> |
||
|
Roey BermanandGitHub
|
06aacf068b |
Fix routing by task token component ref (#9061)
## What changed? Routing logic did not take into account that task token may not have a workflow ID and instead have a component ref. |
||
|
Jacob MoodyandGitHub
|
7d1a144404 |
auto enabling priority and fairness (#8650)
## What changed? Add a new dynamic config for Auto Enabling fairness and priority if we see the relevant tasks coming in. ## Why? Seamlessly start to transition users who start using the fields over to the new code path. ## How did you test it? - [X] built - [x] run locally and tested manually - [x] covered by existing tests - [x] added new unit test(s) - [x] added new functional test(s) ## Potential risks Due to storing this in the userdata we are using that interface a bit more, we also need to change the initialization such that we start it before being able to substantiate the defaultQ, this change in initialization might have unintended side effects that I'm not currently seeing. |
||
|
Dan Davison
|
90dc3eea5d |
Update standalone activity protos (#8549)
Update to latest protos from https://github.com/temporalio/api/pull/640 |
||
|
Roey BermanandDan Davison
|
d191b2e710 | Standalone Activity: initial protos, service routing, and boilerplate | ||
|
Sean KaneandGitHub
|
f1651f3dd5 |
Add a slack notification when main runs fail (#8841)
## What changed? Alert company slack when the main build fails after a PR merge ## Why? Help us combat our issues with flaky tests. ## How did you test it? - [ ] built - [X] run locally and tested manually - [ ] covered by existing tests - [ ] added new unit test(s) - [ ] added new functional test(s) ## Potential risks No, not running in production |
||
|
David ReissandGitHub
|
c7c27e5ab1 |
Use handle for dynamicconfig.Key (#8171)
## What changed? Use unique.Handle for dynamicconfig.Keys instead of strings. ## Why? Force keys to always be lowercase so we don't have to convert at lookup time, and also speed up lookups by interning. ## How did you test it? - [x] covered by existing tests |
||
|
David ReissandGitHub
|
65371358c5 |
Refactor dynamicconfig.FileBasedClient to make reusable (#8147)
## What changed? Pull out parts of FileBasedConfig to make them reusable. ## Why? Make it easier to build other Clients. ## How did you test it? - [x] built - [x] covered by existing tests |
||
|
Lina JodoinandGitHub
|
19fc1ad532 |
Hook up CHASM Scheduler to Frontend handler (#8694)
## What changed? - Frontend workflow_handler can now route to CHASM scheduler, depending on experiment flag/dynamic config - Scheduler functional test has been updated to exercise both V1 and V2 scheduler - Small fixes throughout scheduler codebase to address differences noticed during functional testing - Nexus callback mechanisms updated to allow the CHASM internal callback URL I plan to address the TODO around LastCompletionResult's assertion in a separate follow-up PR. ## How did you test it? - [ ] built - [ ] run locally and tested manually - [ ] covered by existing tests - [ ] added new unit test(s) - [x] added new functional test(s) ## Potential risks - Existing V1 tests are preserved, to show that the V1 codepath still works as-is. All new CHASM handling logic falls back to V1 in absence of a positive response. CHASM can only be opted in for a schedule during creation, controlled by dynamic config/experimental header. |
||
|
Alex StanfieldandGitHub
|
89065d7c84 |
add support for loading svc from the env (#8684)
## What changed? Allows the service the server binary starts to be controls by the `TEMPORAL_SERVICES` environment variable. ## Why? Allows us to remove the shell scripts from the server docker image. **Breaking Change**: the script currently supports `:` as a list separator but this does not ## Testing ``` === Single environment variable value === + API_KEY=from-env ./flagtest Resolved api-key: [from-env] (1 values) === Multiple comma-separated values in ENV === + API_KEY=key1,key2,key3 ./flagtest Resolved api-key: [key1 key2 key3] (3 values) === Multiple comma-separated values in CLI arg === + ./flagtest --api-key=cli1,cli2,cli3 Resolved api-key: [cli1 cli2 cli3] (3 values) === CLI argument overrides ENV variable (single values) === + API_KEY=from-env ./flagtest --api-key=from-cli Resolved api-key: [from-cli] (1 values) === Multiple --api-key arguments === + ./flagtest --api-key=first --api-key=second --api-key=third Resolved api-key: [first second third] (3 values) === Mixed: multiple args with comma-separated values === + ./flagtest --api-key=a1,a2 --api-key=b1,b2 Resolved api-key: [a1 a2 b1 b2] (4 values) ``` |