Files
tasks/chasm/nexus_operation_processor.go
7fec4a9a3f Implement SignalWithStart as a system nexus endpoint (#9833)
## What changed?
This PR adds `SignalWithStartWorkflowExecution` as a synchronous Nexus
operation exposed via `__temporal_system endpoint`, allowing workflows
to signal-with-start other workflows through the CHASM Nexus operation
framework.

Key changes:
- `chasm/lib/workflow/nexus_service.go`: `workflowServiceNexusHandler`
implements the `SignalWithStartWorkflowExecution` Nexus sync operation
by resolving the namespace and delegating to the History service. A
`SignalWithStartOperationProcessor` handles input enrichment (namespace,
request ID, links) and routing via CHASM's
`NexusOperationProcessorResult`.
- `chasm/lib/workflow/library.go` — library now holds the
`workflowServiceNexusHandler`, the workflow `Config`, the SA mapper
provider, and the SA validator. `newLibrary` (used by fx) takes those
dependencies, while the public `NewLibrary` keeps its old signature for
external callers. Adds `NexusServices()` so the library registers its
Nexus service via CHASM.
- `chasm/lib/workflow/validator.go`: `RequestValidator` consolidates the
`SignalWithStartWorkflowExecution` validation logic (previously inlined
in `WorkflowHandler`) into a reusable, injectable struct. This same
validator is used by both the frontend handler and the new CHASM
processor.
- common/dynamicconfig/constants.go — adds
`EnableSignalWithStartFromWorkflow` (namespace-scoped, default false).
- `service/frontend/workflow_handler.go`: Removed the `SignalWithStart`
validation block
- `service/history/fx.go`: Provides a `HistoryServiceServerProvider` so
the CHASM workflow library can call the history handler directly.
- `temporal/fx.go`: Removes the now-redundant `ChasmLibraryOptions`
grouping; each service module registers its own CHASM libraries.
- `components/nexusoperations/workflow/commands.go`: `NotFound` and
`InvalidArgument` errors during Nexus command handling are now surfaced
as workflow task failures instead of being treated as transient handler
errors.
- `common/payloads`: Adds `EncodeSingle`, `MustEncodeSingle`, and
`MustEncode` helpers used in tests.
- `cmd/tools/getproto`: Adds support for nexus-proto-annotations proto
imports.
- `tests/signal_with_start_from_workflow_test.go`: Functional test suite
covering the happy path, duplicate detection, conflict policies, and
validation rejection for the new Nexus operation.


## Why?
This functionality is one of our most requested GitHub issues.

## How did you test it?
- [X] built
- [X] run locally and tested manually
- [ ] covered by existing tests
- [X] added new unit test(s)
- [X] added new functional test(s)

## Potential risks
The history service now directly exposes a `HistoryServiceServer`
interface via `fx` for injection into the CHASM workflow library. This
tight coupling between the CHASM workflow library and the history
handler could complicate future layering — callers outside the history
service should not adopt this pattern. The feature is gated by
`history.enableSignalWithStartFromWorkflow` for rollout.

---------

Co-authored-by: Roey Berman <[email protected]>
2026-05-20 21:49:35 +00:00

202 lines
9.0 KiB
Go

package chasm
import (
"fmt"
"math/rand/v2"
"github.com/nexus-rpc/sdk-go/nexus"
commonpb "go.temporal.io/api/common/v1"
"go.temporal.io/server/common"
"go.temporal.io/server/common/namespace"
sdkconverter "go.temporal.io/server/common/sdk"
)
// NexusOperationProcessorContext contains context for processing a Nexus operation's input, including the target
// namespace and a request identifier used for tracing and idempotency.
type NexusOperationProcessorContext struct {
// Namespace is the target namespace used for routing and validation of Nexus operations.
Namespace *namespace.Namespace
// RequestID is a unique identifier for the incoming operation request, used for correlation and idempotency
// across retries.
RequestID string
Links []nexus.Link
// A boolean indicating whether the operation processor framework should re-serialize the input and store it in the
// [NexusOperationProcessorResult.ReserializedInputPayload] field. Processor implementations may mutate the input regardless.
ReserializeInputPayload bool
}
// NexusOperationRoutingKey determines which history shard should process a Nexus operation.
// Different implementations provide different routing strategies (e.g., by execution, random).
type NexusOperationRoutingKey interface {
// ShardID returns the target shard ID for this routing key given the total number of shards.
ShardID(numShards int32) int32
}
// NexusOperationRoutingKeyExecution routes operations to a specific shard based on an execution key.
// This ensures that operations related to the same execution are processed on the same shard.
type NexusOperationRoutingKeyExecution struct {
// NamespaceID is the ID of the namespace containing the execution whose shard should be targeted.
NamespaceID string
// BusinessID is the business identifier (e.g., workflow ID) of the execution.
BusinessID string
}
// ShardID returns the shard that owns the execution identified by the namespace and business IDs.
func (r NexusOperationRoutingKeyExecution) ShardID(numShards int32) int32 {
return common.WorkflowIDToHistoryShard(r.NamespaceID, r.BusinessID, numShards)
}
// NexusOperationRoutingKeyRandom routes operations to a random shard.
// The ShardID method returns a randomly selected valid shard ID.
type NexusOperationRoutingKeyRandom struct {
}
// ShardID returns a randomly selected shard ID in the range [1, numShards].
func (NexusOperationRoutingKeyRandom) ShardID(numShards int32) int32 {
return rand.Int32N(numShards) + 1
}
// NexusOperationProcessorResult contains the result of processing a Nexus operation input,
// including the routing key that determines which shard should handle the operation.
type NexusOperationProcessorResult struct {
// RoutingKey determines which history shard should process the operation.
RoutingKey NexusOperationRoutingKey
// A field set by the framework to containing the re-serialized input payload if requested in the given context.
ReserializedInputPayload *commonpb.Payload
}
// NexusOperationProcessor is an interface that can be implemented per operation to validate and determine routing for the operation.
type NexusOperationProcessor[I any] interface {
// ProcessInput validates the input, and returns routing information for processing this operation. The
// method may mutate the input to set default values.
ProcessInput(ctx NexusOperationProcessorContext, input I) (*NexusOperationProcessorResult, error)
}
// RegisterableNexusOperationProcessor adapts a typed Nexus operation processor for dynamic registration
// and invocation within a service processor.
type RegisterableNexusOperationProcessor struct {
processInput func(ctx NexusOperationProcessorContext, input *commonpb.Payload) (*NexusOperationProcessorResult, error)
}
func nexusOperationProcessorAdapter[I any](processor NexusOperationProcessor[I]) func(ctx NexusOperationProcessorContext, input *commonpb.Payload) (*NexusOperationProcessorResult, error) {
return func(ctx NexusOperationProcessorContext, input *commonpb.Payload) (*NexusOperationProcessorResult, error) {
var i I
if err := sdkconverter.PreferProtoDataConverter.FromPayloads(&commonpb.Payloads{Payloads: []*commonpb.Payload{input}}, &i); err != nil {
return nil, nexus.NewHandlerErrorf(nexus.HandlerErrorTypeBadRequest, "failed to decode input payload: %v", err)
}
result, err := processor.ProcessInput(ctx, i)
if err != nil {
return nil, err
}
if ctx.ReserializeInputPayload {
pls, err := sdkconverter.PreferProtoDataConverter.ToPayloads(i)
if err != nil {
herr := nexus.NewHandlerErrorf(nexus.HandlerErrorTypeInternal, "failed to re-encode input payload: %v", err)
herr.RetryBehavior = nexus.HandlerErrorRetryBehaviorNonRetryable
return nil, herr
}
if len(pls.Payloads) == 1 {
result.ReserializedInputPayload = pls.Payloads[0]
}
}
return result, nil
}
}
// NewRegisterableNexusOperationProcessor wraps a typed NexusOperationProcessor and returns a registerable adapter.
func NewRegisterableNexusOperationProcessor[I any](op NexusOperationProcessor[I]) RegisterableNexusOperationProcessor {
return RegisterableNexusOperationProcessor{
processInput: nexusOperationProcessorAdapter(op),
}
}
// NexusServiceProcessor handles input processing for operations within a specific Nexus service.
// It uses reflection to dynamically invoke the ProcessInput method on operations that implement it.
type NexusServiceProcessor struct {
name string
operations map[string]RegisterableNexusOperationProcessor
}
// NewNexusServiceProcessor constructs a processor for a single Nexus service that can register and invoke operation
// processors by name.
func NewNexusServiceProcessor(name string) *NexusServiceProcessor {
return &NexusServiceProcessor{
name: name,
operations: make(map[string]RegisterableNexusOperationProcessor),
}
}
// RegisterOperation registers a named operation with this service processor.
// Returns an error if an operation with the same name is already registered.
func (p *NexusServiceProcessor) RegisterOperation(name string, op RegisterableNexusOperationProcessor) error {
if _, exists := p.operations[name]; exists {
return fmt.Errorf("operation %q already registered", name)
}
p.operations[name] = op
return nil
}
// MustRegisterOperation registers a named operation and panics if registration fails.
func (p *NexusServiceProcessor) MustRegisterOperation(name string, op RegisterableNexusOperationProcessor) {
if err := p.RegisterOperation(name, op); err != nil {
// nolint:forbidigo // Panic is acceptable here for Must-style method.
panic(err)
}
}
// ProcessInput routes the input processing request to the appropriate operation processor and returns routing information
// for the operation.
//
// Returns a nexus.HandlerError if the operation is not found or if input processing fails.
func (p *NexusServiceProcessor) ProcessInput(ctx NexusOperationProcessorContext, opName string, input *commonpb.Payload) (*NexusOperationProcessorResult, error) {
op, ok := p.operations[opName]
if !ok {
return nil, nexus.NewHandlerErrorf(nexus.HandlerErrorTypeNotFound, "operation %q not found", opName)
}
return op.processInput(ctx, input)
}
// NexusEndpointProcessor handles input processing for Nexus operations across multiple services within a Nexus
// endpoint. It routes requests to the appropriate service processor based on the service name.
type NexusEndpointProcessor struct {
serviceProcessors map[string]*NexusServiceProcessor
}
// NewNexusEndpointProcessor creates a new NexusEndpointProcessor.
func NewNexusEndpointProcessor() *NexusEndpointProcessor {
return &NexusEndpointProcessor{
serviceProcessors: make(map[string]*NexusServiceProcessor),
}
}
// RegisterServiceProcessor adds a service-level processor to the endpoint keyed by its name.
// Returns an error if a processor with the same name is already registered.
func (p *NexusEndpointProcessor) RegisterServiceProcessor(processor *NexusServiceProcessor) error {
if _, exists := p.serviceProcessors[processor.name]; exists {
return fmt.Errorf("service processor %q already registered", processor.name)
}
p.serviceProcessors[processor.name] = processor
return nil
}
// MustRegisterServiceProcessor registers the service processor and panics if registration fails.
func (p *NexusEndpointProcessor) MustRegisterServiceProcessor(processor *NexusServiceProcessor) {
if err := p.RegisterServiceProcessor(processor); err != nil {
// nolint:forbidigo // Panic is acceptable here for Must-style method.
panic(err)
}
}
// ProcessInput routes the input processing request to the appropriate service processor and returns routing information
// for the operation.
//
// Returns a nexus.HandlerError if the service is not found or if input processing fails.
func (p *NexusEndpointProcessor) ProcessInput(ctx NexusOperationProcessorContext, service, operation string, input *commonpb.Payload) (*NexusOperationProcessorResult, error) {
serviceProcessor, ok := p.serviceProcessors[service]
if !ok {
return nil, nexus.NewHandlerErrorf(nexus.HandlerErrorTypeNotFound, "service %q not found", service)
}
return serviceProcessor.ProcessInput(ctx, operation, input)
}