c8297894ce
## What changed? optimize the batch operation processing in `BatchActivity` and `BatchActivityWithProtobuf` by removing the need to wait for entire pages to complete before fetching the next page. - Implemented proactive page fetching once a worker becomes available - common `processWorkflowsWithProactiveFetching` function to reduce code duplication ## Why? The previous implementation had workers wait for entire pages to complete. This optimization improves resource utilization. The refactoring also eliminates duplicated functions in the `BatchParams` struct and `BatchOperation` protobuf. Addresses issue #8098. ## How did you test it? - [x] built - [x] run locally and tested manually - [x] covered by existing tests - [ ] added new unit test(s) - [ ] added new functional test(s) The changes maintain backward compatibility. ## Potential risks While this change improves performance, it does modify the concurrency model of batch processing: 1. **Timing changes**: The optimization changes when pages are fetched relative to task completion, which could expose edge cases in error handling or heartbeat timing 2. **Memory usage**: Pages may be fetched earlier, potentially increasing peak memory usage if the next page is large 3. **Rate limiting interaction**: The more aggressive task scheduling could interact differently with rate limiting, though the same per-worker limits are maintained 4. **Heartbeat behavior**: heartbeats track the progress of an entire page and are applied after an entire page finishes The changes preserve all existing error handling, retry logic, and rate limiting behavior, but the different execution timing could surface previously hidden race conditions. --------- Co-authored-by: Roey Berman <[email protected]>
Temporal—durable execution platform
Introduction • Getting Started • Contributing • Temporal Docs • Temporal 101
Introduction
Temporal is a durable execution platform that enables developers to build scalable applications without sacrificing productivity or reliability. The Temporal server executes units of application logic called Workflows in a resilient manner that automatically handles intermittent failures, and retries failed operations.
Temporal is a mature technology that originated as a fork of Uber's Cadence. It is developed by Temporal Technologies, a startup by the creators of Cadence.
Getting Started
Download and Start Temporal Server Locally
Execute the following commands to start a pre-built image along with all the dependencies.
brew install temporal
temporal server start-dev
Refer to Temporal CLI documentation for more installation options.
Run the Samples
Clone or download samples for Go or Java and run them with the local Temporal server. We have a number of HelloWorld type scenarios available, as well as more advanced ones. Note that the sets of samples are currently different between Go and Java.
Use CLI
Use Temporal CLI to interact with the running Temporal server.
temporal operator namespace list
temporal workflow list
Use Temporal Web UI
Try Temporal Web UI by opening http://localhost:8233 for viewing your sample workflows executing on Temporal.
Repository
This repository contains the source code of the Temporal server. To implement Workflows, Activities and Workers, use one of the supported languages.
Contributing
We'd love your help in making Temporal great.
Helpful links to get started:
- work on or propose a new feature
- learn about the Temporal Server architecture
- learn how to build and run the Temporal Server locally
- learn about Temporal Server testing tools and best practices
- join the Temporal community forum and Slack