20 KiB
\BenchmarkAPI
All URIs are relative to https://api.hanzo.ai
| Method | HTTP request | Description |
|---|---|---|
| GetBenchmarkCatalog | Get /v1/benchmark/catalog | Is the canonical benchmarks this arena reads — the id, title, axis, item count, upstream source and direction of each, with native marking the ones the standardized harness runs today. |
| GetBenchmarkClaims | Get /v1/benchmark/claims | Lists the effective claims the caller may read, each labelled with the org that made it and the user who recorded it. |
| GetBenchmarkCompare | Get /v1/benchmark/compare | Is the ONLY valid arm-vs-arm test: it pairs the two models on the items BOTH completed, and answers rescue and damage counts with an exact-McNemar p. |
| GetBenchmarkHistory | Get /v1/benchmark/history | Returns each model's measured score per run over time, oldest first, with the change between runs. |
| GetBenchmarkLeaderboard | Get /v1/benchmark/leaderboard | Answers one row per model for the benchmark named — what our own harness measured, beside what the vendor claims, and the gap between them. |
| GetBenchmarkPresets | Get /v1/benchmark/presets | Are the router blends available to compose from — a named set of model arms, the rank they escalate through and the panel width that bounds fan-out — each named for the model layer as enso-<name>. |
| PostBenchmarkClaims | Post /v1/benchmark/claims | Records claims for the caller's org: one to correct a number, many to import a leaderboard. |
| PostBenchmarkPresets | Post /v1/benchmark/presets | Stores a router blend for the caller's org — its name, its arms, the rank they escalate through and the panel fan-out width — and answers with the stored blend and the enso-<name> it would be resolved under. |
| PostBenchmarkRuns | Post /v1/benchmark/runs | Answers what the arena already holds for a model on each benchmark asked about, and spends nothing. |
GetBenchmarkCatalog
BenchmarkBenchmarkCatalog GetBenchmarkCatalog(ctx).Execute()
Is the canonical benchmarks this arena reads — the id, title, axis, item count, upstream source and direction of each, with native marking the ones the standardized harness runs today.
Example
package main
import (
"context"
"fmt"
"os"
openapiclient "github.com/hanzoai/go-sdk/v8"
)
func main() {
configuration := openapiclient.NewConfiguration()
apiClient := openapiclient.NewAPIClient(configuration)
resp, r, err := apiClient.BenchmarkAPI.GetBenchmarkCatalog(context.Background()).Execute()
if err != nil {
fmt.Fprintf(os.Stderr, "Error when calling `BenchmarkAPI.GetBenchmarkCatalog``: %v\n", err)
fmt.Fprintf(os.Stderr, "Full HTTP response: %v\n", r)
}
// response from `GetBenchmarkCatalog`: BenchmarkBenchmarkCatalog
fmt.Fprintf(os.Stdout, "Response from `BenchmarkAPI.GetBenchmarkCatalog`: %v\n", resp)
}
Path Parameters
This endpoint does not need any parameter.
Other Parameters
Other parameters are passed through a pointer to a apiGetBenchmarkCatalogRequest struct via the builder pattern
Return type
Authorization
HTTP request headers
- Content-Type: Not defined
- Accept: application/json, application/problem+json
[Back to top] [Back to API list] [Back to Model list] [Back to README]
GetBenchmarkClaims
BenchmarkClaimsOut GetBenchmarkClaims(ctx).Benchmark(benchmark).Model(model).Provider(provider).Source(source).Protocol(protocol).Org(org).Execute()
Lists the effective claims the caller may read, each labelled with the org that made it and the user who recorded it.
Example
package main
import (
"context"
"fmt"
"os"
openapiclient "github.com/hanzoai/go-sdk/v8"
)
func main() {
benchmark := "benchmark_example" // string | Benchmark filters to one benchmark id. Empty returns every benchmark. (optional)
model := "model_example" // string | Model filters to one model. Empty returns every model. (optional)
provider := "provider_example" // string | Provider filters to one lab or leaderboard — the way to read what a single source claims across every model it covers. (optional)
source := "source_example" // string | Source filters to one citation, which is the finest grain there is: a source is what makes two claims about one model independent rather than a restatement of each other. (optional)
protocol := "protocol_example" // string | Protocol filters by HOW a claim was scored, so provider cards can be read apart from third parties running their own harness. (optional)
org := "org_example" // string | Org filters to the claims one org made; \"admin\" reads the platform's own. It narrows what the caller may already read and never widens it. (optional)
configuration := openapiclient.NewConfiguration()
apiClient := openapiclient.NewAPIClient(configuration)
resp, r, err := apiClient.BenchmarkAPI.GetBenchmarkClaims(context.Background()).Benchmark(benchmark).Model(model).Provider(provider).Source(source).Protocol(protocol).Org(org).Execute()
if err != nil {
fmt.Fprintf(os.Stderr, "Error when calling `BenchmarkAPI.GetBenchmarkClaims``: %v\n", err)
fmt.Fprintf(os.Stderr, "Full HTTP response: %v\n", r)
}
// response from `GetBenchmarkClaims`: BenchmarkClaimsOut
fmt.Fprintf(os.Stdout, "Response from `BenchmarkAPI.GetBenchmarkClaims`: %v\n", resp)
}
Path Parameters
Other Parameters
Other parameters are passed through a pointer to a apiGetBenchmarkClaimsRequest struct via the builder pattern
| Name | Type | Description | Notes |
|---|---|---|---|
| benchmark | string | Benchmark filters to one benchmark id. Empty returns every benchmark. | |
| model | string | Model filters to one model. Empty returns every model. | |
| provider | string | Provider filters to one lab or leaderboard — the way to read what a single source claims across every model it covers. | |
| source | string | Source filters to one citation, which is the finest grain there is: a source is what makes two claims about one model independent rather than a restatement of each other. | |
| protocol | string | Protocol filters by HOW a claim was scored, so provider cards can be read apart from third parties running their own harness. | |
| org | string | Org filters to the claims one org made; "admin" reads the platform's own. It narrows what the caller may already read and never widens it. |
Return type
Authorization
HTTP request headers
- Content-Type: Not defined
- Accept: application/json, application/problem+json
[Back to top] [Back to API list] [Back to Model list] [Back to README]
GetBenchmarkCompare
BenchmarkPairing GetBenchmarkCompare(ctx).A(a).B(b).Benchmark(benchmark).Execute()
Is the ONLY valid arm-vs-arm test: it pairs the two models on the items BOTH completed, and answers rescue and damage counts with an exact-McNemar p.
Example
package main
import (
"context"
"fmt"
"os"
openapiclient "github.com/hanzoai/go-sdk/v8"
)
func main() {
a := "a_example" // string | A is the first model id. It is required.
b := "b_example" // string | B is the second model id. It is required.
benchmark := "benchmark_example" // string | Benchmark is the catalog id to compare on, defaulting to gpqa_diamond. (optional)
configuration := openapiclient.NewConfiguration()
apiClient := openapiclient.NewAPIClient(configuration)
resp, r, err := apiClient.BenchmarkAPI.GetBenchmarkCompare(context.Background()).A(a).B(b).Benchmark(benchmark).Execute()
if err != nil {
fmt.Fprintf(os.Stderr, "Error when calling `BenchmarkAPI.GetBenchmarkCompare``: %v\n", err)
fmt.Fprintf(os.Stderr, "Full HTTP response: %v\n", r)
}
// response from `GetBenchmarkCompare`: BenchmarkPairing
fmt.Fprintf(os.Stdout, "Response from `BenchmarkAPI.GetBenchmarkCompare`: %v\n", resp)
}
Path Parameters
Other Parameters
Other parameters are passed through a pointer to a apiGetBenchmarkCompareRequest struct via the builder pattern
| Name | Type | Description | Notes |
|---|---|---|---|
| a | string | A is the first model id. It is required. | |
| b | string | B is the second model id. It is required. | |
| benchmark | string | Benchmark is the catalog id to compare on, defaulting to gpqa_diamond. |
Return type
Authorization
HTTP request headers
- Content-Type: Not defined
- Accept: application/json, application/problem+json
[Back to top] [Back to API list] [Back to Model list] [Back to README]
GetBenchmarkHistory
BenchmarkHistoryOut GetBenchmarkHistory(ctx).Benchmark(benchmark).Model(model).Execute()
Returns each model's measured score per run over time, oldest first, with the change between runs.
Example
package main
import (
"context"
"fmt"
"os"
openapiclient "github.com/hanzoai/go-sdk/v8"
)
func main() {
benchmark := "benchmark_example" // string | Benchmark is the catalog id to read, defaulting to gpqa_diamond. (optional)
model := "model_example" // string | Model filters to one model. Empty returns every model measured. (optional)
configuration := openapiclient.NewConfiguration()
apiClient := openapiclient.NewAPIClient(configuration)
resp, r, err := apiClient.BenchmarkAPI.GetBenchmarkHistory(context.Background()).Benchmark(benchmark).Model(model).Execute()
if err != nil {
fmt.Fprintf(os.Stderr, "Error when calling `BenchmarkAPI.GetBenchmarkHistory``: %v\n", err)
fmt.Fprintf(os.Stderr, "Full HTTP response: %v\n", r)
}
// response from `GetBenchmarkHistory`: BenchmarkHistoryOut
fmt.Fprintf(os.Stdout, "Response from `BenchmarkAPI.GetBenchmarkHistory`: %v\n", resp)
}
Path Parameters
Other Parameters
Other parameters are passed through a pointer to a apiGetBenchmarkHistoryRequest struct via the builder pattern
| Name | Type | Description | Notes |
|---|---|---|---|
| benchmark | string | Benchmark is the catalog id to read, defaulting to gpqa_diamond. | |
| model | string | Model filters to one model. Empty returns every model measured. |
Return type
Authorization
HTTP request headers
- Content-Type: Not defined
- Accept: application/json, application/problem+json
[Back to top] [Back to API list] [Back to Model list] [Back to README]
GetBenchmarkLeaderboard
BenchmarkLeaderboard GetBenchmarkLeaderboard(ctx).Benchmark(benchmark).Execute()
Answers one row per model for the benchmark named — what our own harness measured, beside what the vendor claims, and the gap between them.
Example
package main
import (
"context"
"fmt"
"os"
openapiclient "github.com/hanzoai/go-sdk/v8"
)
func main() {
benchmark := "benchmark_example" // string | Benchmark is the catalog id to read, defaulting to gpqa_diamond. (optional)
configuration := openapiclient.NewConfiguration()
apiClient := openapiclient.NewAPIClient(configuration)
resp, r, err := apiClient.BenchmarkAPI.GetBenchmarkLeaderboard(context.Background()).Benchmark(benchmark).Execute()
if err != nil {
fmt.Fprintf(os.Stderr, "Error when calling `BenchmarkAPI.GetBenchmarkLeaderboard``: %v\n", err)
fmt.Fprintf(os.Stderr, "Full HTTP response: %v\n", r)
}
// response from `GetBenchmarkLeaderboard`: BenchmarkLeaderboard
fmt.Fprintf(os.Stdout, "Response from `BenchmarkAPI.GetBenchmarkLeaderboard`: %v\n", resp)
}
Path Parameters
Other Parameters
Other parameters are passed through a pointer to a apiGetBenchmarkLeaderboardRequest struct via the builder pattern
| Name | Type | Description | Notes |
|---|---|---|---|
| benchmark | string | Benchmark is the catalog id to read, defaulting to gpqa_diamond. |
Return type
Authorization
HTTP request headers
- Content-Type: Not defined
- Accept: application/json, application/problem+json
[Back to top] [Back to API list] [Back to Model list] [Back to README]
GetBenchmarkPresets
BenchmarkPresetList GetBenchmarkPresets(ctx).Execute()
Are the router blends available to compose from — a named set of model arms, the rank they escalate through and the panel width that bounds fan-out — each named for the model layer as enso-.
Example
package main
import (
"context"
"fmt"
"os"
openapiclient "github.com/hanzoai/go-sdk/v8"
)
func main() {
configuration := openapiclient.NewConfiguration()
apiClient := openapiclient.NewAPIClient(configuration)
resp, r, err := apiClient.BenchmarkAPI.GetBenchmarkPresets(context.Background()).Execute()
if err != nil {
fmt.Fprintf(os.Stderr, "Error when calling `BenchmarkAPI.GetBenchmarkPresets``: %v\n", err)
fmt.Fprintf(os.Stderr, "Full HTTP response: %v\n", r)
}
// response from `GetBenchmarkPresets`: BenchmarkPresetList
fmt.Fprintf(os.Stdout, "Response from `BenchmarkAPI.GetBenchmarkPresets`: %v\n", resp)
}
Path Parameters
This endpoint does not need any parameter.
Other Parameters
Other parameters are passed through a pointer to a apiGetBenchmarkPresetsRequest struct via the builder pattern
Return type
Authorization
HTTP request headers
- Content-Type: Not defined
- Accept: application/json, application/problem+json
[Back to top] [Back to API list] [Back to Model list] [Back to README]
PostBenchmarkClaims
BenchmarkPutClaimsOut PostBenchmarkClaims(ctx).BenchmarkPutClaimsIn(benchmarkPutClaimsIn).Execute()
Records claims for the caller's org: one to correct a number, many to import a leaderboard.
Example
package main
import (
"context"
"fmt"
"os"
openapiclient "github.com/hanzoai/go-sdk/v8"
)
func main() {
benchmarkPutClaimsIn := *openapiclient.NewBenchmarkPutClaimsIn() // BenchmarkPutClaimsIn |
configuration := openapiclient.NewConfiguration()
apiClient := openapiclient.NewAPIClient(configuration)
resp, r, err := apiClient.BenchmarkAPI.PostBenchmarkClaims(context.Background()).BenchmarkPutClaimsIn(benchmarkPutClaimsIn).Execute()
if err != nil {
fmt.Fprintf(os.Stderr, "Error when calling `BenchmarkAPI.PostBenchmarkClaims``: %v\n", err)
fmt.Fprintf(os.Stderr, "Full HTTP response: %v\n", r)
}
// response from `PostBenchmarkClaims`: BenchmarkPutClaimsOut
fmt.Fprintf(os.Stdout, "Response from `BenchmarkAPI.PostBenchmarkClaims`: %v\n", resp)
}
Path Parameters
Other Parameters
Other parameters are passed through a pointer to a apiPostBenchmarkClaimsRequest struct via the builder pattern
| Name | Type | Description | Notes |
|---|---|---|---|
| benchmarkPutClaimsIn | BenchmarkPutClaimsIn |
Return type
Authorization
HTTP request headers
- Content-Type: application/json
- Accept: application/json, application/problem+json
[Back to top] [Back to API list] [Back to Model list] [Back to README]
PostBenchmarkPresets
BenchmarkPresetStored PostBenchmarkPresets(ctx).BenchmarkPreset(benchmarkPreset).Execute()
Stores a router blend for the caller's org — its name, its arms, the rank they escalate through and the panel fan-out width — and answers with the stored blend and the enso- it would be resolved under.
Example
package main
import (
"context"
"fmt"
"os"
openapiclient "github.com/hanzoai/go-sdk/v8"
)
func main() {
benchmarkPreset := *openapiclient.NewBenchmarkPreset() // BenchmarkPreset |
configuration := openapiclient.NewConfiguration()
apiClient := openapiclient.NewAPIClient(configuration)
resp, r, err := apiClient.BenchmarkAPI.PostBenchmarkPresets(context.Background()).BenchmarkPreset(benchmarkPreset).Execute()
if err != nil {
fmt.Fprintf(os.Stderr, "Error when calling `BenchmarkAPI.PostBenchmarkPresets``: %v\n", err)
fmt.Fprintf(os.Stderr, "Full HTTP response: %v\n", r)
}
// response from `PostBenchmarkPresets`: BenchmarkPresetStored
fmt.Fprintf(os.Stdout, "Response from `BenchmarkAPI.PostBenchmarkPresets`: %v\n", resp)
}
Path Parameters
Other Parameters
Other parameters are passed through a pointer to a apiPostBenchmarkPresetsRequest struct via the builder pattern
| Name | Type | Description | Notes |
|---|---|---|---|
| benchmarkPreset | BenchmarkPreset |
Return type
Authorization
HTTP request headers
- Content-Type: application/json
- Accept: application/json, application/problem+json
[Back to top] [Back to API list] [Back to Model list] [Back to README]
PostBenchmarkRuns
BenchmarkMeasurements PostBenchmarkRuns(ctx).BenchmarkSuite(benchmarkSuite).Execute()
Answers what the arena already holds for a model on each benchmark asked about, and spends nothing.
Example
package main
import (
"context"
"fmt"
"os"
openapiclient "github.com/hanzoai/go-sdk/v8"
)
func main() {
benchmarkSuite := *openapiclient.NewBenchmarkSuite([]string{"Benchmarks_example"}) // BenchmarkSuite |
configuration := openapiclient.NewConfiguration()
apiClient := openapiclient.NewAPIClient(configuration)
resp, r, err := apiClient.BenchmarkAPI.PostBenchmarkRuns(context.Background()).BenchmarkSuite(benchmarkSuite).Execute()
if err != nil {
fmt.Fprintf(os.Stderr, "Error when calling `BenchmarkAPI.PostBenchmarkRuns``: %v\n", err)
fmt.Fprintf(os.Stderr, "Full HTTP response: %v\n", r)
}
// response from `PostBenchmarkRuns`: BenchmarkMeasurements
fmt.Fprintf(os.Stdout, "Response from `BenchmarkAPI.PostBenchmarkRuns`: %v\n", resp)
}
Path Parameters
Other Parameters
Other parameters are passed through a pointer to a apiPostBenchmarkRunsRequest struct via the builder pattern
| Name | Type | Description | Notes |
|---|---|---|---|
| benchmarkSuite | BenchmarkSuite |
Return type
Authorization
HTTP request headers
- Content-Type: application/json
- Accept: application/json, application/problem+json
[Back to top] [Back to API list] [Back to Model list] [Back to README]