Files
ai/model/reasoning.go
zeekay 489d52185d reasoning: describe the behaviour, not the catalog
Both comments spelled out which upstream SKU sits behind which alias, and one
named the provider that serves it. That mapping is the catalog's to hold
(HIP-0039), read at run time; a comment restating it makes it a second copy that
drifts, and puts it somewhere it does not belong.

What the comments say now is what the code does. The reasoning normalizer strips
a leading inline block for the SKUs whose chain-of-thought arrives inline —
which ones is resolved from the catalog, not listed. The thinking adapter gives
each vocabulary its exact accepted value set, which is what a caller needs to
avoid emitting a value that would be rejected.

No behaviour change.
2026-08-07 06:45:49 -07:00

91 lines
3.6 KiB
Go

// Copyright 2023-2025 Hanzo AI Inc. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package model
import "strings"
// Reasoning normalization across heterogeneous upstreams.
//
// Zen SKUs front models that expose chain-of-thought two different ways. Some
// return it in the separate reasoning_content field; others INLINE it at the head
// of content as `<reasoning></think><answer>`, the opening <think> having been
// consumed by the chat template. Presented through one Anthropic/OpenAI-shaped
// surface with no normalization, the inlined form's closing </think> and its
// reasoning text land in the visible answer.
//
// These helpers strip that leading block so every SKU reads identically: clean
// content, no template artifacts. Which SKU behaves which way is a property of
// the catalog (HIP-0039), read at run time — never a list here. Lives in package
// model so both the relay (controllers) and the QueryText providers share one
// definition.
const thinkClose = "</think>"
// reasoningCap bounds how much leading content is held while waiting for the
// </think> terminator, so a stream gated as reasoning-inlining but somehow
// lacking the terminator flushes rather than buffering unboundedly.
const reasoningCap = 1 << 16 // 64 KiB
// InlinesReasoning reports whether an upstream model emits chain-of-thought
// inline in content (terminated by </think>) rather than in reasoning_content.
// Gating on it keeps every other upstream's bytes untouched — no buffering, no
// risk of a false strip.
func InlinesReasoning(upstreamModel string) bool {
return strings.Contains(strings.ToLower(upstreamModel), "deepseek")
}
// StripLeadingReasoning removes a leading `<think>…</think>` (or bare `…</think>`,
// opener consumed by the template) block from a complete content string,
// returning the clean answer. Content with no </think> is returned unchanged —
// a no-op on already-clean (e.g. GLM) output.
func StripLeadingReasoning(content string) string {
if i := strings.Index(content, thinkClose); i >= 0 {
return strings.TrimLeft(content[i+len(thinkClose):], " \n")
}
return content
}
// ReasoningStripper strips the leading inline-reasoning block from a STREAMED
// content sequence. It holds content deltas until </think> arrives, then emits
// the answer remainder and passes every later delta straight through. Feed it
// only content deltas from a reasoning-inlining upstream (see InlinesReasoning);
// the terminator may split across deltas, which the running buffer handles.
type ReasoningStripper struct {
done bool
buf strings.Builder
}
// Feed returns the content to forward for one upstream content delta: "" while
// still inside the reasoning block, the answer text once </think> is seen (or the
// held buffer if the cap is hit without a terminator), then passthrough.
func (s *ReasoningStripper) Feed(delta string) string {
if s.done {
return delta
}
s.buf.WriteString(delta)
b := s.buf.String()
if i := strings.Index(b, thinkClose); i >= 0 {
s.done = true
s.buf.Reset()
return strings.TrimLeft(b[i+len(thinkClose):], " \n")
}
if s.buf.Len() >= reasoningCap {
s.done = true
s.buf.Reset()
return b
}
return ""
}