Skip to content

Common LLM Hyperparameters

Every rt.llm.* model wrapper accepts a shared set of common hyperparameters for controlling sampling, output length, and reasoning behavior.

Hyperparameter Controls
temperature Randomness/diversity of the response.
top_p Nucleus sampling: cumulative-probability cutoff for candidate tokens.
max_tokens Maximum tokens to generate.
frequency_penalty Penalizes tokens by how often they've already appeared.
presence_penalty Penalizes tokens that have already appeared at all.
reasoning_effort Requested reasoning effort, for reasoning-capable models.
service_tier Requested service tier (provider-specific).
verbosity Requested output verbosity (currently OpenAI GPT-5-series only). 
import railtracks as rt

llm = rt.llm.OpenAILLM(
    "gpt-4o",
    temperature=0.7,
    top_p=0.9,
    max_tokens=256,
    frequency_penalty=0.2,
    presence_penalty=0.1,
)

reasoning_effort accepts one of "minimal", "low", "medium", "high":

import railtracks as rt

reasoning_llm = rt.llm.OpenAILLM("gpt-5-mini", reasoning_effort="low")

Not every model supports every hyperparameter

For example, only OpenAI's GPT-5-series supports verbosity, and newer Anthropic models restrict temperature/top_p in ways that vary by model. Railtracks checks this when you construct the model, before making any network call, and raises immediately if a hyperparameter (or combination of hyperparameters) isn't supported:

import railtracks as rt

try:
    # Claude Opus 4.7+ rejects non-default temperature/top_p server-side.
    opus_llm = rt.llm.AnthropicLLM("claude-opus-4-7", temperature=0.5)
except rt.llm.UnsupportedHyperparameterError as e:
    print(e.reason)
    # Model anthropic/claude-opus-4-7 does not support 'temperature' (got temperature=0.5).

Railtracks Recommendation

Treat UnsupportedHyperparameterError / MutuallyExclusiveHyperparametersError at construction as an actionable signal, not a bug to work around: it's telling you the provider would reject the request either way, just with a much less obvious error deeper into a run.

Things to know

  • Anthropic: Opus 4.7 and later reject non-default temperature/top_p. Separately, specifying temperature and top_p together is rejected on Anthropic models in general. Pass at most one of the two.
  • OpenAI: verbosity is only supported on the GPT-5 series, and not on the Codex variants (gpt-5-codex, gpt-5.1-codex, etc.).
  • Gemini: frequency_penalty and presence_penalty are not currently supported.

Railtracks keeps this list current as providers change their behavior. If a hyperparameter you expect to work gets rejected, check here first.

Invalid values

Railtracks does not validate hyperparameter values, only whether a hyperparameter is supported at all for the model you're using. An out-of-range or wrong-type value (e.g. temperature=999, temperature="not a number") is sent through as-is and the provider will reject it with a clear, specific error (e.g. "Expected a value <= 2, but got 999").

One exception: OpenAI currently accepts an invalid verbosity value without complaint. If you're setting verbosity, double-check the value against OpenAI's docs rather than relying on an error to catch a typo.