Skip to main content

LLM Parameters

Temperature is a setting you choose, not one you inherit.


What it is

LLM parameters are the configuration knobs you pass to the API to alter how the model selects the next token. The most critical parameters are:

  • Temperature: Controls randomness. 0.0 means strictly deterministic (always picking the highest probability token); 1.0 allows more creative, varied choices.
  • Top P (Nucleus Sampling): Restricts token selection to a subset of tokens whose cumulative probability reaches P. It acts as a dynamic cutoff for wild guesses.
  • Frequency/Presence Penalty: Penalizes tokens based on how often they have already appeared in the text, preventing repetitive loops.

Why it matters in production

Leaving these parameters unconfigured is reckless. A coding agent running at temperature: 1.0 will eventually invent syntax that does not exist. A creative writing agent running at temperature: 0.0 will produce robotic, lifeless text.

In production, mismatched parameters lead to unstable agents. If you need JSON output, high entropy will cause formatting errors. You must explicitly configure these parameters based on the specific cognitive task the agent is performing.


How Agenthood implements it

Agenthood defines strict parameter boundaries within the LLMRequest type. Each call to an ILLMProvider declares its parameters.

This ensures agents cannot accidentally run amok. It is implemented in src/llm/types.ts (shipped in v2.0.0):

export interface LLMRequest {
  prompt: string;
  temperature: number; // 0.0 for strict tasks, 0.7 for creative tasks
  topP?: number;
  maxTokens?: number;
  stopSequences?: string[];
}

The Doorman's commit validation sets temperature: 0, so the same input gives the same verdict.


Hands-on example

You can configure these parameters directly when initializing the runtime layer.

The Agenthood runtime does not expose a --temperature CLI flag. Temperature can only be set programmatically via the TypeScript API:

import { LLMRequest } from '@agenthood/llm';

const strictRequest: LLMRequest = {
  prompt: 'Format this data as JSON.',
  temperature: 0.0,
  maxTokens: 1000
};

Further reading