Skip to main content

LLM Parameters

If you leave temperature at its default, you are surrendering control of your application's entropy.

What it is

LLM parameters are the configuration knobs you pass to the API to alter how the model selects the next token. The most critical parameters are:

  • Temperature: Controls randomness. 0.0 means strictly deterministic (always picking the highest probability token); 1.0 allows more creative, varied choices.
  • Top P (Nucleus Sampling): Restricts token selection to a subset of tokens whose cumulative probability reaches P. It acts as a dynamic cutoff for wild guesses.
  • Frequency/Presence Penalty: Penalizes tokens based on how often they have already appeared in the text, preventing repetitive loops.

Why it matters in production

Leaving these parameters unconfigured is reckless. A coding agent running at temperature: 1.0 will eventually invent syntax that does not exist. A creative writing agent running at temperature: 0.0 will produce robotic, lifeless text.

In production, mismatched parameters lead to unstable agents. If you need JSON output, high entropy will cause formatting errors. You must explicitly configure these parameters based on the specific cognitive task the agent is performing.

How Agenthood implements it

Agenthood defines strict parameter boundaries within the LLMRequest type. Every call to an ILLMProvider must declare its parameters.

This ensures agents cannot accidentally run amok. It is implemented in src/llm/types.ts (shipped in v2.0.0):

export interface LLMRequest {
  prompt: string;
  temperature: number; // 0.0 for strict tasks, 0.7 for creative tasks
  topP?: number;
  maxTokens?: number;
  stopSequences?: string[];
}

The Society does not guess. It sets temperature: 0 for operations like The Doorman's commit validation to guarantee absolute determinism.

Hands-on example

You can configure these parameters directly when initializing the runtime layer.

# Agenthood runtime defaults to low temperature for code tasks
npx agenthood run the-scribe "Write a commit" --temperature 0.1

Or in TypeScript (future milestone):

import { LLMRequest } from '@agenthood/llm';

const strictRequest: LLMRequest = {
  prompt: 'Format this data as JSON.',
  temperature: 0.0,
  maxTokens: 1000
};

Further reading