Understanding AI Model Parameters: Temperature, Top-K, and Top-P

Same prompt, different answer every run? Hidden sampling settings control how the model picks each word, and they can change the output more than your wording does. Here is what each dial does.

Temperature: predictable to creative

Temperature is a dial between predictable and creative. At 0 the model always picks the most likely next word: consistent, good for factual work. Low (0.1 to 0.3) stays mostly predictable with slight variation, right for code and analysis. High (0.7 to 1.0) gets random and creative, right for brainstorming and stories.

Top-K and Top-P: trimming the candidate list

At every step the model ranks all possible next words by probability. Top-K limits how many candidates it considers: Top-K of 1 looks only at the single most likely word, Top-K of 40 picks from the top 40. Top-P reaches the same goal differently: it adds probabilities from most to least likely and stops at the cutoff (default 0.95). Top-P adapts to the model's confidence, so it is usually the more flexible of the two.

Max output tokens and stop sequences

Max output tokens caps response length; a token is roughly four characters, so 100 tokens is about 60 to 80 words. A stop sequence tells the model to halt at a specific string, like "---" after the last item of a list. Pick a marker that would not naturally appear in the response.

The trade-off

Predictability costs variety: temperature 0 can loop on boring, repetitive phrasing, while high temperature invents structure you did not ask for. Defaults exist because they balance this for most tasks. Tweak only when the output is consistently wrong for your use case, and change one dial at a time so you know what caused the shift.

What to do first

Leave the defaults. When answers drift, lower temperature before touching Top-K, Top-P, or token limits.

Based on: Gemini API Prompting Strategies (ai.google.dev)