noun · AI setting
Max output tokens
also called max tokensoutput limit
A cap on how long the AI's answer can be. When it hits the cap, it stops, even mid-sentence.
A request setting that caps the number of tokens the model may generate in its reply.
A per-request limit on how many tokens the model may generate, after which generation halts with a finish reason saying so. In many setups the reply shares the context window with the input, so both must fit.
Real-life example
You ask for a long report and the answer stops halfway through a sentence. The model did not run out of ideas. It reached the limit you, or the app, set for the reply, and the harness cut it off. Raising the cap, or asking for the report in parts, fixes it.
Response stopped: output limit reachedWhere the analogy breaks
-
It is not a target length. The model does not aim for it, so a high cap does not make answers longer.
-
Hitting it cuts the reply off. The text may end mid-sentence, and structured output can be left invalid.
-
Hidden thinking may count too. With some reasoning setups, tokens spent thinking use up the same limit, so the visible answer can be shorter than expected.