JanitorAI Generation Settings Explained: Temperature, Max Tokens and Context Size
Disclosure: Composite runs this site, and one section covers Composite features. The settings advice works with any proxy or model.
Short answer: start with temperature 0.7, max new tokens 500, and a context size equal to the model's context window minus 500 (or lower on very large models). Leave everything else at its neutral value. Then change one setting at a time: temperature for how creative replies are, max new tokens for cut-off replies, and context size for memory, speed and cost.
What generation settings are
A language model writes a reply one token at a time. A token is roughly three quarters of an English word. At each step it ranks every possible next token by probability, and the generation settings decide how it picks from that list and when it stops. They do not change what the model knows or how it was trained, so no slider turns a weak model into a strong one. They decide how adventurous, how long and how repetitive its replies are.
JanitorAI shows three of these settings in its generation settings: temperature, max new tokens and context size. Apps such as SillyTavern show many more. The labels differ between apps, but they map to the same fields in the request your proxy receives:
| JanitorAI | SillyTavern | API field | What it controls |
|---|---|---|---|
| Temperature | Temperature | temperature | How adventurous each word choice is |
| Max new tokens | Response (tokens) | max_tokens | The longest reply the model may write |
| Context size | Context (tokens) | (not sent; the app trims the prompt) | How much card, preset and history the app sends |
| Not shown | Top P | top_p | Keeps only the most likely words that add up to P |
| Not shown | Top K | top_k | Keeps only the K most likely words |
| Not shown | Min P | min_p | Drops words far less likely than the top choice |
| Not shown | Repetition Penalty | repetition_penalty | Makes words already in the chat less likely |
| Not shown | Frequency / Presence Penalty | frequency_penalty, presence_penalty | OpenAI-style penalties for repeated words |
JanitorAI updates its settings panel from time to time, so the exact labels on your screen may differ slightly.
Temperature: how creative replies are
Temperature reshapes the probability list before a token is picked. A low value makes the most likely token even more likely, so replies are predictable and consistent. A high value flattens the list, so less likely words get picked more often, which reads as more creative until it reads as random. SillyTavern's documentation puts it simply: below 1.0 leads to more predictable text, above 1.0 increases creativity and diversity, and 1 leaves the model's original probabilities alone.
JanitorAI's help center groups values as low (0.2 to 0.4), medium (0.5 to 0.7) and high (0.8 to 1.2 and up). It suggests starting at 0.6, adding 0.1 at a time until replies have some spark, and keeping the value below 1.0. That matches what works on most proxy models: 0.6 to 0.8 for roleplay.
Signs that temperature is wrong:
- Too high: the bot changes the subject, invents details that contradict the card, switches language mid-reply or writes word salad.
- Too low: every reply opens the same way, the character agrees with everything, and regenerating gives almost the same text.
Some models want less than the usual range. Mistral's model card recommends 0.3 for Mistral Nemo, and our Mistral Nemo guide explains why a 0.8 preset makes it ramble. Small models in general lose the thread at high temperatures sooner than large ones.
Max new tokens: how long a reply can be
Max new tokens is the longest reply the model may write. It is a ceiling, not a target: the model stops when it thinks the reply is finished, and the limit only matters when the reply would run longer. If replies stop in the middle of a sentence, they hit the limit. If they are simply short, the limit is not the cause; ask for longer replies in the custom prompt or the character card instead.
- 300 to 400 tokens suits quick back-and-forth chat, roughly 225 to 300 words.
- 400 to 600 tokens suits most roleplay, with room for action, dialogue and a little description.
- 800 or more suits long narrative replies, or reasoning models that think before they answer.
Reasoning models need the most room. On many providers, the hidden thinking is generated as output too, so it can use part of the same budget and end a reply before the visible text is finished. Give them more max new tokens, or lower the reasoning effort. On Composite you can add a suffix such as [low] to the model ID on models that support effort levels; the model modifiers reference lists them.
A very high value has costs even when replies are short. Your app subtracts it from the context it sends, so less chat history fits, and on paid requests it can raise the amount reserved for the reply up front. If your app sends no limit at all, Composite uses 8,192 tokens.
Context size: how much the model reads
Context size is how many tokens your app sends with each message: the character card, your persona, the preset or custom prompt, memory and lorebook entries, and as much chat history as fits. SillyTavern's documentation describes its Context setting as the maximum the app will send as the prompt, minus the response length. When the history no longer fits, the app leaves out the oldest messages.
The rule: context size plus max new tokens must fit inside the model's context window. If it does not, the proxy refuses the request with a context-window error, which our proxy error guide covers. A few examples:
- Llama 3 8B Lunaris has 8,192 tokens. With 500 max new tokens, set context size to about 7,600.
- Synth 2.5 Flash has 30,000 tokens. With 600 max new tokens, set it to about 29,000.
- GLM 5.3 Flash has just over a million. You do not need anywhere near that much.
On large models, smaller is often better. Every token is sent again with every message, long prompts take longer before the first word appears, and models are less reliable about details buried in the middle of a very long prompt. JanitorAI's own guide notes that most well-made bots work best with under 1,500 tokens of permanent information, which is a reminder that a lean card leaves more room for the story. A context size of 32,000 to 64,000 holds a long session on most models; past that, a summary or memory layer that keeps the important facts is usually cheaper than sending everything.
Top P, top K and min P
These three cut the list of candidate tokens before one is picked. JanitorAI does not show them, but SillyTavern and most presets do.
- Top P keeps the smallest set of tokens whose probabilities add up to P. 1 turns it off. 0.9 to 0.95 trims the oddest choices.
- Top K keeps only the K most likely tokens. 0 turns it off on most backends (some use -1). It is a blunt tool, so many presets leave it off.
- Min P drops any token much less likely than the top choice, scaled by how confident the model is. SillyTavern's documentation says it works best at low values such as 0.01 to 0.1, and 0 turns it off. Many roleplay presets use min P around 0.05 instead of top P and top K, because it lets you run a higher temperature without the replies falling apart.
A simple setup that works on models that accept min P: temperature 0.8 to 1.0, min P 0.05, top P 1 and top K 0. If a model does not accept min P, use top P 0.9 with a temperature around 0.7 instead.
Repetition, frequency and presence penalties
These make the model less likely to reuse words that are already in the chat. They help with loops, where the bot repeats the same phrase or reply shape, but they work on single words, not ideas, so a strong penalty pushes the model away from ordinary words like names and pronouns too.
- Repetition penalty: 1 turns it off. Values a little above 1, such as 1.05 to 1.1, are a common starting point. SillyTavern's documentation warns that a repetition penalty range set too high can break responses.
- Frequency and presence penalties: 0 turns them off, and the usual range is -2 to 2. Frequency penalty grows with each repeat of a word; presence penalty applies once a word has appeared at all. Small values such as 0.1 to 0.4 are a common starting point.
Use one kind of penalty at a time and keep it small. If a bot loops, raising the temperature a little or regenerating often fixes it faster than a heavy penalty.
Starting values for roleplay
These are starting points for the common cases, not tested optima for any one model. Change one setting at a time and regenerate a few replies before you judge it.
| Goal | Temperature | Max new tokens | Min P | Repetition penalty |
|---|---|---|---|---|
| Balanced roleplay (start here) | 0.6 to 0.8 | 400 to 600 | 0.05 | 1.0 (off) or 1.05 |
| More creative, surprising replies | 0.9 to 1.0 | 400 to 800 | 0.05 to 0.1 | 1.05 |
| Consistent, in-character, fewer slips | 0.4 to 0.6 | 300 to 500 | 0.05 | 1.0 (off) |
| Small or older models (8B to 12B) | 0.3 to 0.6 | 300 to 400 | 0.1 | 1.05 to 1.1 |
In JanitorAI, only the temperature and max new tokens columns apply, plus context size from the section above. Min P and repetition penalty need an app that sends them, such as SillyTavern, or a preset attached to your proxy key (see below).
Which settings each free model accepts
Models do not all accept the same settings. A setting the model does not list is still sent by your app, but it may have no effect, so a preset tuned around min P can behave very differently on a model that ignores it. This table lists every free model on Composite with the settings its catalog entry lists, taken from Composite's public model list on November 19, 2026. Every model with a list accepts temperature, top P and max tokens.
| Model | Context window | Top K | Min P | Repetition penalty | Frequency / presence | Reasoning |
|---|---|---|---|---|---|---|
DeepSeek V4 Flash 0731deepseek/deepseek-v4-flash-0731:free | 1,048,576 | Yes | Yes | Yes | Yes | Yes |
DeepSeek V4.1 Flashdeepseek/deepseek-v4.1-flash:free | 1,048,576 | Yes | Yes | Yes | Yes | Yes |
Gemma 3 4Bgoogle/gemma-3-4b-it:free | 131,072 | Yes | Yes | Yes | Yes | No |
Gemma 4 26B A4Bgoogle/gemma-4-26b-a4b-it:free | 262,144 | No | No | No | No | Yes |
GLM 5.3 Flashz-ai/glm-5.3-flash:free | 1,048,576 | Yes | Yes | Yes | Yes | Yes |
Ling 3.0 Flashinclusionai/ling-3.0-flash:free | 262,144 | Yes | Yes | Yes | Yes | Yes |
Llama 3 8B Lunarissao10k/l3-lunaris-8b:free | 8,192 | Yes | Yes | Yes | Yes | No |
MiMo-V2.5xiaomi/mimo-v2.5:free | 1,000,000 | Yes | No | No | Yes | Yes |
MiMo-V2.6-Flashxiaomi/mimo-v2.6-flash:free | 1,050,000 | Yes | Yes | Yes | Yes | Yes |
Mistral Nemomistralai/mistral-nemo:free | 131,072 | Yes | Yes | Yes | Yes | No |
Muse Spark 1.2 Contributormeta/muse-spark-1.2-contributor:free | 1,048,576 | Yes | No | Yes | No | Yes |
Muse Spark 1.3 Contributormeta/muse-spark-1.3-contributor:free | 1,048,576 | Yes | No | Yes | No | Yes |
Nex-N2.5-Mininex-agi/nex-n2.5-mini:free | 262,144 | Yes | No | No | No | Yes |
Nova Micro 1.0amazon/nova-micro-v1:free | 128,000 | Yes | No | No | No | No |
Qwen3.7 Flashqwen/qwen3.7-flash:free | 1,000,000 | No | No | No | Presence only | Yes |
Synth 2.5 Flash Previewlucidityai/synth-2.5-flash | 30,000 | Not listed | Not listed | Not listed | Not listed | Not listed |
Dots3-Note Previewdots-studio/dots-3-note-preview:free | 512,000 | No | No | No | No | Yes |
ERNIE-5.5-Flash-Previewbaidu/ernie-5.5-flash-preview:free | 1,024,000 | No | No | No | No | Yes |
Flintrecursive/flint-v1.0:free | 262,144 | No | No | No | No | Yes |
Inkling 2 Smallthinkingmachines/inkling-2-small:free | 1,048,576 | Yes | Yes | Yes | Yes | Yes |
LFM3-1.7Bliquid/lfm-3-1.7b:free | 262,144 | Yes | Yes | Yes | Yes | Yes |
Ling 3.2 Flashinclusionai/ling-3.2-flash:free | 1,048,576 | Yes | No | No | Yes | Yes |
Nemotron 3 Ultranvidia/nemotron-3-ultra-550b-a55b:free | 1,000,000 | No | No | No | No | Yes |
Nemotron 3.5 Lightningnvidia/nemotron-3.5-lightning:free | 1,000,000 | No | No | No | No | Yes |
Nemotron 3.5 Super VLnvidia/nemotron-3.5-super-vl:free | 1,000,000 | No | No | No | No | Yes |
North Mini Codecohere/north-mini-code:free | 256,000 | Yes | No | No | Yes | Yes |
Particle-1particle-ai/particle-1:free | 1,048,576 | Yes | Yes | Yes | Yes | Yes |
Synth 2.5 Pro Previewlucidityai/synth-2.5-pro:free | 30,000 | Not listed | Not listed | Not listed | Not listed | Not listed |
"Not listed" means the catalog publishes no settings list for that model, not that it rejects settings; try changes and compare a few replies. For roleplay presets that rely on min P and repetition penalty, the models with "Yes" in both columns, such as DeepSeek V4.1 Flash, GLM 5.3 Flash and MiMo-V2.6-Flash, apply them as written. The free list changes over time; the models page always has the current one.
Fixes for common problems
- Replies stop mid-sentence: raise max new tokens by 100 to 200. On reasoning models, lower the reasoning effort as well.
- Replies are too long, or the bot writes your character's actions: lower max new tokens to 300 to 400, and say in the custom prompt that the bot only writes for its own character. A lower ceiling gives it less room to keep going after its own turn.
- The same phrases every reply: raise temperature by 0.1, or add a small repetition or frequency penalty. Check that the model accepts the penalty you use in the table above.
- Nonsense, random topic changes or the wrong language: lower temperature by 0.1 to 0.2, or add min P 0.05 on a model that accepts it.
- The bot forgets early events: context size is too small for the story, or the history no longer fits. Raise it up to the model's limit, trim the card, or carry key facts in chat memory or a lorebook.
- Slow first word or a context-window error: context size is too large. Lower it, or use a model with a bigger window.
Setting samplers on the Composite side
JanitorAI only exposes three settings, but a proxy can add more. On Composite, a preset attached to your API key can carry its own sampling values: temperature, top P, top K, min P, repetition, frequency and presence penalties, and max tokens. A preset value is only used when your app did not send that field itself, so your JanitorAI sliders still win, and it is only applied on models that list the setting. That makes a key preset a way to give JanitorAI chats min P or a repetition penalty they could not otherwise use. Presets made by the community are in Discovery, and the preset stack reference explains the order in which several presets apply.
Two more options help with repetitive prose. The free [tool: antislop] tool tells every model to avoid overused words and stock phrases such as "tapestry" and "barely above a whisper", and on models that accept token biasing it also makes those words less likely at the source. The [ORCHESTRATOR] helper reads the recent chat and picks temperature, top P, top K and reasoning effort for each reply; it uses your provider or credits. Both are described in the tags and tools reference.
New to proxies? The JanitorAI setup guide connects a Composite key in a few minutes, and Synth 2.5 Flash is free on every account with no daily cap, which makes it a good model to test settings on.
JanitorAI generation settings FAQ
What is the best temperature for JanitorAI?
JanitorAI's own guide suggests starting at 0.6 and raising it by 0.1 at a time until replies feel lively, staying below 1.0. Most proxy models do well between 0.6 and 0.8 for roleplay. Some models want less: Mistral recommends 0.3 for Mistral Nemo.
What should I set max new tokens to in JanitorAI?
For normal roleplay, 400 to 600 tokens is a good range, which is roughly 300 to 450 English words. Raise it if replies stop mid-sentence, and give reasoning models more room because their thinking can use part of the same budget. Max new tokens is a ceiling, not a target, so a high value does not make replies longer by itself.
What context size should I use with a proxy?
Set it to the model's context window minus your max new tokens, or lower. For a 30,000-token model with 600 max new tokens, that is about 29,000. On million-token models, a smaller value such as 32,000 to 64,000 keeps replies faster and cheaper, and a memory layer or summary can carry older events.
Why does JanitorAI cut off replies mid-sentence?
The reply hit the max new tokens limit. Raise max new tokens by 100 to 200, or ask for shorter replies in the custom prompt. On reasoning models, lower the reasoning effort so less of the budget goes to thinking.
Do sampler settings work with every model?
No. Each model accepts its own set of settings, and the ones it does not list may have no effect. Every free model on Composite listed with samplers takes temperature, top P and max tokens, but min P, top K and the repetition penalties vary by model. The table on this page shows which free models accept which.
Sources
- JanitorAI: Understanding Temperature
- JanitorAI: Tokens, your AI's memory budget
- SillyTavern: Common settings
- Composite: model list (sampler support, retrieved November 19, 2026)