Disclosure: Composite runs this site, and two of the fixes below are Composite features. Every other fix works with any proxy or model.

Short answer: the bot has not lost anything. Each reply is written from a fixed-size prompt, and once your story is longer than that prompt can hold, the oldest messages are left out. The fix is to make room (a correct context size, a shorter card), to carry the important facts forward in a small note (Chat Memory, a summary, a lorebook), or to let the proxy summarize old history for you.

Why AI roleplay bots forget

A language model has no memory between replies. Every time you send a message, your frontend builds one prompt that contains the character card, the preset, any lorebook entries and as much of the chat history as fits, and the model writes its reply from that prompt alone. The limit on how much fits is the model's context window, measured in tokens. A token is roughly three quarters of an English word.

When the history outgrows the window, something has to go, and it is almost always the oldest messages. That is why a bot remembers what you said five minutes ago but not the name of the tavern from the first scene. JanitorAI's help center puts JLLM, its free model, at around 8k to 9k tokens, and says that when the model runs out of room "it loses track of who you are, what you were doing, or even the tone of the conversation."

A larger window delays the problem but does not end it. Every token in the prompt is sent and billed again on each reply, long prompts take longer to start streaming, and models are less reliable about details buried in the middle of a very long prompt. JanitorAI's guide calls this "context rot."

What fills the context window

The history only gets what the fixed parts leave behind. These are example sizes for a typical setup, not measurements; your app's token counter shows the real numbers for your card and preset.

Part of the promptExample tokens
Character card (personality, scenario, example dialogue)1,500 to 3,000
Preset or custom prompt500 to 3,000
Lorebook entries that fired this turn0 to 2,000
Chat memory, summary or author's note200 to 1,000
Room kept free for the reply300 to 1,000
Chat historyWhatever is left

With those example sizes on an 8,000-token window, the fixed parts can take more than half the space, leaving room for perhaps fifteen to twenty exchanges of history. On a 128,000-token model the same setup leaves room for hundreds, which is why switching models is often the first fix people notice.

Signs the bot has run out of room

  • It forgets a name, a promise or an injury from early in the story but remembers recent events clearly.
  • Its personality drifts toward a generic assistant voice as the chat gets longer.
  • It brings back a character who left, or describes a place you already destroyed.
  • The proxy returns an error saying the request exceeds the context window. The proxy error guide covers that one.

If the bot forgets something from two messages ago, the cause is usually the model or the sampling settings, not the window.

Nine fixes, cheapest first

FixJanitorAISillyTavernFree
Set the context size to match the modelYesYesYes
Shorten the card and presetYesYesYes
Chat Memory boxYesNoYes
Summarize extensionNoYesYes
Author's NoteNoYesYes
World Info / lorebooks in the appNoYesYes
Lorebook attached to a Composite keyYesYesYes
Composite state toolYesYesYes
Composite premium memory layerYesYesNo

1. Set the context size to match the model

Both JanitorAI and SillyTavern let you set a context size in their generation settings. Set it to the model's context window minus your maximum reply length. Too high and the request fails or the proxy trims it somewhere you did not choose; too low and you throw away history the model could have read. The Models page lists the window for every model on Composite.

2. Shorten the card and the preset

Every token in the card and preset is sent on every reply, so a 4,000-token card on an 8,000-token model leaves little for the story. Cut example dialogue to two or three exchanges, remove instructions that repeat each other, and move world details that only matter now and then into a lorebook (fix 6), where they are sent only when they come up.

3. Use JanitorAI's Chat Memory box

Chat Memory is a text box in the chat menu whose contents are sent with your messages, so whatever you put there stays in view after the early scenes are gone. JanitorAI's guide recommends short bullet points under headings like these, facts rather than scenes, and describing what a character is rather than what they did:

Environment: abandoned lighthouse on the north coast, winter, night
Relationship: Mira distrusts {{user}} but owes them her life
Current plot: searching for the missing ledger; the harbor master lied
{{char}}: injured left arm, carries her brother's compass
{{user}}: posing as a merchant named Teo
Past events: the fire at the docks (chapter 1), the deal with the smugglers

Keep it short; the box takes tokens from the same window as everything else. To refresh it, ask the model for a recap every so often and paste a corrected version in. A prompt like this works with most models:

(OOC: Pause the roleplay. Summarize the story so far as bullet points under
Environment, Relationship, Current plot, {{char}}, {{user}} and Past events.
Facts only, under 200 words. Do not continue the story.)

Check the recap before you paste it. Models get small facts wrong, and a wrong fact in memory is repeated on every reply.

4. Turn on SillyTavern's Summarize extension

SillyTavern's Summarize extension writes a running summary of the chat and inserts it into the prompt, updating every set number of messages or words. You can read and edit the summary, roll it back, or pause it and write your own. Its documentation is frank that summaries "may lose some important details or contain hallucinations," so read it every few dozen messages and fix what is wrong. If you change to a model with a different window, re-tune the update interval.

5. Pin facts with Author's Note

Author's Note inserts a short text at a depth you choose: depth 0 is the very end of the history, and a note near the end has more influence on the next reply. Use it for the few facts the bot keeps getting wrong, such as the time of day or who is in the room. You can also set it to be inserted only every few messages.

6. Move world details into a lorebook

A lorebook (World Info in SillyTavern) is a set of entries, each with keywords. An entry is added to the prompt only when one of its keywords appears in the recent messages, so a hundred places and side characters cost nothing until the story mentions them. SillyTavern's scan depth sets how many recent messages are checked for keywords, and a token budget caps how much lore goes in at once. JanitorAI offers lorebook-style behavior through its Scripts feature.

On Composite you can also attach lorebooks to the API key itself, from the community library or your own in Create. They then apply in any app the key is used in, including JanitorAI. Adding [AGENTS] to a message lets a helper model search the lorebooks on the key for up to three lookups before the reply; see the reference for options and cost.

7. Track changing facts with the state tool

Some facts change during a story: a door is now open, the sword is broken, it is the third day. Composite's state tool lets the model write those down as named values that are added to every request, up to 32 values. Turn it on with [STATE] in a message, or add [tool: state] to the key's default tags so frontends with no room for tags still get it. It is free, and nothing is stored on Composite's servers: the values are kept inside the replies in your chat, so if you edit or swipe a message, the state follows.

8. Pick a model with a larger window

If your story needs a long reach and the fixes above are not enough, a model with a bigger window holds more history. Weigh it against the cost, since you pay for the whole prompt on each reply. The roleplay model guide compares models for long stories, and the free model shortlist lists the free ones with their windows.

9. Let the proxy handle memory: Composite memory layers

Every fix above happens in your frontend, which means you maintain it. A memory layer does the work at the proxy instead, on every request, in whichever app you use. Each Composite API key has a Memory setting:

  • Satori Lite is the default and free for everyone. When the chat no longer fits, it drops the oldest messages, which is what most proxies do.
  • Satori summarizes old history and brings back earlier messages that matter to the current scene. It suits most long roleplay.
  • Heavy and Ultra are for chats that run to hundreds of turns, and search further back for the details the current scene needs. Ultra also keeps a record of established characters, relationships, location and open plot threads.
  • None sends the chat up to the model's full window with no memory layer, and you pay for every token of it.

Satori, Heavy and Ultra need credits or a plan, and so does None unless the key uses your own provider. A chat that still fits in the window is sent unchanged on every layer, so a premium layer costs nothing extra until the story outgrows the model. The memory lives in your chat, not on Composite's servers. The memory layers guide has the details, and plans and credits lists the prices.

Set it up in five minutes

  1. Connect your frontend to Composite with the JanitorAI guide or the SillyTavern guide.
  2. Set your app's context size to the model's window minus your reply length (fix 1).
  3. On the API Keys page, pick a Memory layer for the key, and add [tool: state] to its default tags if your story has facts that change.
  4. Write a short Chat Memory or Author's Note with the five or six facts that must never be wrong.
  5. Play a long session, then ask the bot a question about the first scene to see what it kept.

AI roleplay memory FAQ

Why does my JanitorAI bot forget things?

Every model reads a fixed number of tokens per reply. The character card, preset and memory take their share first, and once the chat history no longer fits, the oldest messages are left out of the request. The model never sees them again unless something brings them back, such as Chat Memory, a summary or a lorebook entry.

How big is JanitorAI's memory?

JanitorAI's own help center gives JLLM, its free model, a context window of around 8k to 9k tokens. With a proxy, the limit is the context window of the model you pick, up to the context size you set in JanitorAI's generation settings.

Does a bigger context window fix forgetting?

Partly. A larger window holds more history, but every token is sent and billed on every reply, replies get slower, and very long prompts can make a model miss details in the middle. A summary or memory layer that keeps the important facts is usually cheaper than sending the whole chat.

What should I put in JanitorAI Chat Memory?

Short bullet points about facts that must stay true: where the scene is, how the characters feel about each other, open plot threads and important past events. Leave out anything already in the character card. Refresh it every so often by asking the model for a recap and pasting the corrected version in.

What do Composite memory layers do?

They decide what happens when a chat outgrows the model's window. The default, Satori Lite, is free and drops the oldest messages. Satori, Heavy and Ultra need credits or a plan; they summarize old history and keep a record of characters, relationships and open threads. A chat that still fits is sent unchanged on every layer.

Sources