Fretux/KrisHD_AI
3
0
Fork 0
Code Issues 3 Pull Requests Actions Packages Projects 1 Releases Wiki Activity

show reasoning in frontend and some other stuff

Browse Source
This commit is contained in:
fretuks
2026-09-21 20:24:40 +02:00
parent ceb3c752ff
commit fd76a79aaf
12 changed files with 220 additions and 89 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+14 -4
View File
@@ -33,9 +33,13 @@ Node.js 22 or newer is required.
- `CHAT_LIMIT`: per-account maximum chat sessions.
- `CHAT_HISTORY_LIMIT`: number of recent messages included in model prompts.
- `MODEL_API_BASE_URL`: base URL of the remote model backend. The server expects `/chat` and `/tags`.
- `MODEL_REQUEST_TIMEOUT_MS`: timeout for chat generation requests.
- `MODEL_REQUEST_TIMEOUT_MS`: legacy fallback for the inactivity timeout (default 180000 ms).
- `MODEL_CONNECT_TIMEOUT_MS`: time to receive Ollama response headers (default 30000 ms).
- `MODEL_INACTIVITY_TIMEOUT_MS`: maximum silence between received stream chunks (default 180000 ms).
- `MODEL_MAX_GENERATION_MS`: overall generation safety limit (default 600000 ms).
- `OLLAMA_KEEP_ALIVE`: Ollama model residency after a request (default `30m`).
- `MODEL_LIST_TIMEOUT_MS`: timeout for model list requests.
- `MODEL_UNLOAD_AFTER_MS`: delay before an idle model unload is attempted.
- `MODEL_UNLOAD_AFTER_MS`: delay before idle local request bookkeeping is cleared. Model unloading is managed by Ollama through `OLLAMA_KEEP_ALIVE`.
- `MODELS_CACHE_TTL_MS`: model list cache TTL.
- `AUTH_RATE_LIMIT_*`: rate-limit window and thresholds for registration/login.
- `CHAT_RATE_LIMIT_*`: rate-limit window and thresholds for chat and retry endpoints.
@@ -46,12 +50,18 @@ Node.js 22 or newer is required.
The server is configured around an Ollama-compatible API surface:
- `POST /chat`: accepts `{ model, messages, stream: true }` and returns newline-delimited JSON stream chunks with `message.content`.
- `POST /chat`: accepts `{ model, messages, stream: true, think: boolean, keep_alive: "30m" }` and returns newline-delimited JSON chunks with optional `message.content` / `message.thinking`, ending in `done: true`.
- `GET /tags`: returns the available model list.
`qwen3:14b` is the default model for chat and roleplay. The model selector loads available models from `/tags` and selects `qwen3:14b` when available (otherwise the first returned model). Existing chats retain their saved model selection.
Chat replies display Ollama's optional `message.thinking` output in a collapsed “Reasoning” section above the answer, updated through SSE `thinking` events during generation. This text is stored separately from the answer and excluded from conversation prompts. An additive SQLite migration preserves it when reopening chats; exports, imports and branches preserve it too. Models without thinking output show no section.
The composer has independent Web Search and Reasoning toggles, both off by default and reset on server acceptance. It sends `{message, chatId, model, webSearch, reasoning}` using streamed fetch to `POST /chat/stream`. Existing SSE events carry `status` (accepted/searching/generating), `thinking` (chunk/fullThinking), `chunk` (chunk/fullReply), `done` (reply/sources), and `error`. Final answers stream incrementally, including persona responses. Persona validation still runs; a corrective pass resets provisional output before streaming its replacement. Only the validated complete answer is persisted.
Every Ollama request explicitly sets `stream: true`, `think`, and `keep_alive`. Reasoning is requested and forwarded only when `reasoning === true` and the model supports the boolean thinking API. The shared conservative capability check supports Qwen3, DeepSeek R1 and DeepSeek v3.1 model IDs; unknown models, Qwen coder variants and models requiring thinking levels have the toggle disabled. Internal calls, roleplay openers and regeneration default to `think: false`. There is no extra model call to obtain reasoning.
Reasoning appears only after actual thinking text arrives, expanded during generation and collapsed when the answer begins. Users can reopen it; stored messages start collapsed. The optional nullable `chat_messages.thinking` field remains separate from answer content and sources, is excluded from future prompts and logs, and survives reloads, exports, imports and branches. Old messages remain valid.
Ollama reads reset the inactivity deadline on each received chunk; the overall safety limit still applies to active streams. Client disconnects and the existing Stop action abort upstream generation. Missing `done: true`, invalid streams, timeouts and upstream failures use the existing error path and never save a partial assistant response as complete. Browser EOF without a terminal SSE event is likewise an interruption. The existing UI discards temporary output on failure.
The refactor adds explicit upstream timeout handling, cached model listing, and `GET /health` for degraded-state visibility.