add optional web search functionality with UI updates and backend integration

This commit is contained in:
2026-09-20 20:41:01 +02:00
parent abd5238252
commit 781f15b503
16 changed files with 603 additions and 41 deletions
+34
View File
@@ -87,6 +87,40 @@ The browser UI remains framework-free and uses native ES modules. Shared behavio
Pure chat message, retry, memory, summary, and persona-state formatting lives in `src/services/chatMessageHelpers.js`; `chatService.js` remains responsible for orchestration and persistence.
## Optional Web Search
The globe button next to the composer enables search for one message. It defaults to off and resets once the server accepts the message. The browser sends `webSearch: true` (or `false`) through the existing authenticated `POST /chat/stream`; `POST /chat` supports the same optional boolean. Omitting it preserves the normal chat flow. No generic tool/agent system is needed.
The existing chat service saves the user message, performs one server-side SearXNG search using the current message (whitespace normalized, at most 500 characters), then supplies up to five numbered results as a separate context alongside the existing history, model, personas and generation settings. Instructions require numbered citations, acknowledgement of insufficient results and treating all result content as untrusted data. This is a basic prompt-injection precaution, not a guarantee that a model will follow instructions. No result pages are fetched.
Set environment variables in the server/container environment, or load the example explicitly with `node --env-file=.env server.js` (the existing npm start command does not load `.env` automatically):
```dotenv
SEARXNG_URL=http://searxng:8080
WEB_SEARCH_TIMEOUT_MS=8000
```
`SEARXNG_URL` is optional and server-only. It must be an HTTP(S) base URL reachable from Express, for example over the same Docker network. Use the final URL: redirects are rejected. Missing/invalid configuration leaves ordinary chat available and returns `Web search is currently unavailable.` for search requests. `WEB_SEARCH_TIMEOUT_MS` limits the whole search response, including body download (default 8000 ms). Responses are limited to 1 MiB; each source has at most a 300-character title, 2048-character URL and 1500-character snippet. Only HTTP(S) source links without embedded credentials are accepted. No user input can override the provider URL.
Enable JSON in SearXNG's `settings.yml`:
```yaml
search:
formats:
- html
- json
```
The provider calls `GET {SEARXNG_URL}/search?q=...&format=json` using encoded query parameters. SearXNG must allow requests from the application server; disabled JSON output returns HTTP 403. See the [official Search API documentation](https://docs.searxng.org/dev/search_api.html). Only the current query is sent to SearXNG, not conversation history or personas; configured SearXNG engines can forward that query to their search services.
The server-side `WebSearchProvider` JSDoc contract is `search(query, {signal, limit}?) -> Promise<SearchResult[]>`; `SearXNGSearchProvider` is the initial implementation. Results have `{title, url, snippet}`. This repository uses JavaScript ES modules, so types are documented with JSDoc rather than introducing a TypeScript build.
Search adds SSE `status` events: `{status: "searching"}` after message acceptance and `{status: "generating", sourceCount}` before model generation. Existing `chunk` events are unchanged. `done` adds optional `sources` to `{reply}`; the non-streaming response does likewise. Empty results produce `sources: []` and an explicit insufficient-results instruction. Failures produce the existing error event (HTTP 503 on `/chat`); no model answer is generated after search failure. The saved failed user message remains visible, and the composer restores its text for resending without search.
Sources appear below assistant messages as numbered links. They persist in a nullable SQLite `chat_messages.sources` JSON column, added automatically on startup using the existing additive schema mechanism; no manual migration is needed. Old messages remain valid. Sources are preserved in JSON exports/imports and branches. Regenerating an answer reuses its stored results without performing another search; a fresh search requires sending another message with the toggle enabled.
Run `node --test --test-isolation=none` and `npm run lint` to check the feature and existing flows. `test/webSearch.test.js` uses local HTTP protocol fixtures and an isolated in-memory database; no production search results are mocked. Live SearXNG/Ollama verification still requires configured, reachable services. There is no TypeScript or build command in this framework-free project.
## Legal
- [Privacy Policy](PRIVACY_POLICY.md)