add optional web search functionality with UI updates and backend integration
This commit is contained in:
@@ -87,6 +87,40 @@ The browser UI remains framework-free and uses native ES modules. Shared behavio
|
||||
|
||||
Pure chat message, retry, memory, summary, and persona-state formatting lives in `src/services/chatMessageHelpers.js`; `chatService.js` remains responsible for orchestration and persistence.
|
||||
|
||||
## Optional Web Search
|
||||
|
||||
The globe button next to the composer enables search for one message. It defaults to off and resets once the server accepts the message. The browser sends `webSearch: true` (or `false`) through the existing authenticated `POST /chat/stream`; `POST /chat` supports the same optional boolean. Omitting it preserves the normal chat flow. No generic tool/agent system is needed.
|
||||
|
||||
The existing chat service saves the user message, performs one server-side SearXNG search using the current message (whitespace normalized, at most 500 characters), then supplies up to five numbered results as a separate context alongside the existing history, model, personas and generation settings. Instructions require numbered citations, acknowledgement of insufficient results and treating all result content as untrusted data. This is a basic prompt-injection precaution, not a guarantee that a model will follow instructions. No result pages are fetched.
|
||||
|
||||
Set environment variables in the server/container environment, or load the example explicitly with `node --env-file=.env server.js` (the existing npm start command does not load `.env` automatically):
|
||||
|
||||
```dotenv
|
||||
SEARXNG_URL=http://searxng:8080
|
||||
WEB_SEARCH_TIMEOUT_MS=8000
|
||||
```
|
||||
|
||||
`SEARXNG_URL` is optional and server-only. It must be an HTTP(S) base URL reachable from Express, for example over the same Docker network. Use the final URL: redirects are rejected. Missing/invalid configuration leaves ordinary chat available and returns `Web search is currently unavailable.` for search requests. `WEB_SEARCH_TIMEOUT_MS` limits the whole search response, including body download (default 8000 ms). Responses are limited to 1 MiB; each source has at most a 300-character title, 2048-character URL and 1500-character snippet. Only HTTP(S) source links without embedded credentials are accepted. No user input can override the provider URL.
|
||||
|
||||
Enable JSON in SearXNG's `settings.yml`:
|
||||
|
||||
```yaml
|
||||
search:
|
||||
formats:
|
||||
- html
|
||||
- json
|
||||
```
|
||||
|
||||
The provider calls `GET {SEARXNG_URL}/search?q=...&format=json` using encoded query parameters. SearXNG must allow requests from the application server; disabled JSON output returns HTTP 403. See the [official Search API documentation](https://docs.searxng.org/dev/search_api.html). Only the current query is sent to SearXNG, not conversation history or personas; configured SearXNG engines can forward that query to their search services.
|
||||
|
||||
The server-side `WebSearchProvider` JSDoc contract is `search(query, {signal, limit}?) -> Promise<SearchResult[]>`; `SearXNGSearchProvider` is the initial implementation. Results have `{title, url, snippet}`. This repository uses JavaScript ES modules, so types are documented with JSDoc rather than introducing a TypeScript build.
|
||||
|
||||
Search adds SSE `status` events: `{status: "searching"}` after message acceptance and `{status: "generating", sourceCount}` before model generation. Existing `chunk` events are unchanged. `done` adds optional `sources` to `{reply}`; the non-streaming response does likewise. Empty results produce `sources: []` and an explicit insufficient-results instruction. Failures produce the existing error event (HTTP 503 on `/chat`); no model answer is generated after search failure. The saved failed user message remains visible, and the composer restores its text for resending without search.
|
||||
|
||||
Sources appear below assistant messages as numbered links. They persist in a nullable SQLite `chat_messages.sources` JSON column, added automatically on startup using the existing additive schema mechanism; no manual migration is needed. Old messages remain valid. Sources are preserved in JSON exports/imports and branches. Regenerating an answer reuses its stored results without performing another search; a fresh search requires sending another message with the toggle enabled.
|
||||
|
||||
Run `node --test --test-isolation=none` and `npm run lint` to check the feature and existing flows. `test/webSearch.test.js` uses local HTTP protocol fixtures and an isolated in-memory database; no production search results are mocked. Live SearXNG/Ollama verification still requires configured, reachable services. There is no TypeScript or build command in this framework-free project.
|
||||
|
||||
## Legal
|
||||
|
||||
- [Privacy Policy](PRIVACY_POLICY.md)
|
||||
|
||||
Reference in New Issue
Block a user