try to fix gemma messing with other AIs
This commit is contained in:
@@ -107,7 +107,7 @@ Ollama client sends `model: VISION_MODEL`, `think: false`, a JSON Schema in `for
|
||||
and the existing cancellation/timeouts. Its stream is consumed privately; only the
|
||||
final selected chat model's answer and opted-in reasoning reach the user.
|
||||
|
||||
Gemma is the visual perception stage. The user's question guides which visible
|
||||
The configured vision model is the visual perception stage. The user's question guides which visible
|
||||
details it inspects; it does not answer that question. Qwen combines the report
|
||||
with the question, conversation, its own knowledge, and optional Web Search to
|
||||
reason and produce the final answer. The bounded perception record is:
|
||||
@@ -116,30 +116,33 @@ reason and produce the final answer. The bounded perception record is:
|
||||
{
|
||||
"scene": "Detailed factual overview of what is visible",
|
||||
"subjects": [{"type": "bird", "description": "Fluffy gray down", "location": "center"}],
|
||||
"visualDetails": ["Black head with large white facial patches"],
|
||||
"visualDetails": ["The crown is black", "A narrow dark line runs directly beneath the chin"],
|
||||
"visibleText": ["Clearly readable text"],
|
||||
"spatialRelationships": ["The bird faces the camera"],
|
||||
"colorsAndAppearance": ["Gray body and black head"],
|
||||
"recognizedElements": ["Juvenile penguin"],
|
||||
"notableFeatures": ["Large white facial patches"],
|
||||
"notableFeatures": ["Narrow dark line beneath the chin"],
|
||||
"uncertainties": []
|
||||
}
|
||||
```
|
||||
|
||||
Gemma reports direct observations and visually recognizable elements, while avoiding
|
||||
semantic conclusions such as a species inferred from markings or an unseen backend
|
||||
cause. Each field has a length/item limit that allows detailed screenshots. The
|
||||
The vision model reports direct observations and visually recognizable elements,
|
||||
while avoiding semantic conclusions such as a species inferred from markings or an
|
||||
unseen backend cause. The prompt asks for precise location, geometry, boundaries,
|
||||
and named regions where visible, with separate entries for distinguishing markings.
|
||||
Uncertainty concerns what can be seen, not the final identification. Each field has
|
||||
a length/item limit that allows detailed screenshots. The
|
||||
generation cap is 2,400 tokens; simple images can use much less.
|
||||
|
||||
Malformed JSON falls back to a bounded scene plus `rawAnalysis` (up to 6,000
|
||||
characters) as unstructured visual evidence. Valid JSON is parsed before
|
||||
individual fields are bounded, so truncation cannot corrupt its structure.
|
||||
An empty response or unavailable Gemma fails the turn explicitly, without asking the
|
||||
An empty response or unavailable vision model fails the turn explicitly, without asking the
|
||||
chat model to invent image details. The cache key is attachment, configured vision
|
||||
model, `VISION_PROMPT_VERSION` (currently 3), and trimmed user question; case and
|
||||
model, `VISION_PROMPT_VERSION` (currently 4), and trimmed user question; case and
|
||||
internal whitespace are preserved. Records include the analysis and timestamp.
|
||||
Only an exact matching key is reused. Version 2 answer-oriented reports remain in
|
||||
storage but cannot be cache hits for version 3. Legacy model-only snapshots are
|
||||
Only an exact matching key is reused. Earlier prompt-version reports remain in
|
||||
storage but cannot be cache hits for version 4. Legacy model-only snapshots are
|
||||
retained for compatibility but never used as cache hits. Bump the prompt version
|
||||
when changing the perception contract. Cache records cascade away when the
|
||||
attachment is deleted.
|
||||
@@ -149,13 +152,14 @@ attachment order. Earlier images are selected only for explicit historical refer
|
||||
they do not compete with an unrelated current image. Regeneration uses the pinned
|
||||
original prompt and its attachments. Repeated questions and regeneration reuse
|
||||
matching results. Different question results coexist, including across shared
|
||||
branches, so a new question can require extra Gemma calls.
|
||||
branches, so a new question can require extra vision-model calls.
|
||||
|
||||
The final model receives intact structured evidence in a separate untrusted user
|
||||
context message, not a truncated JSON string in a system prompt. Trusted guidance
|
||||
tells Qwen to weigh observed features and visual recognition, perform the final
|
||||
interpretation itself, retain real ambiguity, and distinguish visible UI state from
|
||||
unseen backend causes. It should not mention internal tools/models unless explicitly asked.
|
||||
tells Qwen to weigh discriminating observed features, compare plausible candidates,
|
||||
and keep observed details distinct from inferred traits. It must not sharpen vague
|
||||
descriptions into specific markings or invent a backend cause. It should not mention
|
||||
internal tools/models unless explicitly asked.
|
||||
OCR and raw fallback text are data, never instructions. Analysis remains separate
|
||||
from visible messages; the final model never receives image bytes or filesystem paths.
|
||||
These are model instructions, not a deterministic guarantee of identification accuracy
|
||||
@@ -163,7 +167,7 @@ or final-answer wording. Automated tests cover the contract and data flow; seman
|
||||
quality requires evaluation with real images and the deployed models.
|
||||
|
||||
Web Search and Reasoning remain independent. For requested searches, concise visual
|
||||
evidence from recognized elements, visible text, notable features, and scene helps
|
||||
evidence from notable features, visual details, visible text, and scene helps
|
||||
form the query without overriding the user's search decision. Current images take
|
||||
priority over historical ones. Search finishes before final generation;
|
||||
source ordering/rendering is unchanged. The final
|
||||
|
||||
Reference in New Issue
Block a user