Arrow 
Before changing a page, preserve the answers you observed, the exact questions and collection context, and the source-page version. Write the proposed change and retest rule in advance. Without a real “before” record, the next observation is a baseline, not proof that the edit improved visibility.
Name the decision the edit is supposed to improve
Start with one observable problem. “The assistant describes our annual plan as monthly” suggests a pricing-clarity correction. “We want more visibility” does not identify which page, fact or buyer decision should change. Write the expected reader benefit even if no assistant changes its answer.
Then separate correction from experimentation. Fix a demonstrably incorrect public price or broken link promptly. Do not keep harmful or misleading content live to preserve an experiment. If there is no time to capture an AI baseline, record that limitation and treat the work as a verified website correction.
Freeze the baseline package before the first edit
A baseline package makes the old state inspectable. Save the relevant page content and published facts as well as the answer evidence. A screenshot of an attractive result alone cannot show whether a later answer used a changed source.
The observation kit contains empty journals and a versioned panel. Its existence does not mean any baseline has been collected. Keep genuine completed observations and potentially sensitive evidence private; publish only the evidence you have permission to share.
| Record | What to preserve | Why it matters |
|---|---|---|
| Question panel | Exact wording, IDs, version and frozen file | Prevents a changed question from masquerading as a content effect. |
| Observation context | Surface, model if known, language, intended market, location evidence, session state and UTC capture time | Makes differences in collection visible. |
| Answer evidence | Full answer, observed source links, capture reference and reviewer | Supports the original mention, citation and recommendation decisions. |
| Source version | Page URL, saved content, relevant facts and publication or capture date | Shows what information was available before the edit. |
| Collection exceptions | Errors, unavailable surfaces and incomplete reviews | Prevents gaps being reconstructed as negative answers. |
Write a change log that describes the intervention
One action entry should connect a buyer question to a page, the observed problem and the actual patch. “Optimized for GEO” is too vague to review. “Added an annual billing label beside the displayed price and clarified the cancellation paragraph” identifies an intervention.
Use a sequence of planned, implemented, retested and closed. Implementation requires a dated patch or deployment reference. Retesting requires real later observations. Closure requires a written decision, which may be “corrected source, answer effect inconclusive.” An edit completed today is not a completed measurement cycle today.
- Before: link the relevant observation IDs and current source version.
- Change: name the exact section, facts and internal links affected; assign an owner.
- Release: retain the patch reference and production verification date.
- Retest: state the panel version, comparable contexts and intended collection window.
- Decision: retain, revise or investigate, with evidence and known confounders.
Specify what a comparable retest would look like
Reuse frozen questions in independent sessions and keep interface, language and collection method comparable. Do not reuse a successful answer under a new record ID. A new model, a changed location setting or a revised prompt can be worth studying, but belongs in a distinct comparison group.
OpenAI documents that memory can personalize responses and searches. Record the session conditions you can observe rather than assuming that a fresh chat removed all personal context. Preserve unknown settings as unknown.
Choose an observation window that permits repeated collection and record when source access was checked. There is no universal waiting period that proves an assistant has incorporated an edit. A before-and-after difference remains descriptive because other sources, systems and buyer context may also have changed.
Illustrative change record: clarifying annual pricing
Illustrative example, not a measured result: a B2B software company finds two captured answers calling its annual price a monthly charge. Its pricing page places the amount beside a vague “per plan” label. The team archives the page, the correct billing reference and both responses, then labels the annual amount clearly.
The action remains implemented until the planned retest exists. If later answers state the correct billing period, the report can say the error was absent in those comparable observations. It cannot claim that two accurate answers prove a permanent correction across all users. If the retest fails to return answers, record failed attempts instead of a 0% error rate.
An unaffected comparison page may help reveal wider changes, but choosing one does not turn this small observational exercise into a controlled causal study. Use the main measurement method for the answer classifications.
If the content is already changed, do not reconstruct a win
When no before evidence exists, report the completed implementation and its direct checks: the corrected fact is visible, the link works, the metadata matches the page and the deployed URL serves the new content. Keep these checks separate from observed AI outcomes.
Collect the first real panel now and label its date honestly. It can become the baseline for the next intervention. The report should distinguish the publication date, the first observation date and any later review date so readers cannot mistake an update timestamp for performance evidence.
Sources and editorial scope
This is Arrow AI's implementation guidance. Examples are illustrative unless identified as dated observations. Source access and good content do not guarantee a recommendation.
Continue through the GEO evidence library, inspect Arrow GEO's measurement limits, or start an audit.