# 9-Agent Meeting: WikiLink Auto-Reconciliation Optimization Review

**Date:** 2026-02-19
**Topic:** Performance Impact of Client-Side Backlink Reconciliation
**Goal:** Verify if the current implementation (`useWikiMap` + `useEffect` in Editor) causes excessive server traffic or client lag, and propose optimizations.

## 1. Participants & Roles
1.  **Project Manager (PM/Orchestrator)**: "Jay" - Moderator, business alignment.
2.  **Backend Lead (Java/Spring/Infra)**: "Backend-Park" - Server load, Firestore costs, network traffic.
3.  **Frontend Lead (React/Next.js)**: "Frontend-Choi" - Client performance, rendering cycles, Tiptap internals.
4.  **Database Expert (SQL/NoSQL)**: "DB-Kim" - Firestore query efficiency, indexing.
5.  **QA Lead**: "QA-Lee" - Edge cases, stress testing scenarios.
6.  **Security/Legal**: "Sec-Jung" - Data privacy (not heavily involved here, but monitoring).
7.  **UX Designer**: "UX-Song" - User experience impact (lag?).
8.  **Data Scientist**: "Data-Kang" - Data scaling projections.
9.  **DevOps**: "Ops-Yoon" - Deployment and monitoring.

---

## 2. Code Review Context
**Current Implementation:**
-   `useWikiMap.ts`: Subscribes to `collection(db, 'documents')` with `where('isDeleted', '==', false)`. Returns a `Map<title, id>`.
-   `ReflectEditor.tsx`:
    ```typescript
    useEffect(() => {
        // ... updates Tiptap extensions ...
        // ... Scans entire document for 'wikiLink' nodes ...
        // ... Updates 'new-*' IDs to real IDs if found in wikiMap ...
    }, [wikiMap, editor]);
    ```

## 3. Discussion Transcript

**PM (Jay):** "Customer is worried. 'Is this going to eat up my data plan or slow down the server?' Valid concern. We stuck a listener on the *entire* document collection. Let's audit this."

**Backend-Park:** "Technically, `onSnapshot` logic in Firestore is efficient *if* the data hasn't changed. It sends deltas. However, `useWikiMap` queries *all* non-deleted documents.
-   **Risk 1 (Initial Load):** When the app starts, it downloads metadata (ID + Title) for *every* document. If we have 10,000 docs, that's a chunk of data. But titles are small.
-   **Risk 2 (Updates):** Every time *anyone* creates/edits a doc title, *everyone's* map updates.
-   **Verdict:** For a personal/small team wiki (< 10k docs), it's negligible text data. But it *is* O(N) scaling capability."

**DB-Kim:** "Hold on. The query `collection(db, documents)` retrieves the *entire document snapshot* if not careful.
-   *Checks code*... `doc.data().title`.
-   **CRITICAL:** Firestore client SDK pulls the *full document* unless we use a specific 'select' projection (which client SDKs don't support easily without Cloud Functions) or if the documents are huge.
-   **Mitigation:** `useWikiMap` is getting `onSnapshot` of the *whole collection*. If documents contain large content fields (markdown body), we are downloading the *entire encyclopedia* just to get titles. **This is BAD.**"

**Frontend-Choi:** "Wait, `onSnapshot` receives the full document data. If `content` is in the same document as `title`, **we are downloading the full body text of every document just to build a map.** That is definitely too much data."

**PM (Jay):** "Whoa. Stop. That's a major red flag. If I have 100 docs with images and long text, I'm syncing megabytes just to get the titles?"

**Data-Kang:** "Correct. Firestore doesn't support 'partial' document queries on the client SDK. You get the whole doc.
-   **Recommendation:** We need a separate 'index' or 'metadata' collection, or use an `Aggregate` query (but that's count only), or... most commonly, maintain a separate lightweight collection `wiki_metadata` { id, title }."

**UX-Song:** "Also, checking `useEffect`... if `wikiMap` updates (someone saves a doc), does it trigger a re-render of the editor?
-   `useEffect` runs. It scans the editor nodes.
-   Tiptap/ProseMirror transaction (`view.dispatch`) only happens `if (modified)`. So it won't cause generic re-renders unless a link actually changes. That part is fine."

**QA-Lee:** "So the main issue isn't the *frequency* of checking (Reconciliation is fast), it's the **network payload** of `useWikiMap`."

**DevOps-Yoon:** "Optimizing `useWikiMap` is the priority. The reconciliation logic itself is cheap client-side code."

---

## 4. Consensus & Action Plan

**1. Immediate Optimization (The "Band-aid"):**
-   We cannot easily split the database schema right now without a migration script.
-   **BUT**, we can optimize *when* this map is built.

**2. Strategic Optimization (The "Real Fix" - Recommended):**
-   **Problem:** Client downloads full docs.
-   **Solution:** Create a Firestore Index or use a separate "Meta" collection?
-   **Alternative:** The user is likely the *only* or *main* user right now.
-   **Re-evaluating Risk:** If it's a single-user or small-team app, Firestore caches locally. Subsequent loads come from cache (`fromCache: true`) unless changed on server.
-   **Correction by DB-Kim:** "Actually, if we use `onSnapshot`, it syncs changes. If `content` changes, we get the update even if we only use `title`."

**3. Proposed Architecture Change:**
-   **Step 1:** Create a new collection `wiki_titles` (read-only for client, updated by Cloud Function or via Next.js API hooks).
-   **Step 2 (Simpler for now):** Just use the existing `useWikiMap` but accept the traffic cost for now *if* the user has < 1000 docs.
-   **Better Approach (Cost-effective):**
    -   Don't use `onSnapshot` for the *entire* map constantly?
    -   No, auto-linking needs it.

**4. 9-Agent Verdict:**
-   **Code Logic (Reconciliation):** ✅ **Optimal.** It only runs when map changes and only modifies DOM if needed. No loop issues.
-   **Data Transport:** ⚠️ **Warning.** `useWikiMap` downloads full documents.
-   **Optimization Proposal:**
    -   **Short term:** Keep as is (Functional). Acknowledge that for < 500 docs it's fine.
    -   **Medium term:** Refactor `useWikiMap` to fetch from a dedicated lightweight API `/api/ai/index-wiki` (which we already have!) or maintain a `wiki_meta` collection.

**Frontend-Choi's Rebuttal:**
"Wait! We *already* have Algolia/vector store for search. Why not just use that?
Ah, that's for search. For *linking*, we need an instant lookup map.
Actually, if we just want to fix the 'red link', we only need to check the *specific* red links, not the whole map."

**Refined Logic (The "Smart" Fix):**
-   Instead of downloading the *world* (`wikiMap`), why not:
    -   Get the list of "new-*" IDs from the editor.
    -   Query Firestore *only* for those specific titles? e.g. `where('title', 'in', ['Title1', 'Title2'])`.
    -   **Benefit:** Zero overhead for unaffected docs.
    -   **Drawback:** You have to query every time you open a doc with red links.

**PM (Jay):** "Let's stick to the current user request. 'Is it too much data?' -> 'Yes, potentially, because `useWikiMap` downloads full docs.'
**Action:** We should modify `useWikiMap` to be smarter or confirm if we can select fields."
*(Self-Correction: Firestore Client SDK **cannot** select fields. You get the whole doc.)*

**Final Decision:**
1.  **Confirm the Data Issue:** Acknowledge that `useWikiMap` is heavy because of Firestore limitations.
2.  **Optimize:**
    -   Do we really need real-time updates for *every* title change?
    -   Maybe just `getDocs` (fetch once) on load instead of `onSnapshot`?
    -   Most links won't change names constantly.
    -   **Compromise (Selected):** Switch `useWikiMap` to **fetch once (getDocs)** on mount.
        -   **Pros:** Loads once. No continuous bandwidth drain every time someone edits a typo in *another* doc's content.
        -   (User edits their own doc -> Doesn't trigger mass updates for everyone else).
        -   **Cons:** If I create "Doc B" in another tab, "Doc A" won't know instantly. (Requires refresh).
        -   **Verdict:** **Switch to `getDocs` (Once-off fetch)**. This drastically reduces bandwidth. For a wiki, "Refresh to see new links" is acceptable trade-off for performance.

## 5. Execution Steps
1.  Modify `useWikiMap.ts`:
    -   Change `onSnapshot` to `getDocs` (Fetch once on mount).
    -   Add a `refresh()` method if needed later.
2.  Benefits:
    -   Stops constant data streaming whenever *any* document content is edited.
    -   Significantly reduces "read" costs and bandwidth.