Kip2dZddlmZddlZddlZddlZddlmZmZddl m Z m Z eje ZddZddZd d dd Z d dd Z d dd Z d ddZ d ddZy)zHybrid search engine combining FTS5 (BM25) and vector embeddings. Uses Reciprocal Rank Fusion (RRF) to merge results from full-text search and semantic similarity, with query-aware kind boosting and context-file boosting for relevance tuning. ) annotationsN)AnyOptional) GraphStore_sanitize_namec4|j}|jd|jd|j|jd|j|jdjd}tj d||S)zRebuild the FTS5 index from the nodes table. Checks whether the ``nodes_fts`` virtual table exists, clears it, then repopulates it from every row in ``nodes``. Returns: Number of rows indexed. zDROP TABLE IF EXISTS nodes_ftsz CREATE VIRTUAL TABLE nodes_fts USING fts5( name, qualified_name, file_path, signature, tokenize='porter unicode61' ) z INSERT INTO nodes_fts(rowid, name, qualified_name, file_path, signature) SELECT id, name, qualified_name, file_path, COALESCE(signature, '') FROM nodes zSELECT count(*) FROM nodes_ftsrz"FTS index rebuilt: %d rows indexed)_connexecutecommitfetchoneloggerinfo)storeconncounts d/home/jay/workspace/scripts/.codegraph-venv/lib/python3.12/site-packages/code_review_graph/search.pyrebuild_fts_indexrs ;;D LL12LL  KKM LL  KKM LL9 : C C Ea HE KK4e< Lci}|r|js|S|j}tjd|r|js d|d<d|d<d|vrtjd|rd|d<d|vrd |d <|S) apDetect query patterns and return kind-specific boost multipliers. Heuristics: - PascalCase queries (e.g. ``MyClass``) boost Class/Type by 1.5x - snake_case queries (e.g. ``get_users``) boost Function by 1.5x - Queries containing ``.`` boost qualified name matches by 2.0x Returns: Dict mapping node kind strings to boost multipliers. z ^[A-Z][a-z]?ClassType_z[a-zA-Z]Function.@ _qualified)striprematchisuppersearch)queryboostsqs rdetect_query_kind_boostr'Bs "F    A xx"199;wv axBIIk1- z ax"| Mr<)kci}|D]9}t|D])\}\}}|j|dd||zdzz z||<+;t|jdd}|S)a5Merge multiple ranked result lists using Reciprocal Rank Fusion. Each input list contains ``(id, score)`` tuples, ordered by score descending. The RRF score for each item is the sum of ``1 / (k + rank + 1)`` across all lists it appears in, where rank is the 0-based position. Args: *result_lists: Variable number of ranked result lists. k: RRF constant (default 60). Higher values reduce the impact of rank differences. Returns: Merged list of ``(id, rrf_score)`` tuples sorted by score descending. g?rc |dSNrxs rzrrf_merge..s !A$rTkeyreverse) enumerategetsorteditems)r) result_listsscores result_listrankitem_id_scoremergeds r rrf_merger@is{ "F#N '0'= N #D#7F$jj#6D1 9MMF7O NNFLLN EF Mrc$d|jddzdz} |jd||fj}|Dcgc] }|d|d fc}Scc}w#tj$r"}t j d|gcYd}~Sd}~wwxYw)zRun an FTS5 BM25 search against the nodes_fts table. Returns list of ``(node_id, bm25_score)`` tuples. The BM25 score is negated so higher = better (FTS5 returns negative BM25). "z""zOSELECT rowid, rank FROM nodes_fts WHERE nodes_fts MATCH ? ORDER BY rank LIMIT ?rrzFTS5 search failed: %sN)replacer fetchallsqlite3OperationalErrorrwarning)rr$limit safe_queryrowsrowes r _fts_searchrMsu}}S$//#5J || $    (* .22cQ#a&!222  # #/3 s/&AAAAB-B B Bc  ddlm} ||j|} |jr|j dk(rg|j S|j||}g}|D]6\}} |j|} | s|j| j| f8||j S#t$rgcYSwxYw#|j wxYw#t$r"} tjd| gcYd} ~ Sd} ~ wwxYw)zRun a vector similarity search using the embedding store. Returns list of ``(node_id, similarity_score)`` tuples. Gracefully returns an empty list if embeddings are not available. r)EmbeddingStore)modelrrHzEmbedding search failed: %sN) embeddingsrO ImportErrordb_path availablercloser#get_nodeappendid ExceptionrrG) rr$rHrPrO emb_storeresults id_scoresqnscorenoderLs r_embedding_searchras."5==>  &&)//*;q*@ OO  &&uE&:G13I$ 7 E~~b)$$dggu%56 7 OO %  $ OO  4a8 sVB1C CC0C C C1 B?>B?CC D C=7D=Dch|jj}|sgSg}g}|D].}|jd|jd|dd|dg0dj |}|j|d|d} |j ||j } |j} g} | D]I} | dj} | | k(rd}n| j| rd}nd }| j| d |fK| jd d | S#tj$rgcYSwxYw)zFall back to simple LIKE keyword matching. Each word in the query must match independently (AND logic). Returns ``(node_id, score)`` tuples with a basic relevance score. z4(LOWER(name) LIKE ? OR LOWER(qualified_name) LIKE ?)%z AND z1SELECT id, name, qualified_name FROM nodes WHERE z LIMIT ?nameg@rr+rYc |dSr-r.r/s rr1z!_keyword_search.. qtrTr2) lowersplitrXextendjoinr rDrErF startswithsort)rr$rHwords conditionsparamswordwheresqlrJq_lowerr\rK name_lowerr_s r_keyword_searchrusU KKM   !E  J F2 B   4&{avQK01 2 LL $E MM% =eWH MC||C(113 kkmG')G+[&&(  E  " "7 +EED 5)*+ LL^TL2 N#  # # s DD10D1ct|r|jsgS|j}|dz}g}g} t|||}t ||||} |s| r1g} |r| j|| r| j| t| } nt|||} | sgS| } t|}|r t|n t}| Dcgc]\}}| }}}i}d}tdt||D]U}||||z}djd |D}|jd |d |j!}|D] }|||d < Wg}| D]\}}|j#|}|s|d }|d}|d}d}||vr|||z}d|vr,d|vr(|j%|j%vr||dz}|r ||vr|dz}|j|||zf|j'ddg} |D]\}}!t| |k\r| S|j#|}|s+|d }|r||k7r8| jt)|dt)|d||d|d|d|dxsd|d|dd|j+vr|dndt-|!dd | S#t$r!} tj d| Yd} ~ sd} ~ wwxYwcc}}w)!a]Hybrid search combining FTS5 BM25 and vector embeddings via RRF. Attempts FTS5 + embedding search first, falling back to FTS5-only, then keyword LIKE matching if FTS5 is unavailable. Args: store: The graph store to search. query: Search query string. kind: Optional node kind filter (e.g. ``"Function"``, ``"Class"``). limit: Maximum results to return (default 20). context_files: Optional list of file paths. Nodes in these files receive a 1.5x score boost. Returns: List of dicts with node metadata and ``score`` field. rQz'FTS5 unavailable, will use fallback: %sN)rHrPir,c3 K|]}dyw)?Nr.).0rs r z hybrid_search..Ms33s z!SELECT * FROM nodes WHERE id IN ()rYkind file_pathqualified_namer+rrrc |dSr-r.r/s rr1zhybrid_search..krfrTr2rd line_startline_endlanguagero return_type signature) rdrr~rrrrrorrr_)rr rMrZrrGrarXr@rur'setrangelenrjr rDr6rgrlrkeysround)"rr$r~rH context_filesrPr fetch_limit fts_results emb_resultsrLlists_to_merger?keyword_results kind_boosts context_setnode_idr candidate_ids node_rows batch_sizeibatch placeholdersrJrKboostedr_ node_kindrrboostr\ final_scores" r hybrid_searchrs]0   ;;D!)K,.K+-KE!$[A $E5 5QKk   ! !+ .   ! !+ .N+*$[II *%0K(5#m$35K066!W6M6 "IJ 1c-(* 5'aJ/xx3U33 ||/ ~Q ?   (*  'C#&Ic$i  ''(*G 1mmG$ K  $ -.  # [+ +E ; &3%<{{} 4 4 66\22 9 3 SLE/0%1( LL^TL2%'G '  w<5 0 N-mmG$ K I% "3v;/,S1A-BC[)l+JJ-2(m}--8CHHJ-F[)D;*   4 Nw E@!DDE47sJ8 J4 J1J,,J1)rrreturnint)r$strrzdict[str, float])r9list[tuple[int, float]]r)rrr)2)rzsqlite3.Connectionr$rrHrrr)rN) rrr$rrHrrPz str | Nonerr)NNN)rrr$rr~ Optional[str]rHrrzOptional[list[str]]rPrrzlist[dict[str, Any]])__doc__ __future__rloggingr rEtypingrrgraphrr getLogger__name__rrr'r@rMrarurr.rrrs@#  -   8 $!RN@BD    D " " " "  "  "Z- - - - -p)- C C C C  C ' C  CCr