Ki;\dZddlmZddlZddlmZddlmZd dZd dZ d dZ dd Z dd Z y)zMarkdown report generator for evaluation benchmark results. Takes a list of benchmark result dicts and produces a formatted markdown table suitable for inclusion in documentation or CI output. ) annotationsN)Path)Anyc 6|syg}|jd|jdg}t}|D]6}|D]/}|dk7s ||vs|j||j|18|jd|jdddj|zdz}d djd |Dzdz}|j||j||D]f}|j dd }|Dcgc]}t |j |d } }|jd |ddj| zdzh|jd|jd|jd|D]z}|j dd }|jd||jd|D]+}|j |d } |jd|d| -|jd|dj|Scc}w)aGenerate a markdown report from benchmark results. Each result dict should contain at minimum a ``benchmark`` key identifying the benchmark name, plus any metric keys (e.g. ``ratio``, ``reduction_percent``, ``mrr``, ``precision``, ``recall``, ``f1``). Args: results: List of result dicts from benchmark runs. Returns: A markdown string containing a summary table and per-benchmark details. z5# Evaluation Report No benchmark results to report. # Evaluation Report benchmarkz ## Summaryz| Benchmark |  |  |z| --- | c3 K|]}dywz---N.0_s k/home/jay/workspace/scripts/.codegraph-venv/lib/python3.12/site-packages/code_review_graph/eval/reporter.py z+generate_markdown_report..0s'@!'@ unknown-| z ## Detailsz### z- **z**:  )appendsetaddjoingetstr) resultslinesall_keysseenrkheader separatornamevaluesvs rgenerate_markdown_reportr*s  IE LL&' LLHUD  AKATM"   LL LL  8 4 4t ;FUZZ'@x'@@@4GI LL LL Auu[),.67#aeeAsm$77 r$s^ejj&884?@A  LL LL LL uu[), tD6]# R ,Aa A LL4s$qc* + ,  R 99U #8s"Hcg}t|jd|dD]>}t|d5}tj|}|j |ddd@|S#1swYLxYw)z@Read all CSV files matching a prefix from the results directory.z*_z_*.csvr)newlineN)sortedglobopencsv DictReaderextend) results_dirprefixrowspfreaders r _read_csvsr9Jsv!#D K$$r&%89 :  !R  A^^A&F KK     K  s 'A$$A- c(g}|jddj|zdz|jddjd|Dzdz|D](}|jddj|zdz*dj|S)z-Build a markdown table from headers and rows.rr r c3 K|]}dywr rrs rrz_md_table..Xs":Q5":rr)rr)headersr5r rows r _md_tabler>Ts E LL 7++d23 LL ":'":::TAB4 TEJJsO+d234 99U c pt|}g}|jd|jd|jd|jd|jd|jd|jd|jdgd}|D]}t||}|s|jdd j }|jd ||jdt |d j }|Dcgc]"}|Dcgc]}|j|d c}$} }}|jt|| |jdt|d kr"|jd|jddj|Scc}wcc}}w)arGenerate a full markdown evaluation report from CSV result files. Reads all CSV files in *results_dir*, groups them by benchmark type, and produces a markdown report with methodology notes and per-benchmark result tables. Args: results_dir: Directory containing CSV result files. Returns: Markdown string with the full report. rrz## Methodologyz9Benchmarks are run against real open-source repositories.z=Token counts use a consistent `len(text) // 4` approximation.z1Impact accuracy uses graph edges as ground truth.)token_efficiencyimpact_accuracyflow_completenesssearch_qualitybuild_performancer z## rrzNo benchmark results found.r) rrr9replacetitlelistkeysrr>lenr) r3r benchmark_typesbtyper5rIr<r#h table_rowss rgenerate_full_reportrQ^sq{#KE LL&' LL LL!" LL LLLM LLPQ LLDE LLO! +u-  c3'--/ s5']# RtAw||~&@DE1g6quuQ}6E E Yw 34 R  5zQ 23 R 99U 7Es F2F-6F2-F2c t|}g}t|d}|r|jd|jdgd}g}|D]}|j|jdd|jdd|jdd|jd d|jd d|jd d|jd dg|jt |||jdt|d }t|d}t|d}|s|s|r|jd|jdgd}i} i} |D]7}|jdd| j |jddid<9|D]7}|jdd| j |jddid<9|D]a}|jdd} | j | i | j | gjt |jddcg}t| jD]\} } | j| g} | r*ttt| t| z dnd}|j| t| jddt| jdd|g|jt |||jdt|d}|r|jd|jdgd}g}|D]h}|j|jdd|jdd|jdd|jdd|jddgj|jt |||jd|syd j|S#ttf$rY/wxYw)!a=Generate concise README-ready tables from CSV result files. Produces three tables: - Table A: Token Efficiency - Table B: Accuracy & Quality - Table C: Performance Args: results_dir: Directory containing CSV result files. Returns: Markdown string with the three tables. rAz### Token Efficiencyr)RepoFilesz Naive TokenszStandard Tokensz Graph Tokensz Naive/Graphz Std/Graphrepor changed_files naive_tokensstandard_tokens graph_tokensnaive_to_graph_ratiostandard_to_graph_ratiorBrCrDz### Accuracy & Quality)rSz Impact F1z Flow Recallz Search MRRf1?recallreciprocal_rankrrEz### Performance)rSrTNodesz Flow Det. (s)z Search (ms) file_count node_countflow_detection_seconds search_avg_mszNo benchmark results found. r)rr9rrr> setdefaultfloat ValueError TypeErrorr-itemsrroundsumrLr)r3r te_rowsr<rPr#ia_rowsfc_rowssq_rows repo_data mrr_accumrUdmrr_valsmrrbp_rowss rgenerate_readme_tablesrws{#KE&89G +, R   A   fc"os+nc*'-nc*,c2/5    Yw 34 R&78G&9:G&67G'W -. RD24 ,.  RAABtSAQI vs!3R 8 > R ZAEFUU8UXEYI vs!3R 8 B Z A55%D  r * $$T2.55eAEEBSUV=O>)rzlist[dict[str, Any]]returnr)r3rr4rrxzlist[dict[str, str]])r<z list[str]r5zlist[list[str]]rxr)r3z str | Pathrxr) __doc__ __future__rr0pathlibrtypingrr*r9r>rQrwrr?rr}s4 # 9x2jfr?