i]dZddlmZddlZddlmcmZddl Z ddl m Z ddl m Z mZmZddlZ ddlmZmZmZ d ddZdddZGd d ZGd d ZGd dZGddZGddZGddZGddZGddZej@jBGddZ"y#e$r dZdZdZYwxYw)u Lightpanda 크롤러 래퍼 테스트 단위 테스트: mock 사용 (네트워크 불필요) 통합 테스트: @pytest.mark.integration — 실제 lightpanda 서버(ws://127.0.0.1:9333) 필요 ) annotationsN)fields) AsyncMock MagicMockpatch) CrawlError CrawlResultLightpandaCrawlerc&t}t||_t||_ttd|_t|_d|_td|_td|_tg|_ |S)u$playwright Page 모의 객체 생성 return_value)statushttps://example.com Test PageN) rrtitlecontentgotocloseurlevaluatequery_selectorquery_selector_all)rrpages 2/home/jay/workspace/tools/tests/test_lightpanda.py_make_mock_pagerso ;D.DJ'2DLy'<=DIDJ$DH;7DM#6D'R8D Kc| t}t}t||_t|_td|_|S)u'playwright Browser 모의 객체 생성r T)rrrnew_pager is_connected)rbrowsers r_make_mock_browserr"/s> | kG d3GKGM$$7G Nrc,eZdZddZddZddZddZy)TestLightpandaCrawlerInitc>d}t|u}|stjd|fdt|fdtjvstj trtj tndtj |dz}tjddzd|iz}ttj|dx}}t}|j}d }||k(}|stjd |fd ||fd tjvstj |rtj |nd tj |tj |d z}dd|iz}ttj|dx}x}}y)u;CDP endpoint 기본값이 ws://127.0.0.1:9333 인지 확인Nis notz%(py0)s is not %(py3)sr py0py3uLightpandaCrawler import 실패 >assert %(py5)spy5zws://127.0.0.1:9333==z4%(py2)s {%(py2)s = %(py0)s.cdp_endpoint } == %(py5)scrawlerr*py2r-assert %(py7)spy7) r @pytest_ar_call_reprcompare @py_builtinslocals_should_repr_global_name _saferepr_format_assertmsgAssertionError_format_explanation cdp_endpoint self @py_assert2 @py_assert1 @py_format4 @py_format6r1 @py_assert4 @py_assert3 @py_format8s rtest_default_cdp_endpointz3TestLightpandaCrawlerInit.test_default_cdp_endpoint@s(,O ,OOO OOOOOO OOO OOOOOO.OOOOOOO#%##<'<<#'<<<<<#'<<<<<<r?r@s rtest_custom_cdp_endpointz2TestLightpandaCrawlerInit.test_custom_cdp_endpointFs(,, ,,,, ,,,,,, ,,, ,,,,,,,,,,#1FG##<'<<#'<<<<<#'<<<<<< timeout_msr@s rtest_default_timeoutz.TestLightpandaCrawlerInit.test_default_timeoutLs(,, ,,,, ,,,,,, ,,, ,,,,,,,,,,#%!!*U*!U****!U******w***w***!***U*******rcd}t|u}|stjd|fdt|fdtjvstj trtj tndtj |dz}dd|iz}ttj|dx}}t}|j}d}||k(}|stjd |fd ||fd tjvstj |rtj |nd tj |tj |d z}d d|iz}ttj|dx}x}}y)u)Chrome fallback endpoint 기본값 확인Nr&r(r r)rKr-zws://127.0.0.1:9222r.)z7%(py2)s {%(py2)s = %(py0)s.chrome_endpoint } == %(py5)sr1r2r4r5) r r6r7r8r9r:r;r=r>chrome_endpointr@s rtest_chrome_fallback_endpointz7TestLightpandaCrawlerInit.test_chrome_fallback_endpointRs(,, ,,,, ,,,,,, ,,, ,,,,,,,,,,#%&&?*??&*?????&*???????w???w???&???*????????rNreturnNone)__name__ __module__ __qualname__rIrLrOrRrrr$r$?s= = + @rr$c$eZdZddZddZddZy)TestCrawlResultDataclasscd}t|u}|stjd|fdt|fdtjvstj trtj tndtj |dz}tjddzd|iz}ttj|dx}}ttDchc]}|j}}hd }|j}||}|stjd ||z d zd tjvstj |rtj |nd tj |d tjvstj |rtj |nd tj |dz}ttj|dx}}ycc}w)u>CrawlResult 에 필수 필드가 모두 존재하는지 확인Nr&r(r r)uCrawlResult import 실패r,r-> rhtmlmetatextlinksrenginer elapsed_msu누락 필드: zL >assert %(py5)s {%(py5)s = %(py2)s {%(py2)s = %(py0)s.issubset }(%(py3)s) }required field_namesr*r3r+r-) r r6r7r8r9r:r;r<r=r>rnameissubset) rArBrCrDrEfrdrcrFs rtest_required_fields_existz3TestCrawlResultDataclass.test_required_fields_exist_s!"&C{$&CCC{$CCCCCC{CCC{CCC$CCC(CCCCCCC'-k':;!qvv; ;f  Y -Y-YYKAW@X/YYYYYYYxYYYxYYY YYYYYYYYYYYY-YYYYYYzhttps://example.com/page descriptiontestrgY@ lightpanda rrr_r]r`r^rrbrar.z+%(py2)s {%(py2)s = %(py0)s.url } == %(py5)sresultr2r4r5)z.%(py2)s {%(py2)s = %(py0)s.engine } == %(py5)sPassert %(py6)s {%(py6)s = %(py0)s(%(py3)s {%(py3)s = %(py1)s.links }, %(py4)s) } isinstancelistr*py1r+py4py6zOassert %(py6)s {%(py6)s = %(py0)s(%(py3)s {%(py3)s = %(py1)s.meta }, %(py4)s) }dict)r r6r7r8r9r:r;r=r>rrar`rsrtr^ry) rArBrCrDrErqrFrGrH @py_assert5 @py_format7s rtest_instantiationz+TestCrawlResultDataclass.test_instantiationfs"&&{$&&&&{$&&&&&&{&&&{&&&$&&&&&&&% -.(  zz222z22222z2222222v222v222z22222222222}}, ,} ,,,,} ,,,,,,v,,,v,,,},,, ,,,,,,, ,,-z,--------z---z------&---&---,------------------- ++,z+t,,,,,,,,z,,,z,,,,,,&,,,&,,,+,,,,,,t,,,t,,,,,,,,,,rc d}t|u}|stjd|fdt|fdtjvstj trtj tndtj |dz}dd|iz}ttj|dx}}dD]}td d d d gid d | }|j}||k(}|stjd|fd||fdtjvstj |rtj |ndtj |dtjvstj |rtj |nddz}dd|iz} ttj| dx}} y)u8engine 필드는 'lightpanda' 또는 'chrome' 중 하나Nr&r(r r)rKr-rnchromez https://x.comrgror.)z.%(py2)s {%(py2)s = %(py0)s.engine } == %(py4)srrar*r3rwzassert %(py6)srx) r r6r7r8r9r:r;r=r>ra) rArBrCrDrErarrG @py_format5r{s rtest_engine_field_valuesz1TestCrawlResultDataclass.test_engine_field_valuesys "&&{$&&&&{$&&&&&&{&&&{&&&$&&&&&&&. &F# A88 %8v% % % %8v % % % % % %1 % % %1 % % %8 % % % % % %v % % %v % % % % % % % &rNrS)rVrWrXrir|rrYrrr[r[^sZ-&&rr[ceZdZejdZej jddZej jddZ ej jddZ ej jddZ y) TestFetchUnitc#:Ktd5}t}t||j_td|j_t }t |}t||j_|||fdddy#1swYyxYww)z playwright connect_over_cdp mock)tools.lightpanda_crawler.async_playwrightr FN rrr __aenter__ __aexit__rr"chromiumconnect_over_cdprAmock_pw mock_instancerr!s rmock_playwright_connectz%TestFetchUnit.mock_playwright_connects> ? )7%KM.7].SG +-6E-JG *"$D(.G6?W6UM " " 37D( ( ) ) ) BA8B BBBcxKd}t|u}|stjd|fdt|fdtjvstj trtj tndtj |dz}dd|iz}ttj|dx}}|\}}}t4d{} | jdd{} dddd{t t} | s d d tjvstj trtj tnd d tjvstj | rtj | nd d tjvstj trtj tnd tj | d z} ttj| d} y7O797,#1d{7swY=xYww)u2fetch() 가 CrawlResult 를 반환하는지 확인Nr&r(r r)rKr-r5assert %(py4)s {%(py4)s = %(py0)s(%(py1)s, %(py2)s) }rsrqr r*rvr3rw) r r6r7r8r9r:r;r=r>fetchrsr ) rArrBrCrDrE_r!rr1rqrGrs rtest_fetch_returns_crawl_resultz-TestFetchUnit.test_fetch_returns_crawl_resultsQ)-, ,,,, ,,,,,, ,,, ,,,,,,,,,,27D$& @ @'"==)>??F @ @&+........z...z......&...&......+...+.......... @? @ @ @ @s[C H: H H:H$%H&H$* H:5H!6D&H:H$!H:$H7*H- +H72H:c Kd}t|u}|stjd|fdt|fdtjvstj trtj tndtj |dz}dd|iz}ttj|dx}}|\}}}t4d{} | jdd{} dddd{d }t |} | sd d tjvstj trtj tnd d tjvstj | rtj | nd tj |tj | d z}ttj|dx}} d}t| |} | sd d tjvstj trtj tnd d tjvstj | rtj | nd tj |tj | d z}ttj|dx}} d}t| |} | sd d tjvstj trtj tnd d tjvstj | rtj | nd tj |tj | d z}ttj|dx}} d}t| |} | sd d tjvstj trtj tnd d tjvstj | rtj | nd tj |tj | d z}ttj|dx}} y777#1d{7swYxYww)uBfetch() 결과에 title/text/html/links 필드가 있는지 확인Nr&r(r r)rKr-rrz5assert %(py5)s {%(py5)s = %(py0)s(%(py1)s, %(py3)s) }hasattrrq)r*rvr+r-r_r]r`) r r6r7r8r9r:r;r=r>rr) rArrBrCrDrErr!rr1rqrFs r%test_fetch_result_has_required_fieldsz3TestFetchUnit.test_fetch_result_has_required_fieldss)-, ,,,, ,,,,,, ,,, ,,,,,,,,,,27D$& @ @'"==)>??F @ @ ''wvw''''''''w'''w''''''v'''v'''w''''''''''%&wvv&&&&&&&&w&&&w&&&&&&v&&&v&&&v&&&&&&&&&&%&wvv&&&&&&&&w&&&w&&&&&&v&&&v&&&v&&&&&&&&&&&'wvw''''''''w'''w''''''v'''v'''w''''''''''  @? @ @ @ @s[C S R# SR,%R&&R,* S5R)6N.S&R,)S,R?2R5 3R?:ScKd}t|u}|stjd|fdt|fdtjvstj trtj tndtj |dz}dd|iz}ttj|dx}}|\}}}t4d{} | jdd{} dddd{ j}d} || k(} | stjd | fd || fd tjvstj | rtj | nd tj |tj | d z}d d|iz} ttj| dx}x} } y777#1d{7swYxYww)u6fetch() 결과에 요청 URL이 저장되는지 확인Nr&r(r r)rKr-rr.rprqr2r4r5) r r6r7r8r9r:r;r=r>rrrArrBrCrDrErr!rr1rqrFrGrHs rtest_fetch_url_stored_correctlyz-TestFetchUnit.test_fetch_url_stored_correctlys:)-, ,,,, ,,,,,, ,,, ,,,,,,,,,,27D$& @ @'"==)>??F @ @zz222z22222z2222222v222v222z22222222222 @? @ @ @ @[C G G GG %G&G * G5G6CGG G GG GGcKd}t|u}|stjd|fdt|fdtjvstj trtj tndtj |dz}dd|iz}ttj|dx}}|\}}}t4d{} | jdd{} dddd{ j}d } || k\} | stjd | fd || fd tjvstj | rtj | nd tj |tj | d z}dd|iz} ttj| dx}x} } y777#1d{7swYxYww)u*fetch() elapsed_ms 가 양수인지 확인Nr&r(r r)rKr-rr)>=)z2%(py2)s {%(py2)s = %(py0)s.elapsed_ms } >= %(py5)srqr2r4r5) r r6r7r8r9r:r;r=r>rrbrs rtest_fetch_elapsed_ms_positivez,TestFetchUnit.test_fetch_elapsed_ms_positives;)-, ,,,, ,,,,,, ,,, ,,,,,,,,,,27D$& @ @'"==)>??F @ @  %A% A%%%% A%%%%%%v%%%v%%% %%%A%%%%%%% @? @ @ @ @rNrS) rVrWrXpytestfixturermarkasynciorrrrrYrrrrs ^^ ) ) [[// [[ ( ( [[33 [[&&rrcReZdZejdZej jddZej jddZ ej jddZ ej jddZ ej jddZ y) TestFetchManyUnitc#:Ktd5}t}t||j_td|j_t }t |}t||j_|||fdddy#1swYyxYww)Nrr Frrs rrz)TestFetchManyUnit.mock_playwright_connects > ? )7%KM.7].SG +-6E-JG *"$D(.G6?W6UM " " 37D( ( ) ) )rctKd}t|u}|stjd|fdt|fdtjvstj trtj tndtj |dz}dd|iz}ttj|dx}}gd}t4d{}|j|d{}dddd{tt} | s d d tjvstj trtj tnd d tjvstj |rtj |nd d tjvstj trtj tnd tj | d z} ttj| d} y7O797,#1d{7swY=xYww)u4fetch_many() 가 리스트를 반환하는지 확인Nr&r(r r)rKr-rhttps://example.orgzhttps://example.netrrsresultsrtr) r r6r7r8r9r:r;r=r> fetch_manyrsrt rArrBrCrDrEurlsr1rrGrs rtest_fetch_many_returns_listz.TestFetchManyUnit.test_fetch_many_returns_listsB)-, ,,,, ,,,,,, ,,, ,,,,,,,,,,T$& 5 5'#..t44G 5 5'4((((((((z(((z(((((('((('((((((4(((4(((((((((( 54 5 5 5 5s[CH8 H H8H"#H$H"( H83H4D&H8H"H8"H5(H+ )H50H8cTKd}t|u}|stjd|fdt|fdtjvstj trtj tndtj |dz}dd|iz}ttj|dx}}gd}t4d{}|j|d{}dddd{t}d } || k(} | stjd | fd || fd tjvstj trtj tnd d tjvstj |rtj |nd tj |tj | dz} dd| iz} ttj| dx}x} } y7?7)7#1d{7swY-xYww)uCfetch_many() 결과 수가 입력 URL 수와 일치하는지 확인Nr&r(r r)rKr-rr.z0%(py3)s {%(py3)s = %(py0)s(%(py1)s) } == %(py6)slenrr*rvr+rxassert %(py8)spy8 r r6r7r8r9r:r;r=r>rr rArrBrCrDrErr1rrzrFr{ @py_format9s r#test_fetch_many_count_matches_inputz5TestFetchManyUnit.test_fetch_many_count_matches_inputsF)-, ,,,, ,,,,,, ,,, ,,,,,,,,,,T$& 5 5'#..t44G 5 57| q |q    |q      s   s      7   7   |   q        54 5 5 5 5s[CH( H  H(H#H $H( H(3H4DH( HH(H%H H% H(cKd}t|u}|stjd|fdt|fdtjvstj trtj tndtj |dz}dd|iz}ttj|dx}}gd}t4d{}|j|d{}dddd{d D}t|} | sd d tjvstj trtj tnd tj |tj | d z} ttj| dx}} y777#1d{7swYxYww) uCfetch_many() 결과가 모두 CrawlResult 인스턴스인지 확인Nr&r(r r)rKr-rc3<K|]}t|tywNrsr .0rs r zFTestFetchManyUnit.test_fetch_many_all_crawl_results..?!:a-?,assert %(py4)s {%(py4)s = %(py0)s(%(py2)s) }allr) r r6r7r8r9r:r;r=r>rrrs r!test_fetch_many_all_crawl_resultsz3TestFetchManyUnit.test_fetch_many_all_crawl_resultss)-, ,,,, ,,,,,, ,,, ,,,,,,,,,,T$& 5 5'#..t44G 5 5@w??s?????????s???s?????????????? 54 5 5 5 5s[CG F( GF.#F*$F.( G3F,4B5G*F.,G.G4F7 5G<GcpKd}t|u}|stjd|fdt|fdtjvstj trtj tndtj |dz}dd|iz}ttj|dx}}t4d{}|jgd{}dddd{g}|k(}|stjd|fd ||fd tjvstj |rtj |nd tj |dz}dd|iz}ttj|dx}}y777#1d{7swYxYww) u:fetch_many() 빈 리스트 입력 시 빈 리스트 반환Nr&r(r r)rKr-r.z%(py0)s == %(py3)sr) r r6r7r8r9r:r;r=r>r)rArrBrCrDrEr1rs rtest_fetch_many_empty_listz,TestFetchManyUnit.test_fetch_many_empty_lists )-, ,,,, ,,,,,, ,,, ,,,,,,,,,,$& 3 3'#..r22G 3 3w"}w"ww" 32 3 3 3 3s[CF6FF6 F!F F!$ F6/F0B,F6F!F6!F3'F* (F3/F6cXKd}t|u}|stjd|fdt|fdtjvstj trtj tndtj |dz}dd|iz}ttj|dx}}dd g}t4d{}|j|d d{}dddd{t}d } || k(} | stjd | fd|| fdtjvstj trtj tnddtjvstj |rtj |ndtj |tj | dz} dd| iz} ttj| dx}x} } y7A7)7#1d{7swY-xYww)u?fetch_many() concurrency 파라미터가 수용되는지 확인Nr&r(r r)rKr-rr concurrencyr.rrrrrrrrs r!test_fetch_many_concurrency_paramz3TestFetchManyUnit.test_fetch_many_concurrency_param sZ)-, ,,,, ,,,,,, ,,, ,,,,,,,,,,%'<=$& D D'#..t.CCG D D7| q |q    |q      s   s      7   7   |   q        DC D D D Ds[CH* H  H*H%H&H* H*5H6DH*HH*H'H H'"H*NrS) rVrWrXrrrrrrrrrrrYrrrrs ^^ ) ) [[)) [[!! [[@@ [[ [[!!rrceZdZejdZej jddZej jddZ y)TestEvaluateUnitc#\Ktd5}t}t||j_td|j_t }td|_t|}t||j_ |||fdddy#1swYyxYww)Nrr FExample Domain) rrr rrrrr"rrrs rrz(TestEvaluateUnit.mock_playwright_connects > ? )7%KM.7].SG +-6E-JG *"$D%3CDDM(.G6?W6UM " " 37D( ( ) ) )s B,B B  B, B)%B,c~Kd}t|u}|stjd|fdt|fdtjvstj trtj tndtj |dz}dd|iz}ttj|dx}}|\}}}t4d{} | jdd d{} dddd{d } |k(}|stjd |fd | |fd tjvstj | rtj | nd tj |dz}dd|iz}ttj|dx}}y777#1d{7swYxYww)u9evaluate() 가 JS 실행 결과를 반환하는지 확인Nr&r(r r)rKr-rdocument.titlerr.rrq) r r6r7r8r9r:r;r=r>r) rArrBrCrDrErr!rr1rqs rtest_evaluate_returns_valuez,TestEvaluateUnit.test_evaluate_returns_value-s*)-, ,,,, ,,,,,, ,,, ,,,,,,,,,,27D$& U U'"++,ACSTTF U U*)v)))))v)))))))v)))v))))))))))) UT U U U Us[C F= F" F=F(&F$'F(+ F=6F&7B,F=$F(&F=(F:.F1 /F:6F=cfKd}t|u}|stjd|fdt|fdtjvstj trtj tndtj |dz}dd|iz}ttj|dx}}|\}}}t4d{} | jdd d{dddd{|jjy7I717##1d{7swY3xYww) u7evaluate() 가 page.evaluate 를 호출하는지 확인Nr&r(r r)rKr-rr) r r6r7r8r9r:r;r=r>rassert_called_once) rArrBrCrDrErr!rr1s r!test_evaluate_calls_page_evaluatez2TestEvaluateUnit.test_evaluate_calls_page_evaluate8s)-, ,,,, ,,,,,, ,,, ,,,,,,,,,,27D$& L L'""#8:JK K K L L ((* L K L L L LsZC D1 D D1D&D'D+ D16D7 D1DD1D."D% #D.*D1NrS) rVrWrXrrrrrrrrYrrrrsT ^^ ) ) [[** [[++rrceZdZejdZej jddZej jddZ ej jddZ ej jddZ y)TestExtractStructuredUnitc#Ktd5}t}t||j_td|j_t }td|_td|_t}t||_ t||g|_ t|}t||j_ ||||fdddy#1swYyxYww)Nrr Fz Hello World attr_value)rrr rrr inner_text get_attributerrrr"rr)rArrmock_elrr!s rrz1TestExtractStructuredUnit.mock_playwright_connectJs > ? 27%KM.7].SG +-6E-JG *kG!* !FG $-<$HG !"$D"+"AD &/gw=O&PD #(.G6?W6UM " " 37D'1 1 2 2 2s C,C C  C, C)%C,cKd}t|u}|stjd|fdt|fdtjvstj trtj tndtj |dz}dd|iz}ttj|dx}}|\}}}} t4d{} | jdd d id{} dddd{t t} | s d d tjvstj trtj tnd d tjvstj | rtj | nd dtjvstj trtj tndtj | dz} ttj| d} y7R797,#1d{7swY=xYww)u8extract_structured() 가 dict 를 반환하는지 확인Nr&r(r r)rKr-rrh1rrsrqryr) r r6r7r8r9r:r;r=r>extract_structuredrsry)rArrBrCrDrErr!rrr1rqrGrs r$test_extract_structured_returns_dictz>TestExtractStructuredUnit.test_extract_structured_returns_dict]s\)-, ,,,, ,,,,,, ,,, ,,,,,,,,,,$;!7D'$& ^ ^'"556KgW[_]]F ^ ^&$''''''''z'''z''''''&'''&''''''$'''$'''''''''' ^] ^ ^ ^ ^s[C H> HH>H()H"*H(. H>9H%:D&H>"H(%H>(H;.H1 /H;6H>cKd}t|u}|stjd|fdt|fdtjvstj trtj tndtj |dz}dd|iz}ttj|dx}}|\}}}} t4d{} | jdd d id{} dddd{d } |  v}|stjd |fd | | ftj | d tjvstj | rtj | nd dz}dd|iz}ttj|dx} }y777#1d{7swYxYww)u@extract_structured() 결과에 요청한 키가 있는지 확인Nr&r(r r)rKr-rrrinz%(py1)s in %(py3)srqrvr+ r r6r7r8r9r:r;r=r>r rArrBrCrDrErr!rrr1rq @py_assert0s r#test_extract_structured_key_presentz=TestExtractStructuredUnit.test_extract_structured_key_presenths-)-, ,,,, ,,,,,, ,,, ,,,,,,,,,,$;!7D'$& ^ ^'"556KgW[_]]F ^ ^ w&    w&   w      &   &        ^] ^ ^ ^ ^[C F? F$F?F*)F&*F*. F?9F(:B+F?&F*(F?*F<0F3 1F<8F?cKd}t|u}|stjd|fdt|fdtjvstj trtj tndtj |dz}dd|iz}ttj|dx}}|\}}}} t4d{} | jdd d id{} dddd{d } |  v}|stjd |fd | | ftj | d tjvstj | rtj | nd dz}dd|iz}ttj|dx} }y777#1d{7swYxYww)u;::attr(name) 문법으로 속성 추출 가능한지 확인Nr&r(r r)rKr-rhrefz a::attr(href)rrrqrrrs r#test_extract_structured_attr_syntaxz=TestExtractStructuredUnit.test_extract_structured_attr_syntaxss.)-, ,,,, ,,,,,, ,,, ,,,,,,,,,,$;!7D'$& h h'"556KfVeMfggF h hvvv hg h h h hrc 0Kd}t|u}|stjd|fdt|fdtjvstj trtj tndtj |dz}dd|iz}ttj|dx}}|\}}}} t4d{} | jdd d id{} dddd{ j}d } || } t| t}|sGd d tjvstj trtj tnd d tjvstj | rtj | nd tj |tj | tj | dtjvstj trtj tndtj |dz}ttj|dx}x} x} }y777#1d{7swYxYww)u;복수형 키(links)는 리스트를 반환하는지 확인Nr&r(r r)rKr-rr`aznassert %(py10)s {%(py10)s = %(py0)s(%(py7)s {%(py7)s = %(py3)s {%(py3)s = %(py1)s.get }(%(py5)s) }, %(py8)s) }rsrqrt)r*rvr+r-r5rpy10) r r6r7r8r9r:r;r=r>rgetrsrt)rArrBrCrDrErr!rrr1rqrF @py_assert6 @py_assert9 @py_format11s r/test_extract_structured_plural_key_returns_listzITestExtractStructuredUnit.test_extract_structured_plural_key_returns_list~s)-, ,,,, ,,,,,, ,,, ,,,,,,,,,,$;!7D'$& ] ]'"556KgWZ^\\F ] ]!**4W4*W-4z-t44444444z444z444444&444&444*444W444-444444t444t4444444444 ]\ ] ] ] ]s[C J I7JJ)I:*J. J9I=:E>J:J=JJJ JJNrS) rVrWrXrrrrrrrrrrYrrrrIs ^^22$ [[(( [[!! [[   [[55rrceZdZejj ddZejj ddZejj ddZy)TestChromeFallbackc Kd}t|u}|stjd|fdt|fdtjvstj trtj tndtj |dz}dd|iz}ttj|dx}}td5}t}t| |j_ td |j_ t}t|}tt!d |g |j"_t4d{} | j&}d } || k(} | stjd| fd|| fdtjvstj | rtj | ndtj |tj | dz}dd|iz} ttj| dx}x} } dddd{dddy77#1d{7swYxYw#1swYyxYww)uCLightpanda 연결 실패 시 Chrome으로 fallback 되는지 확인Nr&r(r r)rKr-rr FzLightpanda connection refused side_effectrr.z/%(py2)s {%(py2)s = %(py0)s._engine } == %(py5)sr1r2r4r5)r r6r7r8r9r:r;r=r>rrr rrrr" Exceptionrr_engine) rArBrCrDrErrrchrome_browserr1rFrGrHs r*test_chrome_fallback_on_lightpanda_failurez=TestChromeFallback.test_chrome_fallback_on_lightpanda_failures)-, ,,,, ,,,,,, ,,, ,,,,,,,,,, > ? 37%KM.7].SG +-6E-JG *"$D/5N7@=>"7M " " 3)* 3 3g2(2(2222(222222w222w222222(2222222 3 3! 3 3  3 3 3 3 3! 3 3shCIB I  H3I C H7 I %H5&I * I3I 5I 7I =I>I I  IIc Kd}t|u}|stjd|fdt|fdtjvstj trtj tndtj |dz}dd|iz}ttj|dx}}td5}t}t| |j_ td |j_ t}t|}t| |j _t4d{} | j$}d } || k(} | stjd | fd || fdtjvstj | rtj | ndtj |tj | dz}dd|iz} ttj| dx}x} } dddd{dddy77#1d{7swYxYw#1swYyxYww)uBLightpanda 연결 성공 시 engine 이 'lightpanda' 인지 확인Nr&r(r r)rKr-rr Frnr.rr1r2r4r5)r r6r7r8r9r:r;r=r>rrr rrrr"rrr) rArBrCrDrErrr lp_browserr1rFrGrHs r$test_engine_is_lightpanda_on_successz7TestChromeFallback.test_engine_is_lightpanda_on_successs)-, ,,,, ,,,,,, ,,, ,,,,,,,,,, > ? 77%KM.7].SG +-6E-JG *"$D+D1J6?Z6XM " " 3(* 7 7g6,6,6666,666666w666w666666,6666666 7 7 7 7 7 7 7 7 7 7 7shCI A?IH(IC H, IH*I I (I*I,H> 2H53H> :II I c nKd}t|u}|stjd|fdt|fdtjvstj trtj tndtj |dz}dd|iz}ttj|dx}}td5}t}t| |j_ td |j_ t}t|}t| |j _t4d{} | j%d d{} | j&}| j(} || k(} | stjd | fd || fdtjvstj | rtj | ndtj |dtjvstj | rtj | ndtj | dz} dd| iz}ttj|dx}x} } dddd{dddy7J747#1d{7swY#xYw#1swYyxYww)uECrawlResult.engine 이 실제 사용 엔진과 일치하는지 확인Nr&r(r r)rKr-rr Frr.)zK%(py2)s {%(py2)s = %(py0)s.engine } == %(py6)s {%(py6)s = %(py4)s._engine }rqr1)r*r3rwrxrr)r r6r7r8r9r:r;r=r>rrr rrrr"rrrrar)rArBrCrDrErrrrr1rqrzrGr{rs r(test_crawl_result_engine_matches_crawlerz;TestChromeFallback.test_crawl_result_engine_matches_crawlers)-, ,,,, ,,,,,, ,,, ,,,,,,,,,, > ? 87%KM.7].SG +-6E-JG *"$D+D1J6?Z6XM " " 3(* 8 8g&}}-BCC}}77}7777}777777v777v777}7777777777777777777 8 8 8 8 8C 8 8 8 8 8 8szCJ5A?J)J J)JJDJ3 J)>J?J) J5 J)JJ)J& JJ& "J))J2.J5NrS) rVrWrXrrrrrr rYrrrrsZ [[33. [[77  [[88rrceZdZddZej j ddZej j ddZej j ddZ y)TestErrorHandlingcd}t|u}|stjd|fdt|fdtjvstj trtj tndtj |dz}tjddzd|iz}ttj|dx}}ttt}|sd d tjvstj trtj tnd dtjvstj trtj tndd tjvstj trtj tnd tj |d z}ttj|d}y) u5CrawlError 커스텀 예외가 존재하는지 확인Nr&r(rr)uCrawlError import 실패r,r-r issubclassrr) rr6r7r8r9r:r;r<r=r>rr)rArBrCrDrErGrs rtest_crawl_error_existsz)TestErrorHandling.test_crawl_error_existss!%Az%AAAzAAAAAAzAAAzAAAAAA'AAAAAAA*i00000000z000z000000*000*000000i000i0000000000rc Kd}t|u}|stjd|fdt|fdtjvstj trtj tndtj |dz}dd|iz}ttj|dx}}d}t|u}|stjd|fdt|fdtjvstj trtj tndtj |dz}dd|iz}ttj|dx}}td 5}t}t| |j_ td |j_ttd  |j _t%j&t5t4d{} dddd{ddddddy7&7#1d{7swY'xYw#1swY+xYw#1swYyxYww)u7Lightpanda + Chrome 모두 실패 시 CrawlError 발생Nr&r(r r)rKr-rrr FzConnection refusedr)r r6r7r8r9r:r;r=r>rrrr rrrrrrraises)rArBrCrDrErrr1s r+test_both_endpoints_fail_raises_crawl_errorz=TestErrorHandling.test_both_endpoints_fail_raises_crawl_errors)-, ,,,, ,,,,,, ,,, ,,,,,,,,,,!%%z%%%%z%%%%%%z%%%z%%%%%%%%%% > ? 7%KM.7].SG +-6E-JG *6?IVjLk6lM " " 3z* ,.'       sE6I8A>I6IH, I H0 IH.II# I,I.I0I 6H97I >II IIIc Kd}t|u}|stjd|fdt|fdtjvstj trtj tndtj |dz}dd|iz}ttj|dx}}td5}t}t| |j_ td |j_ t}t|}t| |j _t4d{} t%j&t(5} | j+d d{dddg}d } j,} t/| } || v} | }| sd }| j,}t/|}||v}|}|s:tjd| fd|| ftj |dtjvstj t.rtj t.nddtjvstj | rtj | ndtj | tj | dz}dd|iz}|j1|| stjdfdftj |dtjvstj t.rtj t.nddtjvstj | rtj | ndtj tj |dz}dd|iz}|j1|tj2|diz}dd|iz}ttj|dx}x}x}x} x} x} x}x}x}}dddd{dddy77#1swYxYw7 #1d{7swY0xYw#1swYyxYww)u2screenshot() 호출 시 NotImplementedError 발생Nr&r(r r)rKr-rr FrzPlaywright+Chromeu 스크린샷r)zM%(py3)s in %(py10)s {%(py10)s = %(py5)s(%(py8)s {%(py8)s = %(py6)s.value }) }strexc_info)r+r-rxrrz%(py12)spy12)zR%(py15)s in %(py22)s {%(py22)s = %(py17)s(%(py20)s {%(py20)s = %(py18)s.value }) })py15py17py18py20py22z%(py24)spy24zassert %(py27)spy27)r r6r7r8r9r:r;r=r>rrr rrrr"rrrrNotImplementedError screenshotvaluerappend_format_boolop)rArBrCrDrErrrr!r1r @py_assert7rrFr @py_assert14 @py_assert19 @py_assert21 @py_assert16r @py_format13 @py_format23 @py_format25 @py_format26 @py_format28s r&test_screenshot_raises_not_implementedz8TestErrorHandling.test_screenshot_raises_not_implementeds)-, ,,,, ,,,,,, ,,, ,,,,,,,,,, > ? k7%KM.7].SG +-6E-JG *"$D(.G6?W6UM " " 3(* k kg]]#67D8!,,-BCCCDj*j(..jc..Aj*.AAj^j[c[i[ijWZ[iWjj^WjEjjjjj*.Ajjj*jjjjjjcjjjcjjjjjj(jjj(jjj.jjj.Ajjjjjjj^Wjjjj^jjjjjjWZjjjWZjjjjjj[cjjj[cjjj[ijjjWjjjjjjjjjjjjjjjj k k k k kCDD k k k k k ksCQ$A?QP.QQ P45P1 6P4:JQ Q Q!Q% Q$.Q1P44P> 9QQQ Q  Q QQ!Q$cKd}t|u}|stjd|fdt|fdtjvstj trtj tndtj |dz}dd|iz}ttj|dx}}td5}t}t| |j_ td |j_ t}t|}t| |j _t4d{} t%j&t(5| j+d d{ddddddd{dddy7X7'#1swY&xYw7#1d{7swY-xYw#1swYyxYww) u+pdf() 호출 시 NotImplementedError 발생Nr&r(r r)rKr-rr Fr)r r6r7r8r9r:r;r=r>rrr rrrr"rrrrrpdf) rArBrCrDrErrrr!r1s rtest_pdf_raises_not_implementedz1TestErrorHandling.test_pdf_raises_not_implementedsL)-, ,,,, ,,,,,, ,,, ,,,,,,,,,, > ? =7%KM.7].SG +-6E-JG *"$D(.G6?W6UM " " 3(* = =g]]#67=!++&;<<<= = = = = =<== = = = = = =sCGA?GFGF- F5F 6F:F- G F+G GGFF( $F-+G-F? 3F64F? ;GG GNrS) rVrWrXrrrrrr.r1rYrrr r sc1  [[  [[kk$ [[==rr ceZdZejj ddZejj ddZejj ddZejj ddZ y)TestIntegrationc Kd}t|u}|stjd|fdt|fdtjvstj trtj tndtj |dz}dd|iz}ttj|dx}}t4d{}|jdd{}dddd{tt}|s d d tjvstj trtj tnd d tjvstj |rtj |nd d tjvstj trtj tnd tj |d z}ttj|d}|j}|sydd tjvstj |rtj |nd tj |dz} ttj| d}|j}|sydd tjvstj |rtj |nd tj |dz} ttj| d}|j}d} || k(}|stjd|fd|| fd tjvstj |rtj |nd tj |tj | dz}dd|iz} ttj| dx}x}} |j}t|t } | sdd tjvstj trtj tnd d tjvstj |rtj |nd tj |dtjvstj t rtj t ndtj | dz} ttj| dx}} |j"}d} || kD}|stjd|fd|| fd tjvstj |rtj |nd tj |tj | dz}dd|iz} ttj| dx}x}} |j$}d} || v}|stjd|fd|| fd tjvstj |rtj |nd tj |tj | dz}dd|iz} ttj| dx}x}} y777#1d{7swYxYww)uK실제 example.com 크롤링 — title/text/html/links 필드 존재 확인Nr&r(r r)rKr-rrrsrqr rz)assert %(py2)s {%(py2)s = %(py0)s.title })r*r3z(assert %(py2)s {%(py2)s = %(py0)s.html }r.rpr2r4r5rrrtrur>)z1%(py2)s {%(py2)s = %(py0)s.elapsed_ms } > %(py5)sr~r)z.%(py2)s {%(py2)s = %(py0)s.engine } in %(py5)s)r r6r7r8r9r:r;r=r>rrsr rr]rr`rtrbra)rArBrCrDrEr1rqrGr @py_format3rFrHrzr{s rtest_fetch_example_comz&TestIntegration.test_fetch_example_coms)-, ,,,, ,,,,,, ,,, ,,,,,,,,,,$& @ @'"==)>??F @ @&+........z...z......&...&......+...+..........|||vv|{{{vv{zz222z22222z2222222v222v222z22222222222 ,,-z,--------z---z------&---&---,-------------------  $1$ 1$$$$ 1$$$$$$v$$$v$$$ $$$1$$$$$$$}}8 88} 88888} 8888888v888v888}888 88888888 @? @ @ @ @s[C[[[ [ [ [ $ [/[0W[[ [ [[ [[cKd}t|u}|stjd|fdt|fdtjvstj trtj tndtj |dz}dd|iz}ttj|dx}}gd}t4d{}|j|d d{}dddd{t}d }||k(} | stjd | fd ||fd tjvstj trtj tnd dtjvstj |rtj |ndtj |tj |dz} dd| iz} ttj| dx}x} }d|D}t|} | sddtjvstj trtj tndtj |tj | dz} ttj| dx}} y777#1d{7swYxYww)u/3개 URL 병렬 크롤링 — 결과 수 확인Nr&r(r r)rKr-rrrr.rrrrrrc3<K|]}t|tywrrrs rrz=TestIntegration.test_fetch_many_three_urls..1rrrrr) r r6r7r8r9r:r;r=r>rrr)rArBrCrDrErr1rrzrFr{rrGrs rtest_fetch_many_three_urlsz*TestIntegration.test_fetch_many_three_urls$s)-, ,,,, ,,,,,, ,,, ,,,,,,,,,,  %& D D'#..t.CCG D D7| q |q    |q      s   s      7   7   |   q       ?w??s?????????s???s??????????????  DC D D D Ds[CK J: KK%J=&K* K5K6GK=KKK K KKcKd}t|u}|stjd|fdt|fdtjvstj trtj tndtj |dz}dd|iz}ttj|dx}}t4d{}|jdd d{}dddd{tt}|s d d tjvstj trtj tnd d tjvstj |rtj |nd d tjvstj trtj tnd tj |dz}ttj|d}t|}d} || kD} | stjd| fd|| fdtjvstj trtj tndd tjvstj |rtj |nd tj |tj | dz} dd| iz} ttj| dx}x} } y7`7I7<#1d{7swYMxYww)u+evaluate() 로 document.title 반환 확인Nr&r(r r)rKr-rrrrsrrrrr5)z/%(py3)s {%(py3)s = %(py0)s(%(py1)s) } > %(py6)srrrr) r r6r7r8r9r:r;r=r>rrsrr) rArBrCrDrEr1rrGrrzrFr{rs rtest_evaluate_document_titlez,TestIntegration.test_evaluate_document_title3s)-, ,,,, ,,,,,, ,,, ,,,,,,,,,,$& T T'!**+@BRSSE T T%%%%%%%%%z%%%z%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%5zAzA~zAss55zA  TS T T T Ts[CML&M L/ L)!L/% M0L,1H6M)L/,M/M5L8 6M=McKd}t|u}|stjd|fdt|fdtjvstj trtj tndtj |dz}dd|iz}ttj|dx}}t4d{}|jdd d d d{}dddd{tt}|s d d tjvstj trtj tnd dtjvstj |rtj |nddtjvstj trtj tndtj |dz}ttj|d}d} | |v}|stjd|fd| |ftj | dtjvstj |rtj |nddz}dd|iz}ttj|dx} }d} | |v}|stjd|fd| |ftj | dtjvstj |rtj |nddz}dd|iz}ttj|dx} }|d}t|t} | sdd tjvstj trtj tnd tj |dtjvstj trtj tndtj | dz}ttj|dx}} y77|7o#1d{7swYxYww)u+CSS 셀렉터로 구조화 데이터 추출Nr&r(r r)rKr-rrr)headingr`rrsrqryrr?rrrr`z5assert %(py5)s {%(py5)s = %(py0)s(%(py2)s, %(py3)s) }rtre) r r6r7r8r9r:r;r=r>rrsryrt) rArBrCrDrEr1rqrGrrrFs rtest_extract_structured_cssz+TestIntegration.test_extract_structured_css=s{)-, ,,,, ,,,,,, ,,, ,,,,,,,,,,$&  '"55% 3/F   &$''''''''z'''z''''''&'''&''''''$'''$''''''''''"yF""""yF"""y""""""F"""F""""""" w&    w&   w      &   &        /0z/400000000z000z000/000000400040000000000     s[CQ;QQ; Q%#Q$Q%( Q;3Q"4M)Q;Q%"Q;%Q8+Q. ,Q83Q;NrS) rVrWrXrrrr8r;r=r@rYrrr3r3sz [[ 9 9 [[ @ @ [[ [[ 1 1rr3)rzxTest Page

Hello

link)rrrrrTrr)rzMagicMock | NonerTr)#__doc__ __future__rbuiltinsr8_pytest.assertion.rewrite assertionrewriter6r dataclassesr unittest.mockrrrrtools.lightpanda_crawlerrr r ImportErrorrr"r$r[rrrrrr r integrationr3rYrrrLs#55 SSN  " @@>*&*&d<&<&HB!B!T#+#+V>5>5L:8:8D:=:=D616161CKJs B>> C  C