/cjV4dZddlZidddddddddd d d d d ddddddddddddddddddd d!d"d#d$d%d&d'Zd(d)d*d+d,d-d.d/d0Zd1d2d3d4Zid5dd6d d7d d8d d9dd:dd;dddd?dd@ddAddBd!dCd"dDd#dEd$d%d&d(d)dFdGdHdIdJdKdLdMdNdOdPZhdQZidRdSdTdSdUdSdVdSdWdXdYdXdZdXd[dXd\dXd]dXd^dXd_dXd`dXdadadbdadcdddedddddddddddSdfZdSdSdXdXdXdXdXdadgZdhZ diZ djZ dkZ dlZ dmZdnZdoZdpdqdrZdsZdtZduZdvZdwZdxZdzdyZdS){uP Tier 0 faithfulness assessment — deterministic, no LLM, no network. Checks each AI turn's factual number/ID/date claims against the injected context (e.g. [OUTGOING], [AGENTDATA], [EXTENSIONS], agent_data) plus what the caller has already said. A claim is "grounded" if its value appears in that combined grounding set; otherwise it is flagged as unsupported. This module is pure and side-effect-free: it does not touch the network, the model, or any session state. It is called post-call and its result is attached to the saved conversation log. English spoken/digit forms are fully supported; Chinese and Malay currency/duration digit forms are best-effort (digit sequences work in all languages). Design notes: - Only "material" numbers are checked: currency (ringgit/sen/RM), durations (days/months), counts with a unit (payments/instalments), percentages, dates (day + month), and long digit IDs (accounts/phones/ IC spelled as digit words or written as digits). Bare numbers with no unit (e.g. "one moment") are ignored to keep false positives low. - A wrong value that happens to also appear elsewhere in the context can be missed (PASS is "clean on mechanical checks", not "provably correct"). A FAIL is high-confidence: a concrete value with a material unit that is nowhere in the context or prior caller speech. Nzeroohnoughtonetwothreefourfivesixseveneightnine ten eleven twelve thirteen fourteen)fifteensixteen seventeeneighteennineteen(2<FPZ)twentythirtyfortyfiftysixtyseventyeightyninetydii@B)hundredthousandmillionfirstsecondthirdfourthfifthsixthseventheighthninthtentheleventhtwelfth thirteenth fourteenth fifteenth sixteenth seventeenth) eighteenth nineteenth twentieth thirtiethz thirty-firstz twenty-firstz twenty-secondz twenty-thirdz twenty-fourthz twenty-fifthz twenty-sixthztwenty-seventhz twenty-eighthz twenty-ninth>junmacmaymeijulyjuneogosapriljulaimarchaugustjanuarijanuaryoctoberoktoberdecemberdisemberfebruarifebruarynovember septemberringgitmoneysenrmmyrdaydurationdaysmonthmonthsyearyearsharibulantahunpercentperatuspaymentcountpayments) instalment instalments installment installmentszringgit.)u令吉u仙u天u日u个月u月u年u巴仙cPtjd|S)z:Lowercase word tokens, keeping hyphenated ordinals intact.z[a-z]+(?:-[a-z]+)?|\d+)refindalllower)texts J/home/m1-ai/DEPLOYMENTS/AI-PHONE/ZACK-UPDATE/fixing_leaked/faithfulness.py_tokrMs :/ > >>c |sdSd}d}d}|D]o}|tvr|t|z }d}|tvr|t|z }d}:|dkr |pddz}d}J|dvr||pdt|zz }d}d}h|d kro|sdS||zS) z9Convert a run of English number words to an int, or None.NrFTr=rr<)r>r?and)_EN_ONES_EN_TENS _EN_SCALES)tokenstotalcurrentsawts r_en_words_to_numberrRs t EG C    == x{ "G$CC (]] x{ "G$CC )^^|!s*G$CC ) ) ) gljm3 3EqW## %ZZ   t 7?rc.tjdd|S)z>Strip separators from a digit-ish string, return plain digits.z\D)rsub)ss r _norm_digitsrks 6%Q  rcFt|}|rt|ndS)z0Digit string -> int value (no decimal handling).N)rint) digit_strds r _to_valuerps$YA 3q666D rct}t}g}tjd|D]}|d}t |dd}|r7|t|||d|vrm|dd\}}t |} |dzdd}| r5|t| dzt|z|r%|t|dztjd |D]L}t |d} t| d kr|| Mt|} d} | t| kr| | tvs| | tvr| } g}| t| kr| | tvs*| | tvs| | tvs | | d krl|| | | dz } | t| kr9| | tvB| | tvQ| | tv`| | d klt|}|||| } n| dz } | t| k|||fS) zCollect every numeric value + long digit string from grounding text. Returns (values: set of int, money_cents: set of int, digit_runs: list).z\d[\d,]*(?:\.\d+)?r.r00Nr r<z\d+(?:[-\s]\d+)+r r)setrfinditergrouprsplitaddrappendlenrrrrr)rvalues money_cents digit_runsmrawdigitswholefracwdjoinedtoksijrunvals r_extract_grounding_numbersrxsUUF%%KJ[. 5 533ggajjciinnQ/00  & JJs6{{ # # #   f % % % #::))C++KE4e$$B4K!$D ;B# D 9::: 3F c 1222 [,d 3 3&&aggajj)) v;;!     f % % % ::D A c$ii-- 7h  $q'X"5"5ACc$ii--T!W%8%8DGxz"_extract_claims..s&11qG|111111rofr<)rtrwrxrvrrr)rrrrrrstrjoinr _add_writtenr_UNITSgetrrrr _EN_ORDINALranyrrr)rclaimsrtoks_allrrrrnumunitcatrrrrnxtrkrrvseenuniqueckeys r_extract_claimsrsF JJLLEz1599H A c(mm   A;( " "x '>>Ajja))MM#((4!a%=*A*AC,/#IS47#9#9:::AA7g%%aDMM#((4!9*=*=,/#IS4:#<#<===UUAAA FAc c$ii--h 55D F !1X;' ; d?? HHSMMM MM!    Mrcpt|dd}|sdSt|}d|vrV|dd\}}|dzdd}tt|dzt|z}n|dz}||||||ddS)Nrrrrr r<r)rrrr)rrcategoryrrrrrs rrrs #))C..+ , ,F  KKE czziiQ'' tt RaR L''((3.T:  MM##h@@AAAAArc|d}|dpd}|d}|d}|dkr^|||vrdSt|d krA|D]>}t|t|kr||s||vrdS?d S|d kr|||vrdS|||vrdSd S|||vrdSd S) z=True if the claim's value is supported by the grounding sets.rrrrrrNTr Fru)rrendswith) claimrrrrrrrrs r _is_groundedr-s  C YYx &BF ))G  C IIg  E d{{ ?sf}}4 v;;!  !  s88s6{{** V0D0D*RU 44u g~~  +!5!54 ?sf}}4u 3&==t 5r)search_extension) view_cart)dial_extensioncheckout)z "ok": falsez "ok":falsez api errorzapi call failed)sorryapologunablecannotzcan'tcouldnerror unfortunatelyissueproblemz try againtroubleznot ablez out of stockz not foundzno zonly u抱歉u无法u失败u问题u 找不到u没有u缺货maaftidakmasalahgagaltiadaci}|pgD]E} |di}|d}|r|||<6#t$rYBwxYw|S)z(Index tool definitions by function name.functionname)rAttributeError) tool_defsoutrfnrs r_tool_defs_maprcsx C _" z2&&B66&>>D D     H  Js2= A  A c  g}|pidipi}|dipi}|dgpg}|D]<}||vs||dvr|d|d|ddd =|piD]a\}} ||} | rEt| tr0d | vr,| | d vr"|d|d |d | d dd b|S)z?Required-field and enum validation. Returns list of flag dicts. parameters propertiesrequired)Nr tool_schemaz: missing required argument "" violation) claim_typespokenverdictenumrz="z" not in allowed values)rritems isinstancedict) rargsfn_defflagsparamspropsr fieldrrspecs r_check_tool_schemarqsj El   b 1 1 7RF JJ|R ( ( .BEzz*b))/RH33   : = = LL '+$R$R%$R$R$R%022 3 3 3ZR&&((77Syy~~  7JtT** 7v~~$v,&& M+/(U(U#(U(U(U(U(U)466777 Lrc Dg}|piD]\}}tjdt|D]^}t |dkrd|t |dd} t | |||s"|d|d|d||d d _|S) uMaterial numeric argument values (3+ digit runs) must be grounded in context, prior caller speech, or prior tool results. Small numbers (quantities etc.) are intent-arithmetic — deliberately not checked.z\d+r rN)rrrrtool_argr= unsupported)rrrr)rrrrrrrr) rrrrrrrrrrs r_check_tool_args_groundedr!s EZR&&(( 9 9S:fc#hh// 9 9C3xx!||&*c!#hh77Ev{JGG 9 J+/(=(=#(=(=(=(=(+)688999 9 Lrc|rt|tsdS|tfdtDS)NFc3 K|]}|vV dSrrrlows rrz#_result_is_error..s'00AqCx000000r)rrrr_ERROR_MARKERS)resultr&s @r_result_is_errorr)sQ FC00u ,,..C 0000000 0 00rc t|pd\}}}t|t|t|t|t |pgd}d}tdgd} fd} t |pgD]\} } |t |krj||dd| krJ| ||| |dz }|t |kr ||dd| kJ| d pd} | d pd}| d kr3t|\}}}|z|z || d kr`|tfd tDs* | ddddddd| dz } t|D]b}t|sN | |d|d|d|dddc|t |kr9| ||t |pg|dz }|t |k9| t |t rdnddS)uAssess an entire conversation. Args: messages: list of {'speaker': 'ai'|'user'|'system', 'text': str} context_text: the resolved injected context (OUTGOING/AGENTDATA/etc.) tool_events: optional list of {'msg_index': int, 'name': str, 'arguments': dict, 'result': str|None} in call order. msg_index is the conversation_log position at the time the tool ran (used to interleave with speech turns). tool_defs: optional list of tool definitions ({'function': {...}} format) used for required-field/enum validation. Returns a result dict (no timestamp — caller stamps it): {checked_turns, tool_calls_checked, flagged_count, status, flags} rc.|ddS)N msg_indexr)r)es rz%assess_conversation..sQUU;5J5Jr)rrNc |dpd}|dpi}t|ts- ddl}||}n#t $ri}YnwxYw|d}t |dD]&}|vr ||d|d |d d '| vr9t|| |D]!}||d <||d <|"t|| D]!}||d <||d <|" |d d| D}t|tr|d|zz }t|\} } } | |  | t#|r|||ddSdS)zESequence + schema + arg checks, then fold args/result into grounding.rr argumentsrNr(r$ tool_sequencez called without prior r turntoolrrrr3r4rc34K|]}t|VdSr)r)rvs rrz=assess_conversation.._ingest_event..s(==!s1vv======r)eventat_idxr)rrrjsonloads ExceptionSEQUENCE_RULESrrr!rrrrrupdateextendr))evr8rr_jsonr(prereqf ground_textgvgcgrdefs_maprrr pending_error seen_toolsrs r _ingest_eventz*assess_conversation.._ingest_eventsjvvf~~#vvk""(b$%%  $$$${{4((    !!$((r22 7 7FZ'' fd,;+/(O(Ov(O(O)466777 8  'dHTNCC  "&  &  Q*4v{JWW  AAfIAfI LLOOOOthh==t{{}}===== fc " " ( 3< 'K/ << B b2" F # # J&(FDIIMMM J JsA A/.A/r,rspeakerruseraic3 K|]}|vV dSrr$r%s rrz&assess_conversation..s'66AqCx666666rrtool_error_ignoredz4 returned an error; the reply did not acknowledge itr r2rrrrr )r3rrrrrFAILPASS) checked_turnstool_calls_checked flagged_countstatusr)rrlistrsorted enumeraterrrr>r _ACK_MARKERSrrr)messages context_text tool_eventsr base_values base_cents base_runseventsev_ptrcheckedrJidxmsgrKruvucurrrGrrr&rrHrIrs @@@@@@@@rassess_conversationrhs *DLDVTV)W)W&KY  Fj//KiJi((H K%2+J+J K K KF FJM EG(J(J(J(J(J(J(J(J(J(J(JThn"--&&Ss6{{""vf~'9'9+q'I'IS'P'P M&.# . . . aKFs6{{""vf~'9'9+q'I'IS'P'P779%%+2244wwv$" f  3D99JBB bLF 2 K   b ! ! !  d??   $**,,C666666666 7 c=3H,@,9&,A*T*T*T)4 66777 !M1 $T**  Ev{JGG  "'"8#El"YYw//#ii11,   3v;;   fVnc(.b&9&9:::!  3v;;   !!&kkU!-&&v   r)NN)__doc__rrrrrrr _ZH_UNITSrrrrrrrrr<r'rYrrr!r)rhr$rrrks24   A Q  ! %*A /4a 9@!  A q  $+Q 07   r $0 6B2  !" '4R HJ2BRR2r         w w (,g 7