shϓ %SSKrSSKrSSKJr SSKJrJr SSKrSSKJ s J s J r SSK J r SSKJ s Jr SSKJr SSKJrJrJr SSKJrJr SSKJr SSKJr S S KJrJ r J!r! /r"\#\\$S 'S \RJS \&S\RJS\RJ4Sjr'\RP\RR\RT\RV0r,"SS\ RZ5r."SS\ RZ5r/S\.S\/S\RJ4Sjr0"SS\"SSS/55r1\.Re\Rf\RTS9r4\/Re\Rj\RVS9r6\1"\4\6S9r7S\84Sjr9S\84Sjr:S\84S jr;S!\S\84S"jr\?\ RZ4S\@\\\\/44S(jrAS)\S'\>\?\ RZ4S\\4S*jrBS!\S'\>\?\ RZ4S\\.4S+jrCS!\S'\>\?\ RZ4S\\RJ4S,jrDS!\S'\>\?\ RZ4SS4S-jrES!\S'\>\?\ RZ4S.\RJS/\\RJSS4 S0jrFS)\S&\S'\>\?\ RZ4S.\RJS/\\RJSS4 S1jrGS)\S'\>\?\ RZ4SS4S2jrHS&\S!\S3\4S4jrIS&\S'\>\?\ RZ4S\>\?\/44S5jrJS&\S'\>\?\ RZ4S6\>\?\/4SS4S7jrKS&\4S8jrLS9\ RZS:\ RZS;\RJS\>\?\M44S<jrNS=\>\?\M4S>\&S\4S?jrOg)@N) namedtuple)AnyOptional)_get_observed_graph_module_attr) _with_args ObserverBasePerChannelMinMaxObserver) _parent_namecheck_min_max_valid) GraphModule)Node)get_new_attr_name_with_prefixmaybe_get_next_modulenode_arg_is_weightCUSTOM_MODULE_SUPP_LISTscaleaxisinputreturncjS/UR-nURU5X1'URU5$)zMReshapes the scale so that we can multiply it to the input by the given axis.r)ndimsizeview)rrr new_shapes v/Users/tiagomarins/Projetos/claudeai/copy_bank/venv/lib/python3.13/site-packages/torch/ao/quantization/fx/_equalize.py reshape_scalers1ejj Ijj&IO ::i  c^\rSrSrSr\R \RSSS4S U4SjjjrSr Sr Sr Sr \ "\5rS rU=r$) _InputEqualizationObserver,aObserver for tracking the running min/max values of input columns, and computing the quantization parameters for the overall min/max input values. Args: dtype: Quantized data type qscheme: Quantization scheme quant_min: Minimum quantization value. If unspecified, it will follow the 8-bit setup. quant_max: Maximum quantization value. If unspecified, it will follow the 8-bit setup. The running minimum/maximum :math:`x_\text{min/max}` are computed in the same way as :class:`~torch.ao.quantization.observer.PerChannelMinMaxObserver`, with the difference that the running min/max values are stored per column. This observer is intended to be used along with a WeightEqualizationObserver to calculate the equalization scale. Nc >[TU]5 U[R[R1;a [ S5eXlX l[Un[SUUUUUS9Ul [R"S5Ul /Ul g)Nz Input qscheme must be per-tensorrch_axisdtypeqscheme quant_min quant_maxfactory_kwargs)super__init__torchper_tensor_affineper_tensor_symmetric TypeErrorr%r&(qsheme_mapping_per_tensor_to_per_channelr input_obstensorequalization_scaleequalization_shapeselfr%r&r'r(r)per_channel_qscheme __class__s rr+#_InputEqualizationObserver.__init__?s  522E4N4NO O>? ?  FwO1')  #(,,q/-/rcURS:aURS::d [S5eS/UR-UlURS5URS'UR U5$)N>InputEqualizationObserver only supports Linear and Conv layersr)r ValueErrorr4rr1)r6x_origs rforward"_InputEqualizationObserver.forward\se q V[[A%5P  $%# "3%+[[^"~~f%%rcZURRURR4$N)r1min_valmax_valr6s rget_input_minmax+_InputEqualizationObserver.get_input_minmaxhs!&&(>(>??rcUR5S:XaU[R"S5:Xag[R"XR5Ulg)Nr)nelementr,r2reshaper4r3r6r3s rset_equalization_scale1_InputEqualizationObserver.set_equalization_scaleksC  & & (A -2D UV2W "'--  7 7# rcURR5S:Xa;UR[R"S5:Xa[R "S5 gUR 5up[URSU5n[R"[R"X55n[R"[R"X#55nXE4$)z!Returns the scaled min/max inputsrz}Must call calculate_equalization_scale before calling calculate_scaled_minmax. Will not scale the next quantization observer.NNr) r3rJr,r2warningswarnrGrminmulmax)r6 min_inputs max_inputsequalization_scale_reshapedmin_input_scaledmax_input_scaleds rcalculate_scaled_minmax2_InputEqualizationObserver.calculate_scaled_minmaxts  # # , , .! 3''5<<?: MMC  $(#8#8#: &3  # #Q ' #!99UYYz%WX 99UYYz%WX11r)r%r3r4r1r&rN)__name__ __module__ __qualname____firstlineno____doc__r,quint8r-r+r@rGrMr[ classmethodr with_args__static_attributes__ __classcell__r8s@rr r ,sX(ll'' 0 00: &@ 20J'Irr c^\rSrSrSr\R \RSSS4S U4SjjjrSr Sr Sr \ "\ 5rSrU=r$) _WeightEqualizationObserveraCObserver for tracking the running min/max values of weight columns and rows, and computing the quantization parameters for the weight rows. Args: dtype: Quantized data type qscheme: Quantization scheme quant_min: Minimum quantization value. If unspecified, it will follow the 8-bit setup. quant_max: Maximum quantization value. If unspecified, it will follow the 8-bit setup. This observer is made up of 1 PerChannelMinMaxObserver `weight_col_obs` used to record the running minimum and maximum of columns of incoming weight tensors. This observer is intended to be used along with an InputEqualizationObserver to calculate the equalization scale. The running minimum/maximum :math:`w_\text{min/max}` are computed in the same way as :class:`~torch.ao.quantization.observer.PerChannelMinMaxObserver`. Nc >[TU]5 XlX lSUlUnU[ R [ R1;a [Un[SUUUUUS9Ul [ R"S5Ul g)Nrr#) r*r+r%r&r$r,r-r.r0r weight_col_obsr2r3r5s rr+$_WeightEqualizationObserver.__init__s|    % u..0J0JK K"J7"S 6')  #(,,q/rczURS:aURS::d [S5eURU5$)Nr;r<r=)rr>rm)r6w_origs rr@#_WeightEqualizationObserver.forwards< q V[[A%5P ""6**rcZURRURR4$rC)rmrDrErFs rget_weight_col_minmax1_WeightEqualizationObserver.get_weight_col_minmaxs%##++T-@-@-H-HIIrcXlgrC)r3rLs rrM2_WeightEqualizationObserver.set_equalization_scales"4r)r$r%r3r&rmr])r^r_r`rarbr,qint8r-r+r@rsrMrdrrerfrgrhs@rrjrjsS,kk'' 2 228+J5J'Irrjr1 weight_obscUR5up#UR5upE[X#5(a[XE5(d,[R"S5 [ R "S5$URUR:Xd)[SSURSURS3-5e[ R"XT- X2- - 5nSXfS:H'[ R"USSSS9nU$) zCalculates the equalization scale and sets the equalization_scale value in the observers. Args: input_obs: Observer that tracks the ranges for the input columns weight_obs: Observer that tracks the ranges for the weight columns ztMust run observer before calling calculate_equalization_scale. Returning default equalization scale torch.tensor(1).rz6Input and Weight must have the same column dimension. zFound z and z shapes instead.g)nanposinfneginf) rGrsr rQrRr,r2shaper>sqrt nan_to_num)r1rxrVrW min_weights max_weightsr3s rcalculate_equalization_scalers )99;Z!+!A!A!C[ J33  9 9  F ||A    1 1 1 Dz''(k.?.?-@@PQ R    "z'>?56S01))*[U[R5(d[U[R5(a [S5e[TU]XU5nU$)NzEqualizationQConfig received observer instance, please pass observer class instead. Use MyObserver.with_args(x=1) to override arguments to constructor if needed) isinstancennModuler>r*__new__)clsinput_activationweightr6r8s rrEqualizationQConfig.__new__sO & 2 2j6S6Sa wsf= r) r^r_r`rarb __slots__r,rIdentityrrfrgrhs@rrrs2$I&+hh&7&7@Q@Qrrrr)r%r&)rrc[U5[R[R[R[R 4;$)z/Checks if the fused node supports equalization.)typenni LinearReLU ConvReLU1d ConvReLU2d ConvReLU3dmodules r"fused_module_supports_equalizationr$s4 <      rc[U5[R[R[R[R 4;$)z2Checks if the torch.nn node supports equalization.)rrLinearConv1dConv2dConv3drs rnn_module_supports_equalizationr.s*  inp_obs1 -> eq_obs1 -> linear1 -> out_obs1 -> eq_obs2 -> linear2 -> out_obs2 If the node being passed in is the linear1 node, then we want to return eq_obs2, the following equalization observer for linear2. However, if there are no connecting layers: x -> inp_obs1 -> eq_obs1 -> linear1 -> out_obs1 -> add Then we want to return None. In the case of an unfused linear-relu layer with a connecting linear layer: linear1 -> relu -> out_obs1 -> eq_obs2 -> linear2 -> out_obs2 Since it is unfused, we want to skip over the relu layer and return eq_obs2, the following equalization observer for linear2. N)target_functional_type) rrrReLUrrelurr rr)rrmaybe_relu_nodemaybe_obs_nodemaybe_eq_obs_node maybe_eq_obss rmaybe_get_next_input_eq_obsrs( &d 4 44 4,D277CO/ !&&   " d\: "?\ J -!; 3012L l$> ? ?? ? rc[X5nU(aOURR5S:Xa%UR[R"S5:XagUR$g)aAIf the next next node is an InputEqualizationObserver then we want to return its equalization scale, else we return 1 This is used in the case where there are two connecting linear layers: linear1 -> LinearOutObs -> InputEqObs -> linear2 In this case, the node given is linear1 and we want to locate the InputEqObs. rN)rr3rJr,r2)rrnext_inp_eq_obss r!maybe_get_next_equalization_scalersP2$@O  . . 7 7 9Q >22ell1oE111 rcTU[UR5n[U[5(deURSn[U[ 5(deU[UR5n[U[ 5(dgUR5upVUcUcgXTlXdl g)zScales the following input quantization observer's min/max values by updating the values with the scaled min/max values calculated by the input equalization observer rN) rrrr rr rr[rDrE)rr input_eq_obsinput_quant_obs_nodeinput_quant_obsrYrZs rscale_input_observerrs 3t{{+,L l$> ? ?? ?99Q< *D 1 11 1c"6"="=>?O o| 4 4)5)M)M)O&$4$<..rr3next_equalization_scalecfUcg[U[UR55(aU[UR5SnOU[UR5n[U5(d[ U5(deUR n[ U[R5(de[USU5n[R"U[R"U55nUc[R"U5Ulg[USU5n[R"Xx5n[R"U5UlURn U cg[ U [R5(de[USU 5n[R"X5n [R"U 5Ulg)aScale the weights for input-weight equalization by multiplying the weight by 1/equalization_scale and next_equalization_scale Args: node: Current node whose weights we want to scale equalization_scale: Current node's calculated equalization scale next_equalization_scale: Next node's calculated equalization scale if the following node needs to be equalized, 1 otherwise Nrr)rrrrrrrr,TensorrrT reciprocalr Parameterbias) rrr3r op_modulerrX scaled_weight next_equalization_scale_reshapedr scaled_biass rscale_weight_noderso!)'#dkk2B*CDDC ,-a0 C ,- *   ,Y 7 78 8   F fell + ++ +#00BAv"NIIfe&6&67R&STM&<< 6 (55LaQW'X$IImNM||M2I >>D | dELL ) )) )(55LaQU'V$))DCK\\+.INrcjUcg[X5nUcgURSnUcg[U[5(a+[U[ UR 5[ 5(deURSnUcg[U[5(aURS:Xde[UR 5up[X(U 5n [USU 5n [R"U [R"U 55n Uc[X(X5 g[USU 5n [R"X5n [X(X5 [R"UR![ UR 55U 5(deSnURH@n[U[5(dMURS:XdM,SUR";dM>Un O Ucg[UR 5unn[UUU5n[USU5n [R"UU 5n[UUUU5 g)z-Scales the weight value for functional layersNrget_attrrr)rrrr rrrrr getattrrr,rTrsetattrallclose get_bufferr)rrrr3rweight_eq_obs_nodeweight_quant_obs_noderweight_parent_name weight_namerrXrr bias_noderbias_parent_name bias_namerrs rscale_weight_functionalr(s !6gG!/33A6$ +T 2 2z)0012L88  (,,Q/K k4 ( ([^^z-II I&2;3E3E&F# W0+ >F #00BAv"NIIfe&6&67R&STM&+[H(5M($IImNM G 'D >>%**3{/A/A+BC] S SS SI  dD ! !dgg&;$))@SI   ".y/?/?"@i 7+,i 8D(55LaQU'V$))D"BCK G$ %y+>rc [X5nUcgURSnUcg[U[5(deU[ UR 5n[U[ UR 5[ 5(deUR5 g)zlGiven the operation node, we want find the corresponding quantization observer and reset its min/max values Nr)rrrr rrrreset_min_max_vals)rrrrweight_quant_obss rclear_weight_quant_obs_nodervs6gG!.33A6$ +T 2 22 2s#8#?#?@A gc"7">">?@, O OO O'')r prev_nodec[URR55nUHnURX5 M URR U5 g)zaRemoves the given node from the model by replacing all of its users with the given previous node N)listrrreplace_input_withgraph erase_node)rrr orig_users user_nodes r remove_nodersE djjoo'(J $$T5  KK4 rc0nURRGHMnURS:XdM[XR[ 5(dM9XRn[U[ 5(de[ X0U5upVUbUcMtURS:Xa[U[UR55(a@U[UR5Sn[U5(deU"UR5 O(U"U[UR5R5 [XF5nURU5 URU5 XbUR'GMP U$)agUpdate all of the observer's equalization scale. For each InputEqualizationObserver, we will find the location of the next WeightEqualizationObserver, create it, and calculate the equalization scale based on the two observers. We will then return a dictionary mapping operation node names to the corresponding WeightEqualizationObservers for that operation. rr)rnodesrrrr rrrrrrrMr) rrweight_eq_obs_dictrrrrrr3s rupdate_obs_for_equalizationrs> !! 77m # KK "<) ) #;;/Ll,FGG GG%B4PW%X "G-"7zz]*6gc'..>Q6RSS$S%89!"E"EF">"   / /0B C  0 01C D/ InpQuantObs -> InpEqObs -> linear -> OutQuantObs After this function: scaled weight values | equalization scale WeightQuantObs | | x -> mul -> InpQuantObs (scaled min/max) -> linear -> OutQuantObs After convert: equalization scale scaled weight values | | x -> mul -> quantize_per_tensor -> quantized::linear Note that although the equalization observer appeared after the quantization observer after prepare_fx, the mul node appears before the quantization node after convert_fx. This is because placing the equalization observer after the quantization observer in prepare_fx would allow us to keep the invariant that the graph before the current node inserts its observers is not modified. Having the equalization observer before the quantization observer would also cause some inconsistences between the ordering of the quantization and equalization observers. For example, a single linear layer would look like: x -> InpEqObs1 -> InpQuantObs1 -> linear1 -> OutQuantObs1 But between two connected linear layers, it would look like: linear1 -> OutQuantObs1 -> InpEqObs2 -> linear2 -> OutQuantObs2 rrr_equalization_scalerNrrz=Expected operation node to be 'call_module' or 'call_functionzInstead got node z as 'z'.)rrrrrr rrrrrinserting_beforerrr3 create_nodeinserting_afterr,rTrrrjrJr2rrrrrrr>)rrrrinp_quant_obs_noderget_new_eq_scale_namer eq_scale_nodeinputsmul_noderr3maybe_next_equalization_scalers rconvert_eq_obsr sf !! 77m # KK "<) ) "&1 *//2I+9>>Y^^+E);< ! /--.@A(ENN%::)%-W5W[[%9%L%LM % 7 7 D I B,,];#3 ;;22?EIIvV<  1 1)X F %7 8  # #DIIt 4 @.22499=Mm-HII II!.!A!A #++-2&%,,q/9%)",M- ) ww-'!#5O+'&1 &B$%P"%-!C 2 9 9:;=X ,D:/33A6 EyA S)$))E$''"EFu":BA<;sAL/L L  L$ c[URSS95n[X5n[XU5 [ XR 5$)zbReference function which applies changes needed for equalization, but does not quantize the nodes F)remove_duplicate)dict named_modulesrr r r)rrrs r_convert_equalization_refrVsC5&&&>?G5UD5#56 ukk **rmodel_amodel_bxc SSKJs Js Jn SSKJn U"5nUSR [R5 URSUSUURUS9upgU"U5 U"U5 URXgURS5nURUSS[RRRRRS5 0n UH*n XS SSS n XS SSSSn XU 'M, U $) aRuns the Numeric Suite on model_a and model_b and returns a dictionary containing the SQNR between layers in model_a and model_b. Note: In order to support equalized models, this function has a hacky fix in which we do not match any torch.mul operators. This is because equalized models contain extra mul operators to scale the input by the equalization scale, but this edge case has not been resolved yet within the numeric suite code. Args: model_a: A float model model_b: A quantized model x: Inputs to use during calibration rN)get_unmatchable_types_mapfuns_unmatchablefp32int8)unmatchable_types_mapsqnr node_outputfqn)torch.ao.ns._numeric_suite_fxaons_numeric_suite_fxtorch.ao.ns.fx.mappingsraddr,rT add_loggers OutputLoggerextract_logger_info%extend_logger_results_with_comparisonfxutils compute_sqnr) rrrr!rr model_a_ns model_b_nsactivation_comparison_dictlayer_sqnr_dictkeylayerrs rget_layer_sqnr_dictr2is /.A57,-11%))<^^ 3 ,JqMqM!#!7!7",,"  )) O)*/ >vFqI%P).}=fEaHPQRS!%* rr/num_layers_to_equalizec[UR5[R"S5S9nUSUnUVs/sH oDS[4PM nnSU0nU$s snf)aGiven the layer to SQNR dictionary, find the layers with the highest quantization errors, and return an equalization_qconfig_dict specifying to only equalize those top layers. Args: layer_sqnr_dict: Dictionary mapping layer names to SQNR values (found when comparing an equalized model against a float model) num_layers_to_equalize: Number of layers with the highest quantization errors to equalize r)r0Nr module_name)sorteditemsoperator itemgetterdefault_equalization_qconfig)r/r3layer_sqnr_sortedlayers_to_equalizeitemmodule_to_qconfig_listequalization_qconfig_dicts rget_equalization_qconfig_dictr@sv 446HrVs" ##Q J &(c'!!S!!%,,! U55  ; ;,( `(`(F?("))?(D$)$7R$ \\$N$'98&DEB9BB ,, : :C ;DD ++u:: E  309U  $HtH 343 T t ryyT))$/):>sBII~:N) 8D>8$?@ @A)X  bii0 d^$/ /c299n-/ ()/d c299n- ell*/t/d3 >.B/t/,8/ 8/ #ryy. !8/ 8/&ell3 8/  8/vK? K? K?#ryy. !K? K? &ell3 K?  K?\**S"))^8L*QU*$ !{ !$ !4 !* *!%c299n!5* #* *+*ZP P #ryy. !PS"==>P Pf +[ +&4 YY4!#4/4||4 #u*4n%#u*%%?B%%r