shSSKrSSKJr SSKrSSKJr SSKJrJr SSK J r J r \ "SS5r \R\R\R\R \R"/r\R&\R(/r\Vs0sH9o\R-U5R.\R-U5R04_M; snr\R5\Vs0sHEo\"\R9U5R.5\"\R9U5R054_MG sn5 Sr\ R=S 5 \ "\ S S 5S \R>S \ S\S\S\S\RBS\R>4Sj5r"\ "\ S S5S \R>S \ S\S\S\S\RBS\R>4Sj5r#\ R=S5 \ "\ SS 5S \R>S \R>S\R>S\S\S\RBS\R>4Sj5r$\ "\ SS5S \R>S \R>S\R>S\S\S\RBS\R>4Sj5r%\ R=S5 \ "\ SS 5S \R>S \R>S\R>S\R>S\R>S\RBS\R>4Sj5r&\ "\ SS5S \R>S \R>S\R>S\R>S\R>S\RBS\R>4Sj5r'\ R=S5 \ "\ SS 5SS .S \R>S \ S\S\S\S\RBS!\\RBS\R>4S"jj5r(\ "\ SS5SS .S \R>S \R>S\R>S\S\S\RBS!\\RBS\R>4S#jj5r)\ R=S$5 \ "\ S%S 5SS .S \R>S \R>S\R>S\S\S\RBS!\\RBS\R>4S&jj5r*\ "\ S%S5SS .S \R>S \R>S\R>S\S\S\RBS!\\RBS\R>4S'jj5r+\ R=S(5 \ "\ S)S 5SS .S \R>S \R>S\R>S\R>S\R>S\RBS!\\RBS\R>4S*jj5r,\ "\ S)S5SS .S!\\RBS\R>4S+jj5r-\ R=S,5 \ "\ S-S 5S \R>S.\S/\S0\ S\RBS\.\R>\R>44 S1j5r/\ R=S25 \ "\ S3S 5S \R>S.\S/\S0\ S\RBS\.\R>\R>44 S4j5r0\ "\ S-S5S \R>S\S\S0\ S\RBS\.\R>\R>44 S5j5r1\ "\ S3S5S \R>S\S\S0\ S\RBS\.\R>\R>44 S6j5r2S7r3\ R=S85 \ "\ S9S 5S \R>S:\R>S;\R>S<\S\S\S\RBS\R>4S=j5r4\ "\ S9S5S \R>S:\R>S;\R>S<\S\S\S\RBS\R>4S>j5r5\ R=S?5 \ "\ S@S 5SS .S \R>S:\R>S;\\R>S<\S\S\S\RBS!\\RBS\R>4SAjj5r6\ "\ S@S5SS .S \R>S:\R>S;\\R>S<\S\S\S\RBS!\\RBS\R>4SBjj5r7\ R=SC5 \ "\ SDS 5S \R>S\RBS\.\R>\R>44SEj5r8\ "\ SDS5S \R>S\RBS\.\R>\R>44SFj5r9\ R=SG5 \ "\ SHSI5S \R>S\RBS\.\R>\R>44SJj5r:\ R=SK5 \ "\ SLS 5S \R>S\RBS\.\R>\R>44SMj5r;\ "\ SLS5S \R>S\RBS\.\R>\R>44SNj5rS:\R>S;\R>S\S\S\RB4 SRj5r>\ "\ SQS5S \R>S:\R>S;\R>S\S\S\RB4 SSj5r?\ R=ST5 \ "\ SUS 5\R4S \R>S:\R>S;\R>S\S\S\RBSV\RB4SWjj5rA\ "\ SUS5\R4S \R>S:\R>S;\R>S\S\S\RBSV\RB4SXjj5rB\ R=SY5 \ "\ SZS 5SnS \R>S:\R>S;\R>S\S\S\RB4 S\jj5rC\ "\ SZS5SnS \R>S:\R>S;\R>S\S\S\RB4 S]jj5rD\ R=S^5 \ "\ S_S 5S[\R4S`\R>S:\R>S;\\R>S\S\S\RBSa\SV\RB4Sbjj5rE\ R=Sc5 "SdSe\RR5rH\ "\ SfSg5S \R>S:\R>S;\R>S<\S\S\S\R>4Shj5rI\ "\ SfS5S \R>S:\R>S;\R>S<\S\S\S\R>4Sij5rJ\ R=Sj5 \ "\ SkS 5S \R>S\RBS\R>4Slj5rK\ "\ SkS5S \R>S\RBS\R>4Smj5rLgs snfs snf)oN)Optional)_unsqueeze_multiple)determine_qparamsvalidate_qmin_qmax)implLibraryquantized_decomposedDEFcU[;a[SU35e[Uup4X:d SUSU35eX::d SUSU35eg)NzUnsupported dtype: z9quant_min out of bound for dtype, quant_min_lower_bound: z quant_min: z9quant_max out of bound for dtype, quant_max_upper_bound: z quant_max: )_DTYPE_TO_QVALUE_BOUNDS ValueError) quant_min quant_maxdtypequant_min_lower_boundquant_max_upper_bounds x/Users/tiagomarins/Projetos/claudeai/copy_bank/venv/lib/python3.13/site-packages/torch/ao/quantization/fx/_decomposed.py_quant_min_max_bounds_checkrs ++.ug6773J53Q0  - ""7!8 YK Q -  - ""7!8 YK Q -zxquantize_per_tensor(Tensor input, float scale, int zero_point, int quant_min, int quant_max, ScalarType dtype) -> Tensorquantize_per_tensorCompositeExplicitAutogradinputscale zero_pointrrrreturncUR[R[R4;aUR [R 5nUR[R :XdSUR35e[ X4U5 SU- n[R"[R"X-5U-X45R U5$)aAffine quantization for the Tensor using the same quantization parameters to map from floating point to quantized values Args: input (torch.Tensor): original float32 or bfloat16 Tensor scale (float): quantization parameter for affine quantization zero_point (int): quantization parameter for affine quantization quant_min (int): minimum quantized value for output Tensor quant_max (int): maximum quantized value for output Tensor dtype (torch.dtype): requested dtype (e.g. torch.uint8) for output Tensor Returns: Tensor with requested dtype (e.g. torch.uint8), note the quantization parameters are not stored in the Tensor, we are storing them in function arguments instead Tensorzquantize_per_tensor.tensorcUR5S:XdSUR535eUR5S:XdSUR535e[XR5UR5X4U5$zAffine quantization for the Tensor using the same quantization parameters to map from floating point to quantized values Same as `quantize_per_tensor` but scale and zero_point are Scalar Tensor instead of scalar values >Expecting zero_point tensor to be one element, but received : 9Expecting scale tensor to be one element, but received : numelritemr+s rquantize_per_tensor_tensorr5ms" a] G HXHXHZG[\]  S B5;;=/RS  zz|Z__. e rcUR[R[R4;aUR [R 5nUR 5S:XdSUR 535eUR 5S:XdSUR 535eUR[R :XdSUR35e[R"XS9$)Nr/r0r1rr))rrr r!r"r#r3r*r+s rquantize_per_tensor_tensor_metar7s {{u}}enn55'a] G HXHXHZG[\]  S B5;;=/RS  u}}$T Eekk]ST$   E //rzquantize_per_tensor.tensor2(Tensor input, Tensor scale, Tensor zero_point, Tensor quant_min, Tensor quant_max, ScalarType dtype) -> Tensorzquantize_per_tensor.tensor2cBUR5S:XdSUR535eUR5S:XdSUR535e[UUR5UR5UR5UR5U5$r.r2r+s rquantize_per_tensor_tensor2r9s" a] G HXHXHZG[\]  S B5;;=/RS     rc[XX#XE5$N)r7r+s r quantize_per_tensor_tensor2_metar<s + jY rzdequantize_per_tensor(Tensor input, float scale, int zero_point, int quant_min, int quant_max, ScalarType dtype, *, ScalarType? out_dtype=None) -> Tensordequantize_per_tensor out_dtyper?cURU:XdSUSUR35eUc[RnU[;aUR U5U- U-$[ SU35e)aAffine dequantization for the Tensor using the same quantization parameters to map from quantized values to floating point values Args: input (torch.Tensor): Tensor with dtype matching `dtype` argument, e.g. (`torch.uint8`), it is a per tensor quantized Tensor if combined with quantization parameters in the argument of this function (scale/zero_point) scale (float): quantization parameter for affine quantization zero_point (int): quantization parameter for affine quantization quant_min (int): minimum quantized value for input Tensor (not used in computation, reserved for pattern matching) quant_max (int): maximum quantized value for input Tensor (not used in computation, reserved for pattern matching) dtype (torch.dtype): dtype for input Tensor (not used in computation, reserved for pattern matching) out_dtype (torch.dtype?): optional dtype for output Tensor Returns: dequantized float32 Tensor Expecting input to have dtype: z , but got ,Unsupported dtype in dequantize_per_tensor: )rrr#r r"r rrrrrrr?s rr=r=szL  uH (z%++GHMM  ''#j0E99GwOPPrcPUc[Rn[R"XS9$Nr))rr#r*rCs rdequantize_per_tensor_metarFs$MM   E 33rzdequantize_per_tensor.tensor(Tensor input, Tensor scale, Tensor zero_point, int quant_min, int quant_max, ScalarType dtype, *, ScalarType? out_dtype=None) -> Tensorzdequantize_per_tensor.tensorc UR5S:XdSUR535eUR5S:XdSUR535e[UUR5UR5UUUUS9$zAffine dequantization for the Tensor using the same quantization parameters to map from quantized values to floating point values Same as `dequantize_per_tensor` but scale and zero_point are Scalar Tensor instead of scalar values r/r0r1r>r3r=r4rCs rdequantize_per_tensor_tensorrJ's* a] G HXHXHZG[\]  S B5;;=/RS     rcdUc[RnUR5S:XdSUR535eUR5S:XdSUR535eURU:Xd SU35eU[;a[R "XS9$[ SU35e)Nr/r0r1rAr)rB)rr#r3rr r*r rCs r!dequantize_per_tensor_tensor_metarLLsMM a] G HXHXHZG[\]  S B5;;=/RS ;;% J#B5'!JJ  ''77GwOPPrzdequantize_per_tensor.tensor2(Tensor input, Tensor scale, Tensor zero_point, Tensor quant_min, Tensor quant_max, ScalarType dtype, *, ScalarType? out_dtype=None) -> Tensorzdequantize_per_tensor.tensor2c @UR5S:XdSUR535eUR5S:XdSUR535e[UUR5UR5UR5UR5UUS9$rHrIrCs rdequantize_per_tensor_tensor2rNms* a] G HXHXHZG[\]  S B5;;=/RS     rc [XX#XEUS9$)Nr>)rLrCs r"dequantize_per_tensor_tensor2_metarPs - jY rzrchoose_qparams.tensor(Tensor input, int quant_min, int quant_max, float eps, ScalarType dtype) -> (Tensor, Tensor)zchoose_qparams.tensorqminqmaxepsc UR[R[R[R4;dSUR35eU[ ;dS[ R 5SU35e[X5 [R"U5upV[UUUUU[R"U/5SS9$)3Given an input Tensor, derive the per tensor affine quantization parameter (scale and zero_point) for target quantized Tensor from the Tensor Args: input (torch.Tensor): floating point input Tensor quant_min (int): minimum quantized value for target quantized Tensor quant_max (int): maximum quantized value for target quantized Tensor dtype (torch.dtype): dtype for target quantized Tensor Returns: scale (float): quantization parameter for the target quantized Tensor zero_point (int): quantization parameter for the target quantized Tensor CExpecting input to have dtype torch.float32/16/b16, but got dtype: $Expecting target dtype to be one of , but got: F)has_customized_qrange) rrr#r r!r keysraminmaxrTensorrrQrRrSrmin_valmax_vals rchoose_qparams_tensorr`s" ;;    [ MU[[MZ [  ((a -.E.J.J.L-M[Y^X_`a(t"}}U+G     cU# rz|choose_qparams_symmetric.tensor(Tensor input, int quant_min, int quant_max, float eps, ScalarType dtype) -> (Tensor, Tensor)zchoose_qparams_symmetric.tensorc UR[R[R[R4;dSUR35eU[ ;dS[ R 5SU35e[X5 [R"U5upV[UUUUU[R"U/5S[RS9$)rUrVrWrXF)rYqscheme) rrr#r r!r rZrr[rr\per_tensor_symmetricr]s rchoose_qparams_symmetric_tensorrds* ;;    [ MU[[MZ [  ((a -.E.J.J.L-M[Y^X_`a(t"}}U+G     cU#**  rcUR[R[R[R4;dSUR35eX:d SUSU35e[R "S[R URS9[R "S[RURS94$)NrVzKExpecting quant_min to be smaller than quant_max but received min: z max: r/rdevice) rrr#r r!emptydoublergint64rrrrSrs rchoose_qparams_tensor_metarls ;;    [ MU[[MZ [  &  6)&& ;;q U\\ BEKK U\\E rc[R"S[RURS9[R"S[RURS94$)Nr/rf)rrhrirgrjrks r$choose_qparams_symmetric_tensor_metarnsA ;;q U\\ BEKK U\\E rc[[UR555nSX!'XS'UR[ U55nX24$)Nr)listrangedimpermutetuple)xaxis new_axis_listys r_permute_to_axis_zerorysBquuw(MM! % &'A rzquantize_per_channel(Tensor input, Tensor scales, Tensor zero_points, int axis, int quant_min, int quant_max, ScalarType dtype) -> Tensorquantize_per_channelscales zero_pointsrvcUR[R[R4;aUR [R 5nUR[R :XdSUR35eX0R 5:dSUR 535e[XEU5 [X5upS/UR 5-nURSUS'URU5nURU5n[R"[R"USU- -5U-XE5n U R[U55n U R U5$)a<Affine per channel quantization for the Tensor using the same quantization parameters for each channel/axis to map from floating point to quantized values Args: input (torch.Tensor): original float32 or bfloat16 Tensor scales (torch.Tensor): a list of scale quantization parameter for affine quantization, one per channel zero_point (torch.Tensor): a list of zero_point quantization parameter for affine quantization, one per channel quant_min (int): minimum quantized value for output Tensor quant_max (int): maximum quantized value for output Tensor dtype (torch.dtype): requested dtype (e.g. torch.uint8) for output Tensor Returns: Tensor with requested dtype (e.g. torch.uint8), note the quantization parameters are not stored in the Tensor, we are storing them in function arguments instead rExpecting axis to be < r/rr)rrr r!r"r#rrrryshapeviewr$r%rsrt) rr{r|rvrrrpermute_axis_list new_shaperesouts rrzrz,s'6 {{u}}enn55' u}}$T Eekk]ST$ ))+ F!8 FF  e<4UAEeiik!I<<?IaL [[ #F""9-K ++ ES6\*+k99 C ++e-. /C 66%=rcUR[R[R4;aUR [R 5nUR[R :XdSUR35eX0R 5:dSUR 535e[XEU5 [R"XS9$)Nrr~r)) rrr r!r"r#rrrr*)rr{r|rvrrrs rquantize_per_channel_metar\s {{u}}enn55' u}}$T Eekk]ST$ ))+ F!8 FF  e<   E //rzdequantize_per_channel(Tensor input, Tensor scales, Tensor? zero_points, int axis, int quant_min, int quant_max, ScalarType dtype, *, ScalarType? out_dtype=None) -> Tensordequantize_per_channelcURU:XdSUSUR35eUc[RnX0R5:dSUR535e[ XEU5 [ X5upS/UR5-n UR SU S'URU 5nUbXRU 5- U-n OX-n U RU5n U R[U55n U $)aOAffine per channel dequantization for the Tensor using the same quantization parameters for each channel/axis to map from quantized values to floating point values Args: input (torch.Tensor): Tensor with dtype matching `dtype` argument, e.g. (`torch.uint8`), it is a per channel quantized Tensor if combined with quantization parameter in the argument of this function (scales/zero_points/axis) scales (torch.Tensor): a list of scale quantization parameter for affine quantization, one per channel zero_points (torch.Tensor): a list of zero_point quantization parameter for affine quantization, one per channel quant_min (int): minimum quantized value for output Tensor (not used in computation, reserved for pattern matching) quant_max (int): maximum quantized value for output Tensor (not used in computation, reserved for pattern matching) dtype (torch.dtype): requested dtype for output Tensor (not used in computation, reserved for pattern matching) out_dtype (torch.dtype?): optional dtype for output Tensor Returns: dequantized float32 Tensor Expecting input to have dtype , but got dtype: r~r/r) rrr#rrrryrrr"rsrt) rr{r|rvrrrr?rrrrs rrrzsR  uN 'w.? }MNMM ))+ F!8 FF  e<4UAEeiik!I<<?IaL [[ #F'' 22f<n && C ++e-. /C Jrc URU:XdSUSUR35eUc[RnX0R5:dSUR535e[ XEU5 [R "XS9$)Nrrr~r))rrr#rrrr*)rr{r|rvrrrr?s rdequantize_per_channel_metars  uN 'w.? }MNMM ))+ F!8 FF  e<   E 33rzLchoose_qparams_per_token(Tensor input, ScalarType dtype) -> (Tensor, Tensor)choose_qparams_per_tokenchUR5RSSS9nUR[R:XaUR 5nU[R :XaSnSUS- -S- nO[SU35eURSS 9RU5n[R"U5nX%4$) Choose quantization parameters for per token quantization. This means for a N dimension Tensor (M1, M2, ...Mn, N), we calculate scales/zero_points for each N elements and quantize every N elements with the same quantization parameter. The dimension for scales/zero_points will be (M1 * M2 ... * Mn) Args: input (torch.Tensor): original float32/float16 Tensor dtype (torch.dtype): dtype (e.g. torch.uint8) for input Tensor Returns: scales and zero_points, both float32 Tensors Trrkeepdimr/z/unsupported dtype in choose_qparams_per_token: gh㈵>min) absamaxrrr floatint8 Exceptionr$div zeros_like)rrr{n_bitsrr|s rrrs,YY[  "d  3F ||u}}$ LLN   &1*%) =eW E  \\d\ # ' ' 2F""6*K  rc[URSS5S/-n[R"U[RUR S9[R"U[R UR S94$Nrr/rfrprrrhrirgrjrrsizes rchoose_qparams_per_token_metar]  CR !QC 'D ;;t5<< Eu{{ EKK H rz]_choose_qparams_per_token_asymmetric_impl(Tensor input, ScalarType dtype) -> (Tensor, Tensor))_choose_qparams_per_token_asymmetric_implCompositeImplicitAutogradcSup#[R"USSS9n[R"USSS9n[R"U[R"U55n[R "U[R"U55n[R "[R5RnXv- [X2- 5- n U RUS9n Xi- n Xy- n X*-n X;-n [R"X-S:X*- X;- 5n[R"XU5R5nU R5UR54$)r)irTrrr)raminrrrmaxfinfor#rSrr$wherer%r"float64rj)rrrQrRr^r_ min_val_neg max_val_posrSr descaled_min descaled_maxzero_point_from_min_errorzero_point_from_max_errorrs rrrs,,JDjjB5GjjB5G))GU%5%5g%>?K))GU%5%5g%>?K ++emm $ ( (C &% *< >>rzWchoose_qparams_per_token_asymmetric(Tensor input, ScalarType dtype) -> (Tensor, Tensor)#choose_qparams_per_token_asymmetricc[X5$r;)rrrs rrrEs 5U BBrc[URSS5S/-n[R"U[RUR S9[R"U[R UR S94$rrrs r(choose_qparams_per_token_asymmetric_metarQrrc[R"[UR55SS5nX1R 5:XdSUSUR535eX2R 5:XdSUSUR535eg)Nrz num_tokens: z scales: z zero_points: )mathprodrprr3)rr{r| num_tokenss r!_per_token_quant_qparam_dim_checkr`s4 -cr23Jlln$; j\6;;=/:;$ ''))E j\ 0@0@0B/CDE)rz}quantize_per_token(Tensor input, Tensor scales, Tensor zero_points, int quant_min, int quant_max, ScalarType dtype) -> Tensorquantize_per_tokenc[X4U5 [XU5 URSU- 5RU5R 5R X45R U5nU$)aPer token quantization for the Tensor using the quantization parameters to map from floating point to quantized values. This means for a N dimension Tensor (M1, M2, ...Mn, N), we calculate scales/zero_points for each N elements and quantize every N elements with the same quantization parameter. The dimension for scales/zero_points will be (M1 * M2 ... * Mn) Args: input (torch.Tensor): original float32 or bfloat16 Tensor scales (float32 torch.Tensor): quantization parameter for per token affine quantization zero_points (int32 torch.Tensor): quantization parameter for per token affine quantization quant_min (int): minimum quantized value for output Tensor quant_max (int): maximum quantized value for output Tensor dtype (torch.dtype): requested dtype (e.g. torch.uint8) for output Tensor Returns: Tensor with requested dtype (e.g. torch.uint8), note the quantization parameters are not stored in the Tensor, we are storing them in function arguments instead r)rrmuladdr%r$r"rr{r|rrrs rrrpsX6 e<%e[A #, [   y $ E   LrcB[X4U5 [R"XS9$rErrr*rs rquantize_per_token_metars e<   E //rzdequantize_per_token(Tensor input, Tensor scales, Tensor zero_points, int quant_min, int quant_max, ScalarType dtype, ScalarType output_dtype) -> Tensordequantize_per_token output_dtypec4X- nX-nURU5$)aPer token dequantization for the Tensor using the quantization parameters to map from floating point to quantized values. This means for a N dimension Tensor (M1, M2, ...Mn, N), we calculate scales/zero_points for each N elements and quantize every N elements with the same quantization parameter. The dimension for scales/zero_points will be (M1 * M2 ... * Mn) Args: input (torch.Tensor): quantized Tensor (uint8, int8 etc.) scales (float64 torch.Tensor): quantization parameter for per token affine quantization zero_points (int64 torch.Tensor): quantization parameter for per token affine quantization quant_min (int): minimum quantized value for input Tensor quant_max (int): maximum quantized value for input Tensor dtype (torch.dtype): dtype (e.g. torch.uint8) for input Tensor output_dtype (torch.dtype): dtype (e.g. torch.float32) for output Tensor Returns: dequantized Tensor with dtype `output_dtype` )r"rr{r|rrrrs rrrs"8  E NE 88L !!rcB[X4U5 [R"XS9$rErrs rdequantize_per_token_metars e<   E 66rzquantize_per_channel_group(Tensor input, Tensor scales, Tensor zero_points, int quant_min, int quant_max, ScalarType dtype, int group_size) -> Tensorquantize_per_channel_groupcVUS:deX`RS:a"URSS:XaURSnURSU-S:XdeUR5S:XdeURSU5n[R"U5R 5S:XdeURSS5nURSS5nUR SU- 5RU5R5RX45RU5RU5nU$)Nr/rrrr) rrrreshaperisnansumrrr%clamp_r" reshape_as) rr{r|rrr group_sizeto_quant input_int8s rrrs  >>KKO# R(8A(=[[_ ;;r?Z '1 ,, , 99;!  }}R,H ;;x $ $ &! ++ + ^^B "F%%b!,K  S6\" [    % E E   rcUS:deX`RS:a"URSS:XaURSnURSU-S:XdeUR5S:Xde[R"XS9$)aGroupwise quantization within each channel for an 2-d Tensor using the quantization parameters to map from floating point to quantized values. This means for each row of a 2-d Tensor (M, N), we calculate scales/zero_points for each `group_size` elements and quantize every `group_size` elements with the same quantization parameter. The dimension for scales/zero_points will be (M * ceil(N, group_size),) Args: input (torch.Tensor): original float32 or bfloat16 Tensor scales (float32 torch.Tensor): quantization parameter for per channel group affine quantization zero_points (int32 torch.Tensor): quantization parameter for per channel group affine quantization quant_min (int): minimum quantized value for output Tensor quant_max (int): maximum quantized value for output Tensor dtype (torch.dtype): requested dtype (e.g. torch.uint8) for output Tensor Returns: Tensor with requested dtype (e.g. torch.uint8), note the quantization parameters are not stored in the Tensor, we are storing them in function arguments instead r/rrrr))rrrrr*)rr{r|rrrrs rquantize_per_channel_group_metar s{8 >>KKO# R(8A(=[[_ ;;r?Z '1 ,, , 99;!     E //rzdequantize_per_channel_group(Tensor input, Tensor scales, Tensor? zero_points, int quant_min, int quant_max, ScalarType dtype, int group_size, ScalarType output_dtype) -> Tensordequantize_per_channel_groupw_int8rc&US:deX`RS:a"URSS:XaURSnURSU-S:XdeUR5S:XdeURSU5nURSS5nUbURSS5n O.[R"/[R UR S9n URU 5RU5RU5RU5n U $)aGroupwise dequantization within each channel for an 2-d Tensor using the quantization parameters to map from floating point to quantized values. This means for each row of a 2-d Tensor (M, N), we calculate scales/zero_points for each `group_size` elements and quantize every `group_size` elements with the same quantization parameter. The dimension for scales/zero_points will be (M * ceil(N, group_size),) Args: input (torch.Tensor): quantized Tensor (uint8/int8 etc.) scales (float32 torch.Tensor): quantization parameter for per channel group affine quantization zero_points (int32 torch.Tensor): quantization parameter for per channel group affine quantization quant_min (int): minimum quantized value for input Tensor quant_max (int): maximum quantized value for input Tensor dtype (torch.dtype): dtype (e.g. torch.uint8) for input Tensor output_dtype (torch.dtype): dtype (e.g. torch.float32) for output Tensor Returns: dequantized Tensor with dtype `output_dtype` r/rrrrf) rrrrrzerosint32rgsubrrr") rr{r|rrrrrw_int8_groupedzpw_dqs rrr5sD >>LL$$b)9Q)>\\"% << j (A -- - ::<1  ^^B 3N ^^B "F  Q ' [[5;;v}} E   b ! % %f - 8 8 @ C CL QD Krzyfake_quant_per_channel(Tensor input, Tensor scales, Tensor zero_points, int axis, int quant_min, int quant_max) -> Tensorc4\rSrSr\S5r\S5rSrg)FakeQuantPerChannelinc UR[R:waUR[R5nUR[R:waUR[R5nUR[R:XdSUR35eXAR 5:dSUR 535e[ [SU55[ [US-UR55-n[X'5n[X75n [R"USU- -5U -n [R"XU5U - U-n [R"X:X:*5n URU 5 U $)Nrr~rr/r)rrr#r"rrrrprqndimrr%r$ logical_andsave_for_backward) ctxrr{r|rvrrbroadcast_dimsunsqueeze_scalesunsqueeze_zero_pointstemprmasks rforwardFakeQuantPerChannel.forwardosC <<5== (YYu}}-F    +%..5K KK5== ( X I%++ W X (iik!J% Tensorzconvert_element_type.no_fusech[RRRR X5$r;)ropsprimsconvert_element_typedefaultrs rrrs# 99?? / / 7 7 EErc*[R"XS9$rErrs rconvert_element_type_metar s   E //r)r)Mrtypingrr torch._refsrtorch.ao.quantization.utilsrr torch.libraryrrquantized_decomposed_libuint8ruint16int16r_INTEGER_DTYPES float8_e5m2 float8_e4m3fn _FLOAT_DTYPESiinforrr updateintrrdefiner\rrrr,r5r7r9r<r=rFrJrLrNrPrtr`rdrlrnryrzrrrrrrrrrrrr#rrrrrautogradFunctionrrrrr )ks0rrs  +M' ##95A;; ELL%++u{{S""E$7$78 :I9HA AEKKN..//DQRMqU[[^   #ekk!n&8&8"9::MR  @  57RS! <<! !! !  ! ;; ! \\!T!H 5v> 0 << 0  0 0 0  0 ;; 0 \\ 0? 0 @ :G(,Q <<Q <<Q Q Q  Q ;; Q $Q \\QHQ4e #(, << << ||  ||  ;;  $ \\  @ ?H(,  $  \\ I 7  79TU$ <<$"$*-$49$BG++$ 5<< %&$V$N7 % $ <<$"$*-$49$BG++$ 5<< %&$  $N 7@ <<$'47>CLQKK 5<< %&A" A6J <<$'47>CLQKK 5<< %&K@  68ST, <<, LL,,  ,  ,  , ;;, \\,U,^ 6?0 <<0 LL00  0  0  0 ;;0 \\0@0._  8:UV(,; <<; LL;%,,';  ;  ;  ; ;;; $; \\;W;| 8&A(,4 <<4 LL4%,,'4  4  4  4 ;;4 $4 \\4B4*R   <<  ;;  5<< %&   F   << ;; 5<< %&  c / (? <<(? ;;(? 5<< %&(?  (?V] ) C <<C ;;C 5<< %&C  C)   << ;; 5<< %&  E@  46QR# <<# LL## #  # ;; #S#L 4f= 0 << 0 LL 0 0 0  0 ;; 0> 0Y  68ST!& " <<" LL"" "  " ;; "++"U"B 6?!&  7 << 7 LL 7 7 7  7 ;; 7++ 7@ 7A:G0U\\0%++0%,,0H0E%SsAw(A w-