sh6*SSKrSrSrSrSrSrg)NcH[R"SXS9SS2S4RSU5n[R"SXS9RUS5nURS:XaSOSnURS:XaSOSnXF-U-US -U--XF-S --nUS-S:HUS-S:H-R [R 5nUS-S:HUS-S:H-R [R 5n XHU - - nXXU - -nSn XZ-n XZ-n X-U -XJ--U -R S 5$) a This is PyTorch implementation of main part of reorder_meta() function, from tools/util/include/cutlass/util/host_reorder.h file of CUTLASS source tree. Furthermore, CUTLASS template for sparse GEMM decides upon layout of this matrix, and at the moment for the sparse GEMM executed on tensor cores, this is layout described by ColumnMajorInterleaved<2> data structure, in include/cutlass/layout/matrix.h of CUTLASS source tree. The reordering of meta matrix into meta_reordered matrix calculated according to these segments of CUTLASS code is re-implemented here. Note that this calculation produces offsets for scattering metadata matrix elements into reordered metadata matrix elements (or, equivalently, for gathering reordered metadata matrix element back into metadata matrix elements). rdeviceN )torcharangerepeatitemsizetoint8view) m meta_ncols meta_dtyperdst_rowsdst_colsgroup interweavetopright bottomleft interleavecols_majcols_mins }/Users/tiagomarins/Projetos/claudeai/copy_bank/venv/lib/python3.13/site-packages/torch/sparse/_semi_structured_conversions.py*_calculate_meta_reordering_scatter_offsetsr!sP ||Aq0D9@@JOH||Az9@@AFH%%*BE ))Q.AJE! a<: % &   ! " A"x!|q'89==ejjIHa<1$A):;?? KJ :%%H :%%H J%H$H L: %(= = H N Nr RRcUR5S:wa[SUR5S35eURupURn[R nUR [R :Xa[RnOgUR [R[R[R4;a[RnO[SUR S35eURS-S-nUS;a [S 5eU[R:XaUS -S :wa[S US 35eOUS-S :wa[S US35eUSU--S :wa[SUSSU-35eUR [R:wa/SnURSX&-U5nUS :gRS5uppO1SnURSX&-U5n U S :gRS5=upupX&U--n X-nU)U -nU)U )-nUnUnUU-U -nX)-nUUR[R 5S--nUUR[R 5S--nUR [R:walWR#SUR%S55nUR#SUR%S55n[R&"UU4SS9RXS-5nO6W R#SUR%S5S-5RXS-5nUUS--nURSX45RU5nUS:Xa=USS2SS2S 4USS2SS2S4S--USS2SS2S4S--USS2SS2S4S--nOUS:Xa|USS2SS2S 4USS2SS2S4S--USS2SS2S4S--USS2SS2S4S--USS2SS2S4S --USS2SS2S4S--USS2SS2S4S--USS2SS2S4S--nWR)X-45n[+XXC5nUR-S UURS55 UURX54$)z This function converts dense matrix into sparse semi-structured representation, producing "compressed" matrix, in the layout used by CUTLASS backend, and corresponding metadata matrix. rz)Expected 2-dimensional dense tensor, got -dimensional tensorInvalid datatype z of dense matrixr r )r r z6Invalid number of elements per meta element calculatedr rzNumber of rows of dense matrix z must be divisible by 16rz must be divisible by 32z"Number of columns of dense matrix z must be divisible by r r)dimN )r& RuntimeErrorshaperr rdtypeint32halfbfloat16floatint16rrunbindrint64gather unsqueezestack new_emptyr!scatter_)denserkrrquadbits_per_meta_elemksparsedense_4m0m1_m2m3dense_2rexpr0expr1expr2bit0bit1bit2bit3idxs0idxs1sparse0sparse1sparsemeta_4meta_nmetameta_reordered meta_offsetss r )sparse_semi_structured_from_dense_cutlassrY/s  yy{a7 }DW X   ;;DA \\FJ {{ejj [[ U^^U[[A A[[ .u{{m;KLMM'00149V+STTU[[ r6Q;1!4LM   r6Q;1!4LM  A& &'1,03I!NdJdIe f   {{ekk!**Rw7"a<//3R**Rw7#qL0044&"!778JH GE C"HE C2#IE D D 5=2 D 3;D DGGEKK(A- .E DGGEKK(A- .E {{ekk!..U__R%89..U__R%89gw/R8==aaHEOOB$71$<=BB11fM eqj !F [["jA B E Ej QF" 1a7OaAg!# %aAg!# %aAg"$ & 1 $ 1a7OaAg!# %aAg!# %aAg"$ &aAg"$  & aAg"$  & aAg"$  &aAg"$ & ^^Q^$56N= zLA|TYYr]; N''6 77r"cr UR5S:wa[SUR5S35eURup#URnUR5S:wa[SUR5S35eURU:wa[SUSURS35eURnU[ R [ R4;a[SUS 35eURS -S -nUR[ R:waS nOSnURupX:wa[S US U35eX-U-SU-:wa[SUSX-U-S-S35e[X)XT5n [ R"URS5SU 5RX)5n [ R"X)SU-4UUS9n US :XaU S-U SS2SS2S4'U S- S-U SS2SS2S4'U S - S-U SS2SS2S4'U S- S-U SS2SS2S4'U S - S-U SS2SS2S 4'U S- S-U SS2SS2S4'U S- S-U SS2SS2S4'U S- S-U SS2SS2S4'GO$US :XGaU S-U SS2SS2S4'U S- S-U SS2SS2S4'U S - S-U SS2SS2S4'U S- S-U SS2SS2S4'U S - S-U SS2SS2S 4'U S- S-U SS2SS2S4'U S- S-U SS2SS2S4'U S- S-U SS2SS2S4'U S- S-U SS2SS2S 4'U S- S-U SS2SS2S4'U S - S-U SS2SS2S4'U S!- S-U SS2SS2S"4'U S#- S-U SS2SS2S4'U S$- S-U SS2SS2S%4'U S&- S-U SS2SS2S4'U S'- S-U SS2SS2S(4'U RS5[ R"SSU-U-U-US)9S -RSS5RSS5RS5-n [ R "US-U-4URUS9nUR[ R:wa"UR#SXRS55 O[UR[ R$5R#SXR[ R$5RS55 URUSU-5$)*z This function performs reverse of the function above - it reconstructs dense matrix from a pair of "compressed" matrix, given in the layout used by CUTLASS backend, and accompanying metadata matrix. rz*Expected 2-dimensional sparse tensor, got r$z(Expected 2-dimensional meta tensor, got zExpected meta matrix to be on z device, got matrix on z devicer%z of meta matrixr r zNumber of rows of meta matrix z4 must be equal to number of columns of spase matrix z#Number of columns of sparse matrix z different from the z<, expected according to the number of columns of meta matrixr rr1rr'Nrr+ r)r(r-r  r* r, r.r)r&r/r0rr1r r6r2rr5r!r9remptyrrzerosr=r3)rSrWrr?rrr@rA meta_nrowsrrXrVmeta_2 dense_offsetsr>s r 'sparse_semi_structured_to_dense_cutlassrks3zz|q8FY Z   < zL <<++B/L A F Fq UD[[ 223F "+q!Qw19,q!Qw19,q!Qw19,q!Qw19,q!Qw2:-q!Qw2:-q!Qw2:-q!Qw 1 $+q!Qw19,q!Qw19,q!Qw19,q!Qw19,q!Qw2:-q!Qw2:-q!Qw2:-q!Qw2:-q!Qw2:-q!Qw BJ$.q!Rx BJ$.q!Rx BJ$.q!Rx BJ$.q!Rx BJ$.q!Rx BJ$.q!RxKKO QA W,V  ::aQ r"cSnURSSS5RSSS5HnUH nU"U5 M M U$)a This function computes a 2:4 sparse tile by greedily taking the largest values. Since we take the largest values greedily, how the sorting algorithm handles duplicates affects the ultimate sparsity pattern. Note that this function does not have the same sorting semantics as our CUDA backend, which is exposed via `torch._sparse_semi_structured_tile` and thus returns a different pattern. c/SQn/SQnUR5RSSS9RH<nUS-US-pTXS:a"X%S:aX==S- ss'X%==S- ss'M6SXU4'M> g)N)rrrrT) descendingstabler rrr)flattensortindices)tile num_kept_row num_kept_colxrcs r greedy_prune_tile7_sparse_semi_structured_tile..greedy_prune_tile(sx# # $$T$BJJA61q5q"|':1$1$T Kr"rr r)unfold)r>rybatchrss r _sparse_semi_structured_tiler}sH aA&--aA6D d #7 Lr"cUR5R[R5nUR SSS5R SSS5nUR SSS5R SSS5nUR "/UR SSQSPSPSP76nS[R"S[RSS 9-nUR[R5U-R[R5nU$) z@ Calculates the compressed swizzled bitmask from a dense tensor rr rrr r'Ncudar[) boolrr uint8r{reshaper0rr5)r> int_bitmaskbitmask_8x8_chunksbitmask_4x4_chunksbitmask_binary_representation powers_of_twocompressed_swizzled_bitmasks r $_compute_compressed_swizzled_bitmaskr;s **,//%++.K%++Aq!4;;Aq!D,221a;BB1aK%7$>$>%  ! !"1 %%'(%*+%-.%! au{{6JJM &((5 Ebo  '&r")r r!rYrkr}rr"r rs' 'STB8Jf R<)'r"