shXSSKrSSKJrJr SSKJrJrJrJr SSK r SSK J r SSK J r SSK Jr S\4SjrS\\\\ 4\\\\ 44S \\\\ 4\\\\ 44S\\\\44S jrS \ R*S \S \\\\ 44S\\ /\ 4S\\\ S 4\\S 4\\\\444SjrS \ R*S\\\ S 4\\S 4\\\\444SjrS \ R*S\\\ S 4\\S 4\\\\444SjrS3S \ R*S\\S\\ S\SS4 SjjrS \ R*S\\\\4S\\ S\\ 4SjrS3S \ R*S\\S\\ S\SS4 SjjrS4S\ R*S\\ S\\S\\ S\\S\ R*4 S jjrS\ R*4S!jrS\ R*4S"jr"S#S$\ R*5r "S%S&\ R*5r!S3S\ R*S'\S\\!\\ S 444S(jjr"S3S\ R*S'\S\\ \\ S 4\\ S 444S)jjr#S*\\\ S 4S 4S\\ S 44S+jr$S,\\ R*S\\ \\ S 4\\ S 444S-jr%S5S.\&\ R*S/\\S\\'4S0\ RPRR4S1jjr*S5S.\&\ R*S/\\S\\'4S0\ RPRR4S2jjr+g)6N)IterableSequence)AnyCallableNoReturnUnion)Tensor)NamedMemberAccessorreturnc[S5e)Na$make_functional(module): we don't yet support models that do parameter tying (also sometimes known as weight sharing). Please try to rewrite your model by replacing all instances of the tied parameter with another and/or comment your support in https://github.com/pytorch/functorch/issues/446) RuntimeErrort/Users/tiagomarins/Projetos/claudeai/copy_bank/venv/lib/python3.13/site-packages/torch/_functorch/make_functional.pyraise_parameter_tying_errorrs  : r named_paramstied_named_paramsc[U5n[U5n[UR55n[UR55nURU5(de0nUR 5H upVU/4XF'M UR 5H"upVXd;deXFSR U5 M$ [UR 55$)aC named_params is a dictionary of tensors: {'A': A, 'B': B} tied_named_params is another dictionary of tensors {'A': A, 'B': B, 'B_tied': B} with potentially tied (or 'duplicated') tensors This function creates a mapping from the names in named_params to the names in tied_named_params: {'A': ['A'], 'B': ['B', 'B_tied']}. )dictsetkeysissubsetitemsappendvalues)rrtensors_dict_keystied_tensors_dict_keystensor_to_mappingkeytensors rcreate_names_mapr"!s %L./L--/0 !2!7!7!9:  % %&< = == ==?#))+ %("I!,(..0 ***!!$++C01 !((* ++rmod named_members.subclassc6[U"SS95n[U"SS95n[XC5n0n[U5nUH<upX;aU"[R"U SS95Xi'Xin UR X5 M> [ U5S:XaSupO [U6upXU4$)NF)remove_duplicateTmeta)devicerrr)tupler"r torch empty_like set_tensorlenzip) r#r$r%all_named_membersunique_named_members names_mapmemoaccessornamep replacementnamesparamss r_extract_membersr;=s mUCD !EF !5II D"3'H$ =u//&ABDGg D. %   A% v12  ) ##rcJ[XR[R5$)a> This function removes all the Parameters from the model and return them as a tuple as well as their original attribute names. The weights must be re-loaded with `load_weights` before the model can be used again. Note that this function modifies the model in place and after this call, mod.parameters() will be empty. )r;named_parametersnn Parameterr#s rextract_weightsrAVs C!5!5r|| DDrc0[XRS5$)NcU$Nr)xs r!extract_buffers..gsar)r; named_buffersr@s rextract_buffersrIds C!2!2K @@rr9r: as_paramsc[U5nU(a&UVs/sHn[R"U5PM nnURX5 gs snf)z Reload a set of weights so that `mod` can be used again to perform a forward pass. Note that the `params` are regular Tensors (that can have history) and so are left as Tensors. This means that mod.parameters() will still be empty after this call. N)r r>r? set_tensors)r#r9r:rJr5r7s r load_weightsrMjs@#3'H+126a",,q/62 '3s A r3elemsc/n[U5n[UR5U5HUuupVn[U5H>upUS:Xa"UR UR X55 M-UR X5 M@ MW U$)Nr)r r0r enumerater swap_tensorr.) r#r3rNresultr5_ attr_nameselemi attr_names r _swap_staterX{svF"3'H!$Y__%6!>%j1LAAv h229CD##I4 2"? Mrbuffersc<[U5nURX5 grD)r rL)r#r9rYrJr5s r load_buffersr[s #3'H (rrmodelweights weight_names buffer_namesc[U5[U5:Xde[XU5 [U5S:a&[U5[U5:Xde[XU5 U$)zload_state(model, weights, weight_names, buffers=(), buffer_names=()) -> model load_state takes `weights` and `buffers` and assigns them to the model. This is the inverse operation of `make_functional_deprecated_v1`. r)r/rMr[)r\r]r^rYr_s r load_staterasW | G ,, ,g. 7|a< CL000U'2 Lrc^^[TR55n[U5S:a [S5e[ T5unmnUU4SjnX$T4$)amake_functional_deprecated_v1(model) -> weights, func, weight_names Given an nn.Module, make_functional_deprecated_v1 extracts the state (weights) and returns a functional version of the model, `func`. This makes it so that it is possible use transforms over the parameters of `model`. `func` can be invoked as follows: ``` x = torch.randn(4, 3) model = nn.Linear(3, 3) weights, func, _ = make_functional_deprecated_v1(model) func(weights, (x,)) ``` And here is an example of applying the grad transform: ``` x = torch.randn(4, 3) model = nn.Linear(3, 3) weights, _, func = make_functional_deprecated_v1(model) grad_weights = grad(func)(weights, (x,)) ``` To put the state back into a model, use `load_state`. rzmake_functional_deprecated_v1(model): `model` has buffers. Please use make_functional_with_buffers_deprecated_v1(model) instead.cT>[R"T5n[UTU5 U"U6$rD)copydeepcopyrM)r]data mutable_model descriptorsr\s rfun*make_functional_deprecated_v1..funs( e, ]K9d##r)listrYr/r rA)r\rYr]rSrirhs` @rmake_functional_deprecated_v1rlsY45==?#G 7|a I  .e4G[!$  $$rc`^^^[T5unmn[T5unmnUUU4SjnXUTT4$)amake_functional_with_buffers_deprecated_v1(model) -> weights, buffers, func, weight_names, buffer_names Given an nn.Module, make_functional_with_buffers_deprecated_v1 extracts the state (weights and buffers) and returns a functional version of the model, `func`. `func` can be invoked as follows: ``` x = torch.randn(4, 3) model = nn.Linear(3, 3) weights, buffers, func, _, _ = make_functional_with_buffers_deprecated_v1(model) func(weights, buffers, (x,)) ``` And here is an example of applying the grad transform: ``` x = torch.randn(4, 3) model = nn.Linear(3, 3) weights, buffers, func, _, _ = make_functional_with_buffers_deprecated_v1(model) func(weights, buffers, (x,)) grad_weights = grad(func)(weights, buffers, (x,)) ``` To put the state back into a model, use `load_state`. cn>[R"T5n[UTU5 [UTU5 U"U6$rD)rdrerMr[)r]rYrfrgbuf_descriptorsr\weight_descriptorss rri7make_functional_with_buffers_deprecated_v1..funs5 e, ]$6@]OW=d##r)rArI)r\r]rSrYrirorps` @@r*make_functional_with_buffers_deprecated_v1rrs?2&5U%;"G "1%"8G_a$ S"4o EErc ^\rSrSrSrS\R S\\S4S\\S4S\ \\ \4S\ \\ \4S S 4 U4S jjr \ SS \R S \ S \S\\S4\\S444Sjj5rS\\S\\S \4SjrSrU=r$)FunctionalModuleWithBufferszO This is the callable object returned by :func:`make_functional_with_buffers`. stateless_model param_names.r_param_names_mapbuffer_names_mapr Nc>[TU]5 XlX lX0l[ U5UlUR RU5 grD)super__init__rvrwr_r all_names_mapupdate)selfrvrwr_rxry __class__s rr|$FunctionalModuleWithBuffers.__init__sD .&(!/2 !!"23rr\disable_autograd_trackingc[R"U5n[U5up4n[U5upgnU(aUHn U R S5 M [ X$XuU5UU4$NF)rdrerArIrequires_grad_rt) r\r model_copyr:rwrxrYr_ryparams r _create_from(FunctionalModuleWithBuffers._create_from sp ]]5) />z/J,_2A*2M// $$$U+  (HX      rr:rYc"[URUR[U5[U5-5nUR"U0UD6[URURU5 $![URURU5 f=frD)rXrvr}r+)rr:rYargskwargs old_states rforward#FunctionalModuleWithBuffers.forwards|     &ME'N *  M''88 ,,d.@.@) LK,,d.@.@) Ls A++#B)r}r_rwrvF__name__ __module__ __qualname____firstlineno____doc__r>Moduler+strrrkr| staticmethodboolr rrrr__static_attributes__ __classcell__rs@rrtrts4438_4CHo 4 c49n- 4 sDI~. 4 4 [TU]5 XlX lX0lgrD)r{r|rvrwr3)rrvrwr3rs rr|FunctionalModule.__init__3s! .&"rr\rc[R"U5n[U5up4nU(aUHnURS5 M [ X$U5U4$r)rdrerArr)r\rrr:rwr3rs rrFunctionalModule._create_from>sN ]]5) )8)D&Y $$$U+  CVKKrr:c[URURU5nUR"U0UD6[URURU5 $![URURU5 f=frD)rXrvr3)rr:rrrs rrFunctionalModule.forwardJsb 4 4dnnfM  I''88 ,,dnni HK,,dnni Hs A#A9)r3rwrvrrrs@rrr.s # #38_ #T#Y' #  # func, params Given a ``torch.nn.Module``, :func:`make_functional` extracts the state (params) and returns a functional version of the model, ``func``. This makes it so that it is possible use transforms over the parameters of ``model``. ``func`` can be invoked as follows: .. code-block:: python import torch import torch.nn as nn from functorch import make_functional x = torch.randn(4, 3) model = nn.Linear(3, 3) func, params = make_functional(model) func(params, x) And here is an example of applying the grad transform over the parameters of a model. .. code-block:: python import torch import torch.nn as nn from functorch import make_functional, grad x = torch.randn(4, 3) t = torch.randn(4, 3) model = nn.Linear(3, 3) func, params = make_functional(model) def compute_loss(params, x, t): y = func(params, x) return nn.functional.mse_loss(y, t) grad_weights = grad(compute_loss)(params, x, t) If the model has any buffers, please use :func:`make_functional_with_buffers` instead. Args: model (torch.nn.Module): Input model. disable_autograd_tracking (bool): Flag to disable gradients tracking for output parameters. The returned params are unrelated to the set of params from the original model. If False (default), the params will have ``requires_grad=True`` on them (aka they will be trackable with regular PyTorch autograd), matching the requires_grad-ness of the params from the original model. Otherwise, the returned params will have ``requires_grad=False``. Default, False. If you plan on using regular PyTorch autograd (e.g., if you want to call ``.backward()`` or ``torch.autograd.grad()``, then set ``disable_autograd_tracking=False``. Otherwise, if you're only planning on using functorch's gradient transforms, then please set ``disable_autograd_tracking=True`` to avoid unnecessarily tracking history with PyTorch autograd. rzdmake_functional(model): `model` has buffers. Please use make_functional_with_buffers(model) instead.r)rkrYr/r rr)r\rrYs rmake_functionalrTsPv5==?#G 7|a ;    ( (  ) rc([RXS9$)a"make_functional_with_buffers(model, disable_autograd_tracking=False) -> func, params, buffers Given a ``torch.nn.Module``, make_functional_with_buffers extracts the state (params and buffers) and returns a functional version of the model ``func`` that can be invoked like a function. ``func`` can be invoked as follows: .. code-block:: python import torch import torch.nn as nn from functorch import make_functional_with_buffers x = torch.randn(4, 3) model = nn.Linear(3, 3) func, params, buffers = make_functional_with_buffers(model) func(params, buffers, x) And here is an example of applying the grad transform over the parameters of a model: .. code-block:: python import torch import torch.nn as nn from functorch import make_functional_with_buffers, grad x = torch.randn(4, 3) t = torch.randn(4, 3) model = nn.Linear(3, 3) func, params, buffers = make_functional_with_buffers(model) def compute_loss(params, buffers, x, t): y = func(params, buffers, x) return nn.functional.mse_loss(y, t) grad_weights = grad(compute_loss)(params, buffers, x, t) Args: model (torch.nn.Module): Input model. disable_autograd_tracking (bool): Flag to disable gradients tracking for output parameters. The returned params are unrelated to the set of params from the original model. If False (default), the params will have ``requires_grad=True`` on them (aka they will be trackable with regular PyTorch autograd), matching the requires_grad-ness of the params from the original model. Otherwise, the returned params will have ``requires_grad=False``. Default, False. If you plan on using regular PyTorch autograd (e.g., if you want to call ``.backward()`` or ``torch.autograd.grad()``, then set ``disable_autograd_tracking=False``. Otherwise, if you're only planning on using functorch's gradient transforms, then please set ``disable_autograd_tracking=True`` to avoid unnecessarily tracking history with PyTorch autograd. r)rtr)r\rs rmake_functional_with_buffersrs p ' 3 3  4 rtuple_of_tuple_of_tensorscL[[U65n[SU55nU$)Nc3j# UH)n[R"U5R5v M+ g7frDr,stackdetach.0shardss r "transpose_stack..s)3L F""$$3L13)r+r0)rresultss rtranspose_stackrs3!&c+D&E F3LG Nrmodelsc^[U5S:Xa [S5e[SU55(d"[SU55(d [S5e[US5m[U4SjU55(d [S5e[ UVs/sHn[ U5PM sn6up#n[ U5n[ U5nUSX44$s snf)acombine_state_for_ensemble(models) -> func, params, buffers Prepares a list of torch.nn.Modules for ensembling with :func:`vmap`. Given a list of ``M`` ``nn.Modules`` of the same class, stacks all of their parameters and buffers together to make ``params`` and ``buffers``. Each parameter and buffer in the result will have an additional dimension of size ``M``. :func:`combine_state_for_ensemble` also returns ``func``, a functional version of one of the models in :attr:`models`. One cannot directly run ``func(params, buffers, *args, **kwargs)`` directly, you probably want to use ``vmap(func, ...)(params, buffers, *args, **kwargs)`` Here's an example of how to ensemble over a very simple model: .. code-block:: python num_models = 5 batch_size = 64 in_features, out_features = 3, 3 models = [torch.nn.Linear(in_features, out_features) for i in range(num_models)] data = torch.randn(batch_size, 3) fmodel, params, buffers = combine_state_for_ensemble(models) output = vmap(fmodel, (0, 0, None))(params, buffers, data) assert output.shape == (num_models, batch_size, out_features) .. warning:: All of the modules being stacked together must be the same (except for the values of their parameters/buffers). For example, they should be in the same mode (training vs eval). This API is subject to change -- we're investigating better ways to create ensembles and would love your feedback how to improve this. rz?combine_state_for_ensemble: Expected at least one model, got 0.c38# UHoRv M g7frDtrainingrms rr-combine_state_for_ensemble.. s+Fq Fsc3B# UHoR(+v M g7frDrrs rrr s2R6azz>>6szTcombine_state_for_ensemble: Expected all models to have the same training/eval mode.c3@># UHn[U5T:Hv M g7frD)type)rr model0_typs rrrs5ftAw*$fszHcombine_state_for_ensemble: Expected all models to be of the same class.)r/r allrr0rr)rr\funcsr:rYrs @rcombine_state_for_ensemblersP 6{a M   +F+ + +s2R62R/R/R 0  fQiJ 5f5 5 5 V  !;A B6% &u -6 BE7V $Fg&G 8V $$ CsC  model_classensemble_shaper)c^^^UUU4SjnU$)Nc>^^[T 5S:a [S5e[T 5S:Xa"T "T0TD6RT 5n[U5$T SnUS::a[SUS35e[ UU UU 4Sj[ U555n[T "T0TD65upVn[ SU55n[ [ U65n[ SU55nXU4$) N,NYI: ensemble_shape with more than 1 elementr num_models  should be > 0c3R># UHnT"T0TD6RT5v M g7frDtorrSrr)rrs rr3functional_init..wrapped...- =NK ( ( + +F 3 3=N$'c3># UHn[U5Sv M g7f)rN)rlrr\s rrr2sTVE5e.wrapped$s ~ ! #KL L ~ ! #0033F;E07 7#A& ?{:,nEF F =B:=N  5[$5Q&5QR uTVTTW &K7KKE!!rrrrr)rs``` rfunctional_initrs "& Nrc^^^UUU4SjnU$)Nc >^^[T 5S:a [S5e[T 5S:Xa"T "T0TD6RT 5n[U5$T SnUS::a[SUS35e[ UU UU 4Sj[ U555n[ T "T0TD65unnnnn[[ SU556up[ [U 65n [ SU 55n [ [U 65n [ S U 55n XXgU4$) Nrrrrrc3R># UHnT"T0TD6RT5v M g7frDrrs rr@functional_init_with_buffers..wrapped..Irrc3># UHn[U5SSv M g7f)Nr)rrrs rrrTs##E;5A"1E#rc3j# UH)n[R"U5R5v M+ g7frDrrs rrrZrrc3j# UH)n[R"U5R5v M+ g7frDrrs rrr\rr)r/rrrlr+rrrr0)rrr\rrrSrr^r_r]rYr)rrs`` rr-functional_init_with_buffers..wrapped?s$ ~ ! #KL L ~ ! #0033F;E07 7#A& ?{:,nEF F =B:=N   7{D7SF7S T     #  W &K7KKW &K7KK<??rrrs``` rfunctional_init_with_buffersr:s @@ Nrrr*)rcpu),rdcollections.abcrrtypingrrrrr,torch.nnr>r %torch.nn.utils._named_member_accessorr rrrr+rkr"rr;rArIrrMrXr[rarlrrrtrrrrrrinttypesDevicerrrrrrs .11 EX,S&[)8E#v+4F+GGH,T#v+.sF{9K0LLM, #tCy.,8$ $C%V *$x'($ 5 uS#XS$s)^0D DE $2 E  E 5 uS#XS$s)^0D DE EA A 5 uS#XS$s)^0D DEA ( ( C=( V ( (  ("   #CcN3 C 99C15C U63;/ /0CN9>: 99:15: &fck(:E&#+