sh5FSSKrSSKrSSKrSSKJr SSKJr SSKJrJ r J r SSK J r SSK r SSKJr SSKJrJr SSKJr SSKJr SS KJr SS KJr SS KJr SS KJr SS K J!r!J"r" SSK#J$r$J%r% SSK&J'r' SSK(J)r) SSK*J+r+ SSK,J-r- SSK.J/r/J0r0J1r1 /SQr2"SS\5r3\ "S\4S9S-S\"S\+S\ \RjS\6S\7S \ \%S!\!4S"jj5r8\"S#S$9\/SSSSSS%.S\"S&\ \9\RtS4S\ \+S \ \%S\ \RjS\7S!\!4S'jj55r;\"S#S$9SSSS\3RxS(.S\"S&\ \9\RtS4S\ \+S \ \%S\ \RjS)\3S!\4S*jj5r=S\"S!\"4S+jr>S-S\"S\+S\ \RjS\6S\7S \ \%S!\!4S,jjr?g).N)Future)Enum)castOptionalUnion) deprecated)_copy_state_dict_create_cpu_state_dict)_AsyncCheckpointExecutor)$_ProcessBasedAsyncCheckpointExecutor)#_ThreadBasedAsyncCheckpointExecutor)_storage_setup)DefaultSavePlanner)_dcp_method_logger)MetadataSTATE_DICT_TYPE)SavePlan SavePlanner) AsyncStager)Stateful) StorageWriter)_get_default_group) _api_bc_check _DistWrapper_profile)save_state_dictsave async_saveAsyncCheckpointerTypec \rSrSrSrSrSrSrg)r 'z!Enum for async checkpointer type.threadprocessN)__name__ __module__ __qualname____firstlineno____doc__THREADPROCESS__static_attributes__r%ځ/Users/tiagomarins/Projetos/claudeai/copy_bank/venv/lib/python3.13/site-packages/torch/distributed/checkpoint/state_dict_saver.pyr r 's+ FGr.r za`save_state_dict` is deprecated and will be removed in future versions.Please use `save` instead.)categoryF state_dictstorage_writer process_groupcoordinator_rankno_distplannerreturnc UR5 [5 [UUUUUU5sSSS5 $!,(df  g=f)z3This method is deprecated. Please switch to 'save'.N)resetr_save_state_dict)r1r2r3r4r5r6s r/rr.s;          s 5 AT)log_exceptions) checkpoint_idr2r6r3r5r<c [RRS5 U=(d; [R"5(+=(d [R "5(+nU(a[ R"S5 [5 [[[X!SS95n[[U5UUUUS9sSSS5 $!,(df  g=f)a Save a distributed model in SPMD style. This function is different from ``torch.save()`` as it handles ``ShardedTensor`` , and ``DTensor`` by having each rank only save their local shards. For each ``Stateful`` object (having both a ``state_dict`` and a ``load_state_dict``), save will call ``state_dict`` before serialization. .. warning:: There is no guarantees of Backwards Compatibility across PyTorch versions for saved state_dicts. .. warning:: If using the `process_group` argument, make sure that only its ranks call `save_state_dict` and that all data in state_dict belong to it. .. note:: When saving checkpoint for FSDP's `ShardingStrategy.HYBRID_SHARD`, only one of the shard_group should be calling `save_state_dict` and the corresponding process group needs to be passed in. .. note:: If no process group is available, this function assumes the intention is to save the state_dict in the local process. .. note: Rank 0 is assumed to be the coordinator rank. Args: state_dict (Dict[str, Any]): The state_dict to save. checkpoint_id (Union[str, os.PathLike, None]): The ID of this checkpoint instance. The meaning of the checkpoint_id depends on the storage. It can be a path to a folder or to a file. It can also be a key if the storage is a key-value store. (Default: ``None``) storage_writer (Optional[StorageWriter]): Instance of StorageWriter used to perform writes. If this is not specified, DCP will automatically infer the writer based on the checkpoint_id. If checkpoint_id is also None, an exception will be raised. (Default: ``None``) planner (Optional[SavePlanner]): Instance of SavePlanner. If this is not specificed, the default planner will be used. (Default: ``None``) process_group (Optional[ProcessGroup]): ProcessGroup to be used for cross-rank synchronization. (Default: ``None``) no_dist (bool): If ``True``, this function will assume the intent is to load a checkpoint without using cross-rank synchronization. (Default: ``False``) Returns: Metadata: Metadata object for the saved checkpoint. Example: >>> # xdoctest: +SKIP >>> my_model = MyModule() >>> state_dict = {"model": my_model} >>> fs_storage_writer = torch.distributed.checkpoint.FileSystemWriter( ... "/checkpoint/1" ... ) >>> torch.distributed.checkpoint.save( >>> state_dict=state_dict, >>> storage_writer=fs_storage_writer, >>> ) .. note:: save_state_dict uses collectives to coordinate writes across ranks. For NCCL-based process groups, internal tensor representations of objects must be moved to the GPU device before communication takes place. In this case, the device used is given by ``torch.cuda.current_device()`` and it is the user's responsibility to ensure that this is set so that each rank has an individual GPU, via ``torch.cuda.set_device()``. z!torch.distributed.checkpoint.savezptorch.distributed is disabled, unavailable or uninitialized, assuming the intent is to save in a single process.Freader)r1r2r3r5r6N)torch_C_log_api_usage_oncedist is_availableis_initializedwarningswarnrrrrr:_stateful_to_state_dict)r1r<r2r6r3r5s r/rrJsr HH  !DEQd//11Q4;N;N;P7PG ~   >.PUV  .z:)'  s .C C)r<r2r6r3async_checkpointer_typerIc [RRS5 [R"5(aX[R "5(a>U=(d [ 5n[R"S5UR;dS5e[[[X!SS95n[U5n[U[5(aURU5nO[!U5n[#XSS9 U[$R&:Xa [)5O [+5nUR-UUUUUS9n [U[5(a!UR.(aUR15 U $)aAsynchronous version of ``save``. This code first de-stages the state_dict on to the staging storage (defaults to CPU memory), and then calls the `save` in a separate thread. .. warning:: This feature is experimental and subject to change. Args: state_dict (Dict[str, Any]): The state_dict to save. checkpoint_id (Union[str, os.PathLike, None]): The ID of this checkpoint instance. The meaning of the checkpoint_id depends on the storage. It can be a path to a folder or to a file. It can also be a key if the storage is a key-value store. (Default: ``None``) storage_writer (Optional[StorageWriter]): Instance of StorageWriter used to perform 'stage' and 'save'. If this is not specified, DCP will automatically infer the writer based on the checkpoint_id. If checkpoint_id is also None, an exception will be raised. (Default: ``None``) planner (Optional[SavePlanner]): Instance of SavePlanner. If this is not specificed, the default planner will be used. (Default: ``None``) process_group (Optional[ProcessGroup]): ProcessGroup to be used for cross-rank synchronization. (Default: ``None``) Returns: Future: A future holding the resultant Metadata object from `save`. Example: >>> # xdoctest: +SKIP >>> my_model = MyModule() >>> state_dict = {"model": my_model} >>> fs_storage_writer = torch.distributed.checkpoint.FileSystemWriter( ... "/checkpoint/1" ... ) >>> checkpoint_future = torch.distributed.checkpoint.async_save( >>> state_dict=state_dict, >>> storage_writer=fs_storage_writer, >>> ) >>> >>> # ... do some work ... >>> >>> checkpoint_future.result() z'torch.distributed.checkpoint.async_savecpuzfA CPU backend must be enabled for async save; try initializing process group with 'cpu:gloo,cuda:nccl'Fr>) type_check)r<r2r6r3)r@rArBrCrDrErdevice _device_typesrrrrH isinstancerstager r r r,r r execute_save should_synchronize_after_executesynchronize_staging) r1r<r2r6r3rIpgstaged_state_dictexecutorfs r/rrs7r HH  !JK t2244  202 LL 2#3#3 3  u 3 ~nERN)4J.+..*00<2:>5I #&;&C&C C -. 0 2 %%#%# &A >;//  ; ;**, Hr.c0nUR5H.up#[U[5(aUR5OUX'M0 U$)z]Creates a shallow copy of `state_dict` where `state_dict` is called for each Stateful object.)itemsrOrr1)r1stateful_state_dictkeyelems r/rHrHsE%%' !+D(!;!;DOO   ( r.c^^^^ ^ ^[RRS5 [X$(+U5m Tc [ 5mTceSm0n[ TSS5=nbXvS'T R US'[S 0UD6U UUU4Sj5n[S 0UD6UUU4Sj5n T RSX5m [S 0UD6U UU4Sj5n [S 0UD6UU4Sj5n T RS X5$) Nz,torch.distributed.checkpoint.save_state_dictr<r3c>TceTR5nS[R"TR5R;a3[ R "S5 TRTTR5 OTRTUTRS9 TRTR5 TR5nTRU5nU$)N storage_metazThe function definition for SavePlanner.set_up_planner has been updated to include the storage_meta argument. Please update your implementation to include this parameter.)r1r_is_coordinator) r_inspect signatureset_up_planner parametersrFrGr`set_up_storage_writercreate_local_planprepare_local_plan)r_ local_plandistWr6r1r2s r/ local_step$_save_state_dict..local_step>s"""%224 !2!273I3I!J!U!U U MM.   " ":u/C/C D  " "%)$33 #  ,,U-A-AB..0 #66zB r.c\>TceTRU5unmTRU5nU$N)create_global_planprepare_global_plan)all_local_plansglobal_metadatar6r2s r/ global_step%_save_state_dict..global_stepUs<"""+2+E+Eo+V((<<_Mr.planc>TceTRT5nTRUT5nUR5 UR5$rm) finish_plan write_datawaitvalue)final_local_plan all_writes central_planr6r2s r/rw$_save_state_dict..write_data`sL""""..|<#../?I !!r.c2>TceTRTUS9 T$)N)metadataresults)finish) all_resultsrqr2s r/finish_checkpoint+_save_state_dict..finish_checkpointis(*** Lr.writer%) r@rArBrrgetattrgrouprreduce_scatter all_reduce)r1r2r3r4r5r6 ckpt_kwargsckpt_idrjrrrwrr|rirqs`` ` @@@r/r:r:(s  HH  !OP  5E FE$&   OK>?DAAN'.O$',{{ O$&+&',&+&'#11&*RL&+&"'"&+&'   GZ CCr.)NrFN)@raosrFconcurrent.futuresrenumrtypingrrrtyping_extensionsrr@torch.distributed distributedrC#torch.distributed._state_dict_utilsr r ,torch.distributed.checkpoint._async_executorr 4torch.distributed.checkpoint._async_process_executorr 3torch.distributed.checkpoint._async_thread_executorr +torch.distributed.checkpoint._storage_utilsr,torch.distributed.checkpoint.default_plannerr#torch.distributed.checkpoint.loggerr%torch.distributed.checkpoint.metadatarr$torch.distributed.checkpoint.plannerrr$torch.distributed.checkpoint.stagingr%torch.distributed.checkpoint.statefulr$torch.distributed.checkpoint.storager"torch.distributed.distributed_c10drutilsrrr__all__r FutureWarning ProcessGroupintboolrstrPathLikerr+rrHr:r%r.r/rs %((( XGKBKF<:>A88 MD ! 26%)   ! D--.     k "     .4(48.2%)15j j bkk4/0j ]+ j k " j D--. j j j )j Z4(48.2%)155J5Q5Qa a bkk4/0a ]+ a k " a D--. a 3a  a )a HO26%) GDGD!GDD--.GD GD  GD k " GDGDr.