deepspeedai--deepspeed/deepspeed/datastates

DataStates-LLM checkpointing engine.

This feature is not enabled by default. To enable, set the following options in ds_config.json and download the DataStates-LLM checkpointing library. A detailed tutorial is available here.

{
    ... other deepspeed config options,
    "datastates_ckpt": {
        "host_cache_size": 16
	}
}