Support HuggingFace SafeTensor in TorchSharp

[GeorgeS2019]

@shaltielshmid

#1231

Many python pkl files are inaccessible to TorchSharp but could be re-export as binary .safetensor

The approach has been proposed and prototype by @shaltielshmid in 2024 to address HuggingFace increasing promotion of this standard in 2023

None, TorchSharp, Ml.net or latest Donet Generative AI currently is looking into this Hugging face standard

Proposing here to consider integrating the feature to TirchSharp..as pytorch has support for .safetensor

[shaltielshmid]

Hi @GeorgeS2019 , it's currently supported by installing the additional package TorchSharp.PyBridge, example usage is:

using TorchSharp.PyBridge;

var model = ...;
model.save_safetensors("path_to_save_model.safetensors");