Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

let safetensors.load support load weight from bytearray #529

Open
JeryZeng opened this issue Sep 24, 2024 · 0 comments
Open

let safetensors.load support load weight from bytearray #529

JeryZeng opened this issue Sep 24, 2024 · 0 comments

Comments

@JeryZeng
Copy link

Feature request

current safetensors.load only support load memory data from bytes, while read large weight data, bytearray must convert to bytes first,this operation make memoery peak use double and copy ram is a huge cost. so read weight from bytearray can reduce cost.

Motivation

while concurrent read a large file from distribute file system(can speed up read), use bytearray can reduce ram useage,but safetensors read only support bytes memory data

Your contribution

none

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant