[go: up one dir, main page]

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add function load_buffer_on_device #810

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

Conversation

3togo
Copy link
@3togo 3togo commented Oct 20, 2023

Due to unknown reason, repeat calling of forward method of some models might not produce identical results. Therefore, I need to load the model repeatedly, to ensure consistency. By creating a memory buffer, I hope it will speed up the process a little bit.

let mut file = std::fs::File::open(model_path).unwrap();
let mut buffer: Vec = Vec::new();
file.read_to_end(&mut buffer).unwrap();
let mut model = tch::CModule::load_buffer_on_device(&buffer, device).unwrap();

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants