[go: up one dir, main page]
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add a device parameter to perform encoding using different torch devices such as mps #189

Open
CarloNicolini opened this issue Mar 7, 2023 · 2 comments
Open

Add a device parameter to perform encoding using different torch devices such as mps #189

CarloNicolini opened this issue Mar 7, 2023 · 2 comments
Labels
enhancement New feature or request

Comments

@CarloNicolini
Copy link

Working on a MacBook M1Pro, I often enjoy the concrete speedup that the Metal Performance Shader (mps) device has over the cpu.

I typically see increases around 20x at inference time with standard convolutional neural networks on Pytorch 1.12, which is as easy as a .to("mps") for model and tensors.

I am wondering if you think it's possible to add it to the parameters of the CNN encoder for duplicate removal.
@tanujjain
Copy link
Collaborator
tanujjain commented Mar 9, 2023 •
edited
Loading

The current convention to select between cpu and gpu is through the use of the environment variable CUDA_VISIBLE_DEVICES . Instead of explicitly passing a device selection parameter, I would like to stay with this convention.

However, it maybe possible to switch to mps by default if pytorch detects an M1 chip on the system. Need to see what kind of edge cases this might lead to and the fact that one would lose the ability to use cpu if mps is present on the system. I can pick this up somewhere next month, would be useful to add this functionality somehow.

@tanujjain tanujjain added the enhancement New feature or request label Mar 9, 2023
@tanujjain
Copy link
Collaborator

Tried adding mps but it seems some pytorch operations are not yet implemented for mps backend. As a result, CNN usage breaks. Will keep the issue open and keep checking if the corresponding operators are supported.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@CarloNicolini @tanujjain