[go: up one dir, main page]
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

add support for multiprocessing to spacy pipelines #1440

Open
skunkwerk opened this issue Aug 20, 2024 · 1 comment
Open

add support for multiprocessing to spacy pipelines #1440

skunkwerk opened this issue Aug 20, 2024 · 1 comment

Comments

@skunkwerk
Copy link

Is your feature request related to a problem? Please describe.
The BatchAnalyzerEngine which calls the Spacy engine's process_batch method doesn't seem to pass through any kwargs for setting the n_process or batch size.

Describe the solution you'd like
We should update process_batch to pass through kwargs to spacy's pipe method.

Describe alternatives you've considered
Haven't considered any.

Additional context
N/A
@omri374
Copy link
Contributor
omri374 commented Aug 22, 2024

Hi, this is a great suggestion. In fact, we had a PR that was started some time ago but wasn't completed. If you're interested in contributing, that'd be great. #883

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@skunkwerk @omri374