[go: up one dir, main page]
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fixes numerous issues for benchmarking #93

Merged
merged 2 commits into from
Mar 3, 2023
Merged

Fixes numerous issues for benchmarking #93

merged 2 commits into from
Mar 3, 2023

Conversation

kernhanda
Copy link
Contributor
  • Batch size was used in place of number of iterations per batch size when averaging batch size results.
  • C++ benchmarking was developed on top of CallableFunc, which was designed for devices where multiple input sets were not needed. C++ benchmarking on the CPU, at least, needs these multiple input sets. This change fixes this issue.
  • Number of iterations per batch was added as a command line argument
  • CallableFunc instances for devices that need cache flushing will have extra data generated for them

@kernhanda
Fixes numerous issues for benchmarking
722842f 
* Batch size was used in place of number of iterations per batch size when averaging batch size results.
* C++ benchmarking was developed on top of CallableFunc, which was designed for devices where multiple input sets were not needed. C++ benchmarking on the CPU, at least, needs these multiple input sets. This change fixes this issue.
masonremy
masonremy approved these changes Mar 2, 2023
View reviewed changes
hatlib/host_loader.py Outdated Show resolved Hide resolved
@kernhanda kernhanda merged commit 18bc07a into main Mar 3, 2023
@kernhanda kernhanda deleted the benchmarking_fixes branch March 3, 2023 03:28
CaptnJackSparrow
CaptnJackSparrow reviewed Mar 3, 2023
View reviewed changes
hatlib/cuda_loader.py
batch_timings_ms: List[float] = []
iterations = 1
min_time_in_ms = min_time_in_sec * 1000
while sum(batch_timings_ms) < min_time_in_ms or len(batch_timings_ms) >= batch_size:
Copy link
Contributor
@CaptnJackSparrow CaptnJackSparrow Mar 3, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this be "len(batch_timings_ms) < batch_size" instead?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, you're right. good catch. fixed in #95.

hatlib/rocm_loader.py Show resolved Hide resolved
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@CaptnJackSparrow CaptnJackSparrow CaptnJackSparrow left review comments

@masonremy masonremy masonremy approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@kernhanda @masonremy @CaptnJackSparrow