Improvement cql stmt generation #435
Draft
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
776f81b
to
905daff
Compare
DanielHe4rt
reviewed
Oct 25, 2024
There was a problem hiding this comment.
Amazing Refactor, @CodeLieutenant! I just had a couple of questions related to the implementation itself.
Btw, since you're reorganizing the whole code would be a good thing to drop some DocBlocks around things you think that's important for people to understand before jump into the code.
Comment on lines
+32
to
+38
| sleep := time.Duration(0) | ||
|
|
||
| func NewPump(stopFlag *stop.Flag, logger *zap.Logger) chan time.Duration { | ||
| pump := make(chan time.Duration, 10000) | ||
| logger = logger.Named("Pump") | ||
| go func() { | ||
| logger.Debug("pump channel opened") | ||
| defer func() { | ||
| close(pump) | ||
| logger.Debug("pump channel closed") | ||
| }() | ||
| for !stopFlag.IsHardOrSoft() { | ||
| pump <- newHeartBeat() | ||
| } | ||
| }() | ||
| if rand.Int()%chance == 0 { | ||
| sleep = sleepDuration | ||
| } | ||
|
|
||
| return pump | ||
| pump <- sleep |
There was a problem hiding this comment.
Why it should sleep? Can you explain me exactly how it acts?
There was a problem hiding this comment.
Burst mechanism, makes a user of this, sleep after change is hit. practically if you make change = 10.
It will sleep approximatly one in 10. This is just to not overload the server
Statement generation currently does not work as intended, gemini generates values for primary and clustering keys, that are completply invalid. The are not just off by value, but also off by type, e.g. generates `text` for `timeuuid`. This is the first step in resolving this issue, by refactoring how, when and where statements are generated. Signed-off-by: Dusan Malusev <dusan.malusev@scylladb.com>
1. Refactoring the Statement Generators into generators package 2. Each table gets it's own random number for validation and generation 3. Warmup consolidated into Job, runs with its own validator and statement generator -> propably can be removed, as Warmup is just a mutation without DDL and deletes Future Plans 1. Remove stopFlag as context.Context can do the same, hard kill is to CANCEL the Global Parent, and soft kill is cancelation of the current context 2. Generators Refactoring Signed-off-by: Dusan Malusev <dusan.malusev@scylladb.com>
context.Context Go has built it mehanizm to stop the execution of the long running process, like goroutines and network requests, and many libraries using inside gemini support it, lets just make a heavy usage of it, cause we can. Signed-off-by: Dusan Malusev <dusan.malusev@scylladb.com>
bcf3e1f
to
185729d
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
statement generator -> propably can be removed, as Warmup is just a
mutation without DDL and deletes
Future Plans
to CANCEL the Global Parent, and soft kill is cancelation of the
current context