-
Notifications
You must be signed in to change notification settings - Fork 4.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Setting org.gradle.jvmargs
property can lead to "Daemon disappeared" failures
#19750
Comments
One idea that's come up before is to break out some of the common things that people want to set, so instead of changing |
Sure. I think better modelling can certainly help. Perhaps combined with a big warning if a user doesn't set certain important values when they set Note that we already have set of JVM arg defaults that we always set, together with some defaults that we only set if |
Thank you for raising this and articulating so clearly, including linking to the Gradle defaults which are lost with user overrides. This likely explains a lot of weirdness since we upgraded GoCD's Gradle to run on Java 17, since this required us to add a custom Little did I know, I was overriding a whole lot of sensible limits/defaults on Gradle. Fingers crossed that we go back to stability after reinstating these! Have been tearing my hair out a bit, so hope is good enough for now! |
… overridden According to gradle/gradle#19750 as soon as we set the `--add-opens` here, we lost the constraints on JVM size and metaspace that Gradle normally sets. Oops. This possibly explains build instability (especially on Windows) and having to constrain our number of parallel workers in order to avoid our build containers exhausting memory and dying or getting killed by the container runtime.
… overridden According to gradle/gradle#19750 as soon as we set the `--add-opens` here, we lost the constraints on JVM size and metaspace that Gradle normally sets. Oops. This possibly explains build instability (especially on Windows) and having to constrain our number of parallel workers in order to avoid our build containers exhausting memory and dying or getting killed by the container runtime.
Due to gradle/gradle#19750 Gradle doesn't set a default value for MaxMetaspaceSize when org.gradle.jvmargs is set. This sets it explicitly to what the default value is define in https://github.com/gradle/gradle/blob/6ccf8a2cc74c8060992bd36189dec929143caed7/subprojects/launcher/src/main/java/org/gradle/launcher/daemon/configuration/DaemonParameters.java#L39
## Proposed Changes Due to gradle/gradle#19750 Gradle doesn't set a default value for MaxMetaspaceSize when org.gradle.jvmargs is set and the JVM 8 default is unbounded. This sets it explicitly to the default value defined in https://github.com/gradle/gradle/blob/6ccf8a2cc74c8060992bd36189dec929143caed7/subprojects/launcher/src/main/java/org/gradle/launcher/daemon/configuration/DaemonParameters.java#L39 ## Testing Test: None ## Issues Fixed Daemon disappearing in CI This is an imported pull request from #331. Resolves #331 Github-Pr-Head-Sha: 9af0a74 GitOrigin-RevId: 5ec9a52 Change-Id: I669a8761fddee8fe4b1fb0c6e698256bd907612d
Previously, setting any value for `org.gradle.jvmargs` caused all default settings to be lost. This often resulted in important defaults like `-XXMaxMetaspaceSize` being omitted when a user attempted to provide more memory to a build process. With this change, default jvmargs will be retained unless specifically overridden by a user-supplied argument. One exception is that setting either -Xmx or -Xms will cause the default heap size settings to be omitted, preventing user-supplied values from conflicting with default values (like having a min heap larger than max heap). Fixes #19750
Previously, setting any value for `org.gradle.jvmargs` caused all default settings to be lost. This often resulted in important defaults like `-XXMaxMetaspaceSize` being omitted when a user attempted to provide more memory to a build process. With this change, default jvmargs will be retained unless specifically overridden by a user-supplied argument. One exception is that setting either -Xmx or -Xms will cause the default heap size settings to be omitted, preventing user-supplied values from conflicting with default values (like having a min heap larger than max heap). Fixes #19750
Previously, setting any value for `org.gradle.jvmargs` caused all default settings to be lost. This often resulted in important defaults like `-XXMaxMetaspaceSize` being omitted when a user attempted to provide more memory to a build process. With this change, default jvmargs will be retained unless specifically overridden by a user-supplied argument. One exception is that setting either -Xmx or -Xms will cause the default heap size settings to be omitted, preventing user-supplied values from conflicting with default values (like having a min heap larger than max heap). Fixes #19750
Previously, setting any value for `org.gradle.jvmargs` caused all default settings to be lost. This often resulted in important defaults like `-XXMaxMetaspaceSize` being omitted when a user attempted to provide more memory to a build process. With this change, default jvmargs will be retained unless specifically overridden by a user-supplied argument. One exception is that setting either -Xmx or -Xms will cause the default heap size settings to be omitted, preventing user-supplied values from conflicting with default values (like having a min heap larger than max heap). Fixes #19750
Previously, setting any value for `org.gradle.jvmargs` caused all default settings to be lost. This often resulted in important defaults like `-XXMaxMetaspaceSize` being omitted when a user attempted to provide more memory to a build process. With this change, default jvmargs will be retained unless specifically overridden by a user-supplied argument. One exception is that setting either -Xmx or -Xms will cause the default heap size settings to be omitted, preventing user-supplied values from conflicting with default values (like having a min heap larger than max heap). Fixes #19750
According to gradle/gradle#19750 setting the org.gradle.jvmargs property can lead to "Daemon disappeared" failures by removing the default values. Also use -XX:+UseParallelGC to improve build performance.
According to gradle/gradle#19750 setting the org.gradle.jvmargs property can lead to "Daemon disappeared" failures by removing the default values. Also use -XX:+UseParallelGC to improve build performance.
According to gradle/gradle#19750 setting the org.gradle.jvmargs property can lead to "Daemon disappeared" failures by removing the default values. Also use -XX:+UseParallelGC to improve build performance.
According to gradle/gradle#19750 setting the org.gradle.jvmargs property can lead to "Daemon disappeared" failures by removing the default values. Also use -XX:+UseParallelGC to improve build performance.
According to gradle/gradle#19750 setting the org.gradle.jvmargs property can lead to "Daemon disappeared" failures by removing the default values. Also use -XX:+UseParallelGC to improve build performance.
According to gradle/gradle#19750 setting the org.gradle.jvmargs property can lead to "Daemon disappeared" failures by removing the default values. Also use -XX:+UseParallelGC to improve build performance.
The last status of this was that changing this would be a breaking change: |
…aspaceSize` (#143085) Re-sets two jvmargs that were getting cleared because we set a value for `-Xmx`. Could help with #142957. Copied from comment here #142957: >Two random things I ran into while looking into this that might help: > >1. Gradle has defaults for a couple of the jvmargs, and setting any one of them clears those defaults for the others (bug here gradle/gradle#19750). This can cause the "Gradle daemon to consume more and more native memory until it crashes", though the bug typically has a different associated error. It seems worth it to re-set those defaults. >2. There is a property we can set that will give us a heap dump on OOM ([-XX:HeapDumpOnOutOfMemoryError](https://docs.oracle.com/javase/8/docs/technotes/guides/troubleshoot/clopts001.html)) Mostly just a find and replace from `find . -name gradle.properties -exec sed -i '' 's/\-Xmx4G/-Xmx4G\ \-XX:MaxMetaspaceSize=2G\ \-XX:+HeapDumpOnOutOfMemoryError/g' {} \;`, with the templates and the one test that writes from a string replaced by hand. I didn't set a value for `MaxMetaspaceSize` in the template files because I want to make sure this value doesn't cause problems in ci first (changes to the templates are essentially un-revertable for those who `flutter create` while the changes exist).
Signed-off-by: Justin Florentine <justin+github@florentine.us>
Signed-off-by: Justin Florentine <justin+github@florentine.us>
Signed-off-by: Justin Florentine <justin+github@florentine.us>
Signed-off-by: Justin Florentine <justin+github@florentine.us>
Signed-off-by: Justin Florentine <justin+github@florentine.us>
Signed-off-by: Justin Florentine <justin+github@florentine.us>
Signed-off-by: Justin Florentine <justin+github@florentine.us>
See gradle/gradle#19750 and https://developer.android.com/build/optimize-your-build#experiment-with-the-jvm-parallel-garbage-collector for more info. This should enable the parallel GC on Java 9+, as well as other defaults that Gradle would set otherwise.
- Add `-XX:MaxMetaspaceSize` (gradle/gradle#19750) - Remove `-PpreDexEnable` as it's deprecated and no longer used. - Remove `--max-workers` as gradle will automatically use the optimal amount.
* Try to workaround OOM errors and the runner dying * Some more tweaks: - Add `-XX:MaxMetaspaceSize` (gradle/gradle#19750) - Remove `-PpreDexEnable` as it's deprecated and no longer used. - Remove `--max-workers` as gradle will automatically use the optimal amount. * Remove `--max-workers` in recording screenshots too
org.gradle.jvmargs=-Xmx1536m -XX:+UseParallelGC |
Ok |
When investigating reports of "daemon disappeared" in GitHub Actions, and a feature request to address one cause, it was found that the all of our DEFAULT_JVM8_ARGS are being lost as soon as any value is set for
org.gradle.jvmargs
.When
org.gradle.jvmargs
is not set by the user, then the daemon will start with default values specified here. But setting a value oforg.gradle.jvmargs
will overwrite these defaults, resulting in JDK defaults for-Xmx
,-Xms
etc.One of the key defaults set by Gradle is
-XX:MaxMetaspaceSize=256m
, which is required because this value is unbounded on the JDK by default, allowing the Gradle daemon to consume more and more native memory until it crashes.This means that a user setting
org.gradle.jvmargs=-Xmx4g
can result in daemon crashes, unless the user knows that they must also add a-XX:MaxMetaspaceSize
setting.In a similar way, the
-XX:+HeapDumpOnOutOfMemoryError
default is lost when a user specifiesorg.gradle.jvmargs
.Expected Behavior
Important JVM defaults are not lost when a user sets unrelated JVM arguments.
Current Behavior
Users must be aware of all important JVM argument defaults, and include all of these when setting any values for
org.gradle.jvmargs
.Context
This was discovered when investigating "daemon disappeared" issues in GitHub Actions workflows.
Here is an example of a build with such a failure: https://github.com/androidx/androidx/runs/4971463356?check_suite_focus=true#step:10:3845. No build scan was captured due to the nature of the failure.
Daemon log file for that CI run:
gradle-daemon-logs_room.zip
The text was updated successfully, but these errors were encountered: