# Chromium's Java Toolchain This doc aims to describe the Chrome build process that takes a set of `.java` files and turns them into a `classes.dex` file. [TOC] ## Core GN Target Types The following have `supports_android` and `requires_android` set to false by default: * `java_library()`: Compiles `.java` -> `.jar` * `java_prebuilt()`: Imports a prebuilt `.jar` file. The following have `supports_android` and `requires_android` set to true. They also have a default `jar_excluded_patterns` set (more on that later): * `android_library()` * `android_java_prebuilt()` All target names must end with "_java" so that the build system can distinguish them from non-java targets (or [other variations](https://cs.chromium.org/chromium/src/build/config/android/internal_rules.gni?rcl=ec2c17d7b4e424e060c3c7972842af87343526a1&l=20)). Most targets produce two separate `.jar` files: * Device `.jar`: Used to produce `.dex.jar`, which is used on-device. * Host `.jar`: For use on the host machine (`junit_binary` / `java_binary`). * Host `.jar` files live in `lib.java/` so that they are archived in builder/tester bots (which do not archive `obj/`). ## From Source to Final Dex ### Step 1: Create interface .jar with turbine or ijar For prebuilt `.jar` files, use [//third_party/ijar] to create interface `.jar` from prebuilt `.jar`. For non-prebuilt targets, use [//third_party/turbine] to create interface `.jar` from `.java` source files. Turbine is much faster than javac, and so enables full compilation to happen more concurrently. What are interface jars?: * The contain `.class` files with all non-public symbols and function bodies removed. * Dependant targets use interface `.jar` files to skip having to be rebuilt when only private implementation details change. [//third_party/ijar]: /third_party/ijar/README.chromium [//third_party/turbine]: /third_party/turbine/README.chromium ### Step 2a: Compile with javac This step is the only step that does not apply to prebuilt targets. * All `.java` files in a target are compiled by `javac` into `.class` files. * This includes `.java` files that live within `.srcjar` files, referenced through `srcjar_deps`. * The `classpath` used when compiling a target is comprised of `.jar` files of its deps. * When deps are library targets, the Step 1 `.jar` file is used. * When deps are prebuilt targets, the original `.jar` file is used. * All `.jar` processing done in subsequent steps does not impact compilation classpath. * `.class` files are zipped into an output `.jar` file. * There is **no support** for incremental compilation at this level. * If one source file changes within a library, then the entire library is recompiled. * Prefer smaller targets to avoid slow compiles. ### Step 2b: Compile with ErrorProne This step can be disabled via GN arg: `use_errorprone_java_compiler = false` * Concurrently with step 1a: [ErrorProne] compiles java files and checks for bug patterns, including some [custom to Chromium][ep_plugins]. * ErrorProne used to replace step 1a, but was changed to a concurrent step after being identified as being slower. [ErrorProne]: https://errorprone.info/ [ep_plugins]: /tools/android/errorprone_plugin/ ### Step 3: Desugaring (Device .jar Only) This step happens only when targets have `supports_android = true`. It is not applied to `.jar` files used by `junit_binary`. * `//third_party/bazel/desugar` converts certain Java 8 constructs, such as lambdas and default interface methods, into constructs that are compatible with Java 7. ### Step 4: Instrumenting (Device .jar Only) This step happens only when this GN arg is set: `use_jacoco_coverage = true` * [Jacoco] adds instrumentation hooks to methods. [Jacoco]: https://www.eclemma.org/jacoco/ ### Step 5: Filtering This step happens only when targets that have `jar_excluded_patterns` or `jar_included_patterns` set (e.g. all `android_` targets). * Remove `.class` files that match the filters from the `.jar`. These `.class` files are generally those that are re-created with different implementations further on in the build process. * E.g.: `R.class` files - a part of [Android Resources]. * E.g.: `GEN_JNI.class` - a part of our [JNI] glue. * E.g.: `AppHooksImpl.class` - how `chrome_java` wires up different implementations for [non-public builds][apphooks]. [JNI]: /base/android/jni_generator/README.md [Android Resources]: life_of_a_resource.md [apphooks]: /chrome/android/java/src/org/chromium/chrome/browser/AppHooksImpl.java ### Step 6: Per-Library Dexing This step happens only when targets have `supports_android = true`. * [d8] converts `.jar` files containing `.class` files into `.dex.jar` files containing `classes.dex` files. * Dexing is incremental - it will reuse dex'ed classes from a previous build if the corresponding `.class` file is unchanged. * These per-library `.dex.jar` files are used directly by [incremental install], and are inputs to the Apk step when `enable_proguard = false`. * Even when `is_java_debug = false`, many apk targets do not enable ProGuard (e.g. unit tests). [d8]: https://developer.android.com/studio/command-line/d8 [incremental install]: /build/android/incremental_install/README.md ### Step 7: Apk / Bundle Module Compile * Each `android_apk` and `android_bundle_module` template has a nested `java_library` target. The nested library includes final copies of files stripped out by prior filtering steps. These files include: * Final `R.java` files, created by `compile_resources.py`. * Final `GEN_JNI.java` for [JNI glue]. * `BuildConfig.java` and `NativeLibraries.java` (//base dependencies). [JNI glue]: /base/android/jni_generator/README.md ### Step 8: Final Dexing This step is skipped when building using [Incremental Install]. When `is_java_debug = true`: * [d8] merges all library `.dex.jar` files into a final `.mergeddex.jar`. When `is_java_debug = false`: * [R8] performs whole-program optimization on all library `lib.java` `.jar` files and outputs a final `.r8dex.jar`. * For App Bundles, R8 creates a `.r8dex.jar` for each module. [Incremental Install]: /build/android/incremental_install/README.md [R8]: https://r8.googlesource.com/r8 ## Test APKs with apk_under_test Test APKs are normal APKs that contain an `` tag within their `AndroidManifest.xml`. If this tag specifies an `android:targetPackage` different from itself, then Android will add that package's `classes.dex` to the test APK's Java classpath when run. In GN, you can enable this behavior using the `apk_under_test` parameter on `instrumentation_test_apk` targets. Using it is discouraged if APKs have `proguard_enabled=true`. ### Difference in Final Dex When `enable_proguard=false`: * Any library depended on by the test APK that is also depended on by the apk-under-test is excluded from the test APK's final dex step. When `enable_proguard=true`: * Test APKs cannot make use of the apk-under-test's dex because only symbols explicitly kept by `-keep` directives are guaranteed to exist after ProGuarding. As a work-around, test APKs include all of the apk-under-test's libraries directly in its own final dex such that the under-test apk's Java code is never used (because it is entirely shadowed by the test apk's dex). * We've found this configuration to be fragile, and are trying to [move away from it](https://bugs.chromium.org/p/chromium/issues/detail?id=890452). ### Difference in GEN_JNI.java * Calling native methods using [JNI glue] requires that a `GEN_JNI.java` class be generated that contains all native methods for an APK. There cannot be conflicting `GEN_JNI` classes in both the test apk and the apk-under-test, so only the apk-under-test has one generated for it. As a result this, instrumentation test APKs that use apk-under-test cannot use native methods that aren't already part of the apk-under-test. ## How to Generate Java Source Code There are two ways to go about generating source files: Annotation Processors and custom build steps. ### Annotation Processors * These are run by `javac` as part of the compile step. * They **cannot** modify the source files that they apply to. They can only generate new sources. * Use these when: * an existing Annotation Processor does what you want (E.g. Dagger, AutoService, etc.), or * you need to understand Java types to do generation. ### Custom Build Steps * These use discrete build actions to generate source files. * Some generate `.java` directly, but most generate a zip file of sources (called a `.srcjar`) to simplify the number of inputs / outputs. * Examples of existing templates: * `jinja_template`: Generates source files using [Jinja]. * `java_cpp_template`: Generates source files using the C preprocessor. * `java_cpp_enum`: Generates `@IntDef`s based on enums within `.h` files. * `java_cpp_strings`: Generates String constants based on strings defined in `.cc` files. * Custom build steps are preferred over Annotation Processors because they are generally easier to understand, and can run in parallel with other steps (rather than being tied to compiles). [Jinja]: https://palletsprojects.com/p/jinja/ ## Static Analysis & Code Checks We use several tools for static analysis. ### [ErrorProne](https://errorprone.info/) * Runs as part of normal compilation. Controlled by GN arg: `use_errorprone_java_compiler`. * Most useful check: * Enforcement of `@GuardedBy` annotations. * List of enabled / disabled checks exists [within compile_java.py](https://cs.chromium.org/chromium/src/build/android/gyp/compile_java.py?l=30) * Many checks are currently disabled because there is work involved in fixing violations they introduce. Please help! * Custom checks for Chrome: * [//tools/android/errorprone_plugin/src/org/chromium/tools/errorprone/plugin/](/tools/android/errorprone_plugin/src/org/chromium/tools/errorprone/plugin/) * Use ErrorProne checks when you need something more sophisticated than pattern matching. * Checks run on the entire codebase, not only on changed lines. * Does not run when `chromium_code = false` (e.g. for //third_party). ### [Android Lint](https://developer.android.com/studio/write/lint) * Runs as part of normal compilation. Controlled by GN arg: `disable_android_lint` * Most useful check: * Enforcing `@TargetApi` annotations (ensure you don't call a function that does not exist on all versions of Android unless guarded by an version check). * List of disabled checks: * [//build/android/lint/suppressions.xml](/build/android/lint/suppressions.xml) * Custom lint checks [are possible](lint_plugins), but we don't have any. * Checks run on the entire codebase, not only on changed lines. * Does not run when `chromium_code = false` (e.g. for //third_party). [lint_plugins]: http://tools.android.com/tips/lint-custom-rules ### [Bytecode Processor](/build/android/bytecode/) * Performs a single check: * That target `deps` are not missing any entries. * In other words: Enforces that targets do not rely on indirect dependencies to populate their classpath. * Checks run on the entire codebase, not only on changed lines. * This is the only static analysis that runs on prebuilt .jar files. * The same tool is also used for [bytecode rewriting](/docs/ui/android/bytecode_rewriting.md). ### [PRESUBMIT.py](/PRESUBMIT.py): * Checks for banned patterns via `_BANNED_JAVA_FUNCTIONS`. * (These should likely be moved to checkstyle). * Checks for a random set of things in `ChecksAndroidSpecificOnUpload()`. * Including running Checkstyle. * (Some of these other checks should likely also be moved to checkstyle). * Checks run only on changed lines. ### [Checkstyle](https://checkstyle.sourceforge.io/) * Checks Java style rules that are not covered by clang-format. * E.g.: Unused imports and naming conventions. * Allows custom checks to be added via XML. Here [is ours]. * Preferred over adding checks directly in PRESUBMIT.py because the tool understands `@SuppressWarnings` annotations. * Checks run only on changed lines. [is ours]: /tools/android/checkstyle/chromium-style-5.0.xml ### [clang-format](https://clang.llvm.org/docs/ClangFormat.html) * Formats `.java` files via `git cl format`. * Can be toggle on/off with code comments. ```java // clang-format off ... non-formatted code here ... // clang-format on ``` * Does not work great for multiple annotations or on some lambda expressions, but is generally agreed it is better than not having it at all.