Sid Patil

code
search
dark_mode
person_outline

Chasing Monet inside the Android framework

calendar_today favorite_border 2.1K views hourglass_empty 15 min read Android
bookmark_border Featured in AndroidWeekly.net Issue #516

Android 13 Tiramisu is a couple of months away from a full stable release and only last year did Google introduce Material You with Android 12/12L, changing their material design guidelines for the third time.

Use wallpaper generated system colors from Android 12 and Material You to repaint your apps to the users taste by using the material design 3 library and Jetpack compose

Material You graphic from Material.io

Material You is the latest iteration of Google’s design system for Android, it focuses on providing a personalized feel to the user by using system-wide dynamic theming, generating richer tonal palettes, and improving the overall user experience between the Android system UI and third-party apps.


In this article, we will explore the Android framework and changes introduced with Monet to understand how dynamic theming works under the hood and fuels the new Material You design.

Table of Content


What is Monet?

Android 12/12L adds support for dynamic themes a.k.a. Monet. Monet as a term is a codename for a subset of changes within the Android framework, primarily in the System-UI which allows dynamic theming.

Monet is most likely named after the French painter Claude Monet.

With Monet, Tiramisu and UpsideDownCake, it’s good to be back in the fancy Android naming world!

Parts of the logic that encompass Monet were first added in Android 12 and were only available on Pixel devices (6 and 6 Pro), with the 12L release we see the remaining components added to AOSP.

So what exactly is dynamic theming?

Use wallpaper generated system colors from Android 12 and Material You to repaint your apps to the users taste by using the material design 3 library and Jetpack compose

A gif from Material Theme Builder

Let’s say if you set a new wallpaper on Android 12/12L, the system contains a color extraction logic within the framework which will generate tonal palettes and swatches that you can avail for your apps, thus enabling a more uniform look between the Android system UI and your application.


Exploring the Android framework

With a bit of prelude on Material You, we can talk about changes in the Android framework with Monet and understand the entire pipeline where a user selects a wallpaper and the system dynamically adapts to new colors.

While we take a walkthrough of System UI and the logic that forms Monet, I will also slightly explore some other interesting aspects of the framework.

The outline of the logic is simple

  • Events when the user sets a new wallpaper
  • System reacting to the change
  • Color extraction, palette generation
  • Refreshing the system context with new colors

System UI & Monet

Changes made in the framework that form what we call dynamic system theming are largely a part of System-UI with other components from the graphic package which enable major portions of color extraction logic.

And like all good things, Monet is hidden behind a feature flag.

frameworks/base/packages/SystemUI/src/com/android/systemui/theme/ThemeOverlayController.java

    mIsMonetEnabled = featureFlags.isMonetEnabled();

As we continue our deep dive we will take a look at the following parts of the Android framework

  1. ThemeOverlayController.java
  2. WallpaperManagerService.java
  3. WallpaperColors.java

Theme Overlay Controller

The ThemeOverlayController.java from the System UI package of the framework contains the bulk of the logic for determining events that affect dynamic theming. It prepares the theme colors initially when the system boots.

frameworks/base/core/java/android/app/WallpaperManager.java

    public interface OnColorsChangedListener {
        void onColorsChanged(WallpaperColors colors, int which);
        void onColorsChanged(WallpaperColors colors, int which, int userId)
    }

First things first, the color change listener of the WallpaperManager is a simple interface that contains two overloaded functions to notify that the colors have changed. There could be two or more user profiles active on the device and each of the users could have a different wallpaper set, so the framework needs to take into consideration a multi-user situation and only update the colors specific to the user.

The start() method of the overlay controller, first upon device boot, extracts the latest colors from the wallpaper via the WallpaperManagerService and applies them to the System-UI.

frameworks/base/packages/SystemUI/src/com/android/systemui/theme/ThemeOverlayController.java

    @Override
    public void start() {
        ...

        // Do nothing if monet is disabled
        if (!mIsMonetEnabled) {
            return;
        }
        ...

        // Upon boot, make sure we have the most up to date colors
        Runnable updateColors = () -> {
            WallpaperColors systemColor = mWallpaperManager.getWallpaperColors(
                    getLatestWallpaperType(mUserTracker.getUserId()));
            Runnable applyColors = () -> {
                if (DEBUG) Log.d(TAG, "Boot colors: " + systemColor);
                mCurrentColors.put(mUserTracker.getUserId(), systemColor);
                reevaluateSystemTheme(false /* forceReload */);
            };
            if (mDeviceProvisionedController.isCurrentUserSetup()) {
                mMainExecutor.execute(applyColors);
            } else {
                applyColors.run();
            }
        };

A quick tip: You can click on the file paths mentioned above code snippets and view the entire source hosted on the Android Super project.

Two important functions from the above snippet are getWallpaperColors() and reevaluateSystemTheme(), Let’s explore each of them in detail.

Wallpaper Manager & Wallpaper Manager Service

We will now need to look into how exactly the system listens to changes in the wallpaper and changes in the dynamic colors. We start with two important components of this logic, one is the WallpaperManager and the second is the WallpaperManagerService.

frameworks/base/core/java/android/app/WallpaperManager.java

Aptly named, the wallpaper manager is responsible for connecting to the system wallpaper service and utilizes the service to set up observers for changes in the wallpaper and its colors and react to its events.

Step 1: Listening to Changes in Wallpaper

The service contains a wallpaper observer, which simply checks if the wallpaper file has changed and triggers sub-routines for color extraction and theme re-evaluation.

frameworks/base/services/core/java/com/android/server/wallpaper/WallpaperManagerService.java

    class WallpaperObserver extends FileObserver {
        ...
        final boolean sysWallpaperChanged = (mWallpaperFile.equals(changedFile));
        final boolean lockWallpaperChanged = (mWallpaperLockFile.equals(changedFile));
        ...

        // Based on a bunch of conditionals some of the below functions are called
        notifyLockWallpaperChanged();
        notifyWallpaperColorsChanged(wallpaper, FLAG_LOCK);
        notifyWallpaperChanged(wallpaper);
    }

If we look into one of the above functions and its implementation we’ll see that the logic takes into consideration a multi-display scenario. For example, foldable devices or a multi-display device that contains two or more displays. Hence we could have a live wallpaper that supports multiple displays and can contain separate logic for per display surface rendering of the wallpaper or just two separate wallpapers for the home screen and the lock screen.

The color extraction logic needs to be aware of this, hence we find a DisplayConnector which is displayId specific. Our notifier logic runs through a custom iterator for display connectors and notifies color change per connection.

frameworks/base/services/core/java/com/android/server/wallpaper/WallpaperManagerService.java

    private void notifyWallpaperColorsChanged(@NonNull WallpaperData wallpaper, int which) {
        ...
        wallpaper.connection.forEachDisplayConnector(connector -> {
            notifyWallpaperColorsChangedOnDisplay(wallpaper, which, connector.mDisplayId);
        });
        ...
    }

Step 2: Extracting Wallpaper Colors

The getWallpaperColors() function extracts user-specific wallpaper data and determines if colors can be extracted or not.

frameworks/base/services/core/java/com/android/server/wallpaper/WallpaperManagerService.java

    public WallpaperColors getWallpaperColors(int which, int userId, int displayId) throws RemoteException {
        ...
        shouldExtract = wallpaperData.primaryColors == null;
        if (shouldExtract) {
            extractColors(wallpaperData);
        }
        return wallpaperData.primaryColors;
        
    }

The extractColors() function of the service is responsible for taking the wallpaper data, reading the user-specific wallpaper file, and creating a downscaled bitmap of it before color extraction.

Variance in Color Extraction Algorithms

The first step of color extraction that Monet relies on is the calculation of the wallpaper bitmap size. If the bitmap size is greater than the hardcoded limit, a downscale of the bitmap is required before color swatches can be extracted. We don’t want to process an image larger than it needs to be, possibly saving some computation time and resource while still maintaining the effectiveness of the color extraction algorithm from the image sample.

frameworks/base/core/java/android/app/WallpaperColors.java

    private static final int MAX_BITMAP_SIZE = 112;
    private static final int MAX_WALLPAPER_EXTRACTION_AREA = MAX_BITMAP_SIZE * MAX_BITMAP_SIZE;

    final int bitmapArea = bitmap.getWidth() * bitmap.getHeight();
    if (bitmapArea > MAX_WALLPAPER_EXTRACTION_AREA) {
        Size optimalSize = calculateOptimalSize(bitmap.getWidth(), bitmap.getHeight());
        bitmap = Bitmap.createScaledBitmap(bitmap, optimalSize.getWidth(),
                optimalSize.getHeight(), true /* filter */);
    }

Post resizing of the bitmap (if required), we move on to extracting the palette from the bitmap. The extraction logic takes into account if the device is a low RAM device based on build config and uses appropriate quantization logic to extract the palette for efficient performance.

frameworks/base/core/java/android/app/WallpaperColors.java

    final Palette palette;
    if (ActivityManager.isLowRamDeviceStatic()) {
        palette = Palette
                .from(bitmap, new VariationalKMeansQuantizer())
                .maximumColorCount(5)
                .resizeBitmapArea(MAX_WALLPAPER_EXTRACTION_AREA)
                .generate();
    } else {
        palette = Palette
                .from(bitmap, new CelebiQuantizer())
                .maximumColorCount(256)
                .resizeBitmapArea(MAX_WALLPAPER_EXTRACTION_AREA)
                .generate();
    }

Makefiles define if the device is low ram!

If you have ever built Android from scratch and compiled AOSP, you probably know that we use .mk or makefiles to instruct the build system/toolchain about the build configuration of the device you are trying to compile Android for.

The value indicating if device is a Low RAM device is a hard coded static value supplied from build makefile for the device. Take a look at the build file for Pixel 4a for reference:

/device/google/sunfish/device-common.mk

    # Set lmkd options
PRODUCT_PRODUCT_PROPERTIES += \
	ro.config.low_ram = false \
	ro.lmk.log_stats = true \

frameworks/base/core/java/com/android/internal/os/RoSystemProperties.java

    public static final boolean CONFIG_LOW_RAM =
            SystemProperties.getBoolean("ro.config.low_ram", false);

This value is provided by the ActivityManager.

frameworks/base/core/java/android/app/ActivityManager.java

    /**
     * Returns true if this is a low-RAM device.  Exactly whether a device is low-RAM
     * is ultimately up to the device configuration, but currently it generally means
     * something with 1GB or less of RAM.  This is mostly intended to be used by apps
     * to determine whether they should turn off certain features that require more RAM.
     */
    public boolean isLowRamDevice() {
        return isLowRamDeviceStatic();
    }

The key takeaway from the above is that the Android system may use different logic based on system resource constraints, in our case the color extraction algorithm.

K-Means & Celebi Quantizers

To understand it simply, a quantizer/quantization algorithm or a clustering algorithm is used to extract dominant values from a set of values. For our color extraction logic, this set of values comes from our image which is the wallpaper.

This wallpaper, (or any image for that matter) consists of pixels of various colors arranged in specific ways, that together constitute an image.

Our color quantizer thus needs to identify dominant portions of these sets of pixels that share the same color and cluster them together so that we can extract meaningful information from the image.

Android framework relies on two algorithms

  • Variational K-means clustering
  • Celebi K-means clustering
Use wallpaper generated system colors from Android 12 and Material You to repaint your apps to the users taste by using the material design 3 library and Jetpack compose

Finding dominant colors using K-means by Ailephant.com

Read more about the Color quantization K means algorithm used in Android 12.

The K Means algorithm largely works on identifying clusters in an image but is not the most effective when extracting colors.

M Emre Celebi’s paper improved the performance and effectiveness of the K-Means quantizer in their paper published in 2011.

So if you are using any device that has over 1 gigabyte of RAM and it is running Android 12/12L or Tiramisu, Monet will use the Celebi quantizer to extract colors from your wallpaper.

Both the above quantizers are a part of AOSP and can be found at

frameworks/base/core/java/com/android/internal/graphics/palette/CelebiQuantizer.java

frameworks/base/core/java/com/android/internal/graphics/palette/VariationalKMeansQuantizer.java

Palettes and Swatches

Let us now understand what a swatch is and what a palette is.

A swatch consists of a color. Many swatches form a palette. The population of a swatch is the number of pixels the color of the swatch represents. The higher the number of pixels of a certain color in the wallpaper, the higher the population value of the swatch.

The population of a swatch is an important factor in determining the dominant colors of the wallpaper, the higher the population of the swatch, the higher the chances that the color will become the dominant color of the theme.

frameworks/base/core/java/com/android/internal/graphics/palette/Palette.java

    public static class Swatch {
        private final Color mColor;
        private final int mPopulation;
        ...
    }

    public final class Palette {
        private final List<Swatch> mSwatches;
        private final Swatch mDominantSwatch;
        ...
    }

Step 3: Identifying dominant colors

Once our wallpaper is run through the quantization logic, we end up with a list of swatches or colors. This list of swatches now needs to be sorted by their population value to identify the dominant colors from the image.

The final container is a HashMap key-value pair which consists of swatch color and population count, encapsulated within a WallpaperColors object before forwarding it back to System-UIs theme overlay controller.

frameworks/base/core/java/android/app/WallpaperColors.java

    final ArrayList<Palette.Swatch> swatches = new ArrayList<>(palette.getSwatches());
    swatches.sort((a, b) -> b.getPopulation() - a.getPopulation());

    final int swatchesSize = swatches.size();

    final Map<Integer, Integer> populationByColor = new HashMap<>();
    for (int i = 0; i < swatchesSize; i++) {
        Palette.Swatch swatch = swatches.get(i);
        int colorInt = swatch.getInt();
        populationByColor.put(colorInt, swatch.getPopulation());
    }

    int hints = calculateDarkHints(bitmap);

    return new WallpaperColors(populationByColor, HINT_FROM_BITMAP | hints);

Re-evaluating system theme

Let’s go back to our Theme Overlay Controller, which registered a wallpaper change listener in its start() method. Every time the user changes a wallpaper, the intent filter filters the wallpaper change intent and passes the newly calculated colors from our wallpaper manager service through the wallpaper change listener.

The broadcast receiver within the theme controller listens to the Intent.ACTION_WALLPAPER_CHANGED and triggers conditional flows to re-evaluate the system theme.

The wallpaper change listener invokes reevaluateSystemTheme()

frameworks/base/packages/SystemUI/src/com/android/systemui/theme/ThemeOverlayController.java

    // Intercept wallpaper change intent and changes in system user profiles
    private final BroadcastReceiver mBroadcastReceiver = new BroadcastReceiver() {
        @Override
        public void onReceive(Context context, Intent intent) {

            boolean newWorkProfile = Intent.ACTION_MANAGED_PROFILE_ADDED.equals(intent.getAction());
            boolean isManagedProfile = mUserManager.isManagedProfile(
                    intent.getIntExtra(Intent.EXTRA_USER_HANDLE, 0));

            if (newWorkProfile) {
                    ...
                    reevaluateSystemTheme(true /* forceReload */);
            } else if (Intent.ACTION_WALLPAPER_CHANGED.equals(intent.getAction())) {
                    ...
                    mAcceptColorEvents = true;
                    Log.i(TAG, "Wallpaper changed, allowing color events again");
                    ..
            }
        }
    };

    // color change listener triggering system theme reevaluation
    mWallpaperManager.addOnColorsChangedListener((wallpaperColors, which) -> {
        ...
        reevaluateSystemTheme(false /* forceReload */);
        ...

    }, null, UserHandle.USER_ALL);


    // Re evaluation logic
    private void reevaluateSystemTheme(boolean forceReload) {
        final WallpaperColors currentColors = mCurrentColors.get(mUserTracker.getUserId());
        final int mainColor;
        final int accentCandidate;
        if (currentColors == null) {
            mainColor = Color.TRANSPARENT;
            accentCandidate = Color.TRANSPARENT;
        } else {
            mainColor = getNeutralColor(currentColors);
            accentCandidate = getAccentColor(currentColors);
        }

        ...

        mMainWallpaperColor = mainColor;
        mWallpaperAccentColor = accentCandidate;

        // Extract colors from palette according to M3 specs
        if (mIsMonetEnabled) {
            mSecondaryOverlay = getOverlay(mWallpaperAccentColor, ACCENT);
            mNeutralOverlay = getOverlay(mMainWallpaperColor, NEUTRAL);
            mNeedsOverlayCreation = true;
        }

        // Overlay Manager & overlay manager service will update specific packages
        updateThemeOverlays();
    }

Monet Color Schemes

A color scheme typed entity encapsulates the colors generated by above logic from the getOverlay() method. It is the class that calculates the required color values in definition of the Material You specs, we use the WallpaperColors class that contains the extracted colors to build the final color scheme.

frameworks/base/packages/SystemUI/monet/src/com/android/systemui/monet/ColorScheme.kt

public class ColorScheme(@ColorInt seed: Int, val darkTheme: Boolean) {
    val accent1: List<Int>
    val accent2: List<Int>
    val accent3: List<Int>
    val neutral1: List<Int>
    val neutral2: List<Int>
    constructor(wallpaperColors: WallpaperColors, darkTheme: Boolean):
            this(getSeedColor(wallpaperColors), darkTheme)

    ...

To calculate the system theme colors, identify dominant swatches and build them according to the required specifications. The class identifies a seed color, maps a score to each swatch, considers hue proportions and ranks them to find the most prominent colors.

        /**
         * Filters and ranks colors from WallpaperColors.
         *
         * @param wallpaperColors Colors extracted from an image via quantization.
         * @return List of ARGB ints, ordered from highest scoring to lowest.
         */
        @JvmStatic
        fun getSeedColors(wallpaperColors: WallpaperColors): List<Int> 

Theme Overlay Applier

This last section would ask of us to step into the overlay manager service to understand how the Android system persists the new overlay information upon re evaluation with new colors. We’ll look at specific parts of this logic.

frameworks/base/packages/SystemUI/src/com/android/systemui/theme/ThemeOverlayApplier.java

    static final List<String> THEME_CATEGORIES = Lists.newArrayList(
            OVERLAY_CATEGORY_SYSTEM_PALETTE,
            OVERLAY_CATEGORY_ICON_LAUNCHER,
            OVERLAY_CATEGORY_SHAPE,
            OVERLAY_CATEGORY_FONT,
            OVERLAY_CATEGORY_ACCENT_COLOR,
            OVERLAY_CATEGORY_ICON_ANDROID,
            OVERLAY_CATEGORY_ICON_SYSUI,
            OVERLAY_CATEGORY_ICON_SETTINGS,
            OVERLAY_CATEGORY_ICON_THEME_PICKER);

    public void applyCurrentUserOverlays(
            Map<String, OverlayIdentifier> categoryToPackage,
            FabricatedOverlay[] pendingCreation,
            int currentUser,
            Set<UserHandle> managedProfiles)

We have a list of categories the applier will run through to apply the new overlay in a given sequence. A map of the category to the overlay identifier is calculated to determine which packages should be updated with the new colors.

The applier will utilize the overlay manager and the overlay manager service to update the System UI and packages.

Thus the system will reflect the new dynamically calculated colors based on the user’s wallpaper.

I would love to extend this article and talk in finer detail about how the overlay manager and overlay manager service function, doing so may run us out of the scope of this article. Drop a comment if you’d like to read more about them, perhaps in a separate article..


Material You

As a result of the above color extraction we end up with colors that have the highest population, and the most prominent colors are extracted from the wallpaper.

Monet maps the top-most colors/seed colors with the highest rank by the Material You guideline:

  1. Primary color
  2. Secondary color
  3. Tertiary color
  4. Neutral color
  5. Neutral key variant color

Five dominant colors of Material You from Material.io

13 tones in varying degrees of opacity are extracted from each of the five core colors and only selective colors are used for Material You.

Use wallpaper generated system colors from Android 12 and Material You to repaint your apps to the users taste by using the material design 3 library and Jetpack compose

Color tonality based on black and white levels from Material.io


We end up with an operating system that is truly invested in enriching the user’s experience by allowing the expression of their styles, preferences, and aesthetics. It’s perfectly called Material You.

Use wallpaper generated system colors from Android 12 and Material You to repaint your apps to the users taste by using the material design 3 library and Jetpack compose

A promo from Material.io, showcasing Monet in action with Material You

Hope you enjoyed this article and understand the framework a bit more than before, while writing it, I cherry-picked portions of code from the Android framework that I felt were important to give an overview of the entire logic that constitutes Monet. If someone from Google or the framework team by any chance happens to read it, please share your feedback and add to our understanding of Monet where ever applicable!

The source changed quite a lot while I researched into this aspect of the framework, in the future, it will change with new builds and releases of the operating system, yet I think the silhouette of how Monet works should remain the same. (for the most part!)


Further reading & references

Attributions & Credits


About the author

Siddhesh Patil is an Android Engineer at DeliveryHero. Building core ordering experiences at Foodpanda and Foodora. He is the organzier of Kotlin Mumbai, a developer community in Mumbai, India. Find talks, articles and latest developments in Android and Kotlin on this website siddroid.com

Sid Patil is an Android Engineer and Kotlin Advocate based in Berlin, Germany. He works at Delivery Hero, building ordering experiences for food delivery and Q-commerce brands like Foodpanda and Foodora.

Share on

book

More articles to read