good morning everyone thanks for joining us today it's fantastic it's fantastic to be here live in teline and connect with so many of you with Android reaching more devices from phones to foldables to tablets to Chromebooks building apps that seamlessly adapt has never been more crucial We Believe now is the time to elevate your apps and make them shine on any Android device and this talk is just about that we are going to cover all things adaptive the strategies the tools and the mindset shift I'm Fahad product manager on the Android developer team over to Alex hi everyone I'm Alex a developer relations engineer on Android 2023 marked the launch of our cing new tablet and foldable devices the pixel tablet and the pixel fold but what made it phenomenal is working with you our developer Partners optimizing your apps and games for the growing Android device ecosystem of foldables and tablets we have worked with many apps that were optimized for these devices delivering powerfully unique experiences and also worked with more than 50 Google apps to drive optimization for these devices and we have learned a tremendous amount from your incredible work optimizing your apps but also directly from users but how they engage with apps on these devices in unique ways today we'll share those insights with you to help you build outstanding experiences our talk today will cover building adaptive layouts and how the new composed material three adaptive Library helps supporting input Beyond touch and lastly testing your adaptive apps let's get started there are now over 300 million active Android large scen devices out there including foldables tablets and Chromebooks since the last IO in addition to the pixel fold more oems have launched new foldable models showcasing the incredible potential of this unique form factor we can't wait to see the amazing app experiences you'll create for this exciting form factor and talking about foldables Counterpoint research predicts foldable smartphone shipments alone will top 100 million by 2027 that's massive let's switch gears to play now building on the success of our tablet optimized Play Store experience the new foldable specific Play Store experience with tailored recommendations and foldable specific collections is a great way for users to discover amazing large scene experiences and for you to get the spotlight on foldables last year Google Play began updating its featuring and ranking system on large screen devices to prioritize high quality apps and games higher both in editorial picks and organic rankings all based on the large screen app quality guidelines play also increases the visibility for apps to Multi-Device users by allowing users to see tablet specific ratings and reviews for your apps on their phones apps that fail to meet the technical quality threshold will also display a red message on the app Details page that Wars users when apps may not perform well on their device users want apps that meet these essential levels of quality but they also want apps that can help them do so much more so the takeaway maximize the reach for your app as high quality apps are more discoverable across the board on tablets foldables Chromebooks and even phones for Multi-Device users this is why you should build adaptive apps apps that seamlessly adjust layouts based on the available Windows space and more we're committed to making it easier than ever for you to build adaptive UI into your apps because this isn't just about the device of the future it's about optimizing for the experiences that users demand today we'll talk about some new libraries soon but let's first get on the same page what makes an app truly adaptive traditionally apps could assume they were running full screen either on a phone or on a tablet but fast forward to today an apps are running across many many different devices this assumption that apps will always take up the full screen is brittle as apps can be placed in any arbitrary window size independent of the physical display size for example you might be in split screen mode browsing the web to research a topic while also watching a video on YouTube and we're seeing this firsthand with pixel fold and pixel tablet users 47% of pixel fold monthly active users use split screen mode for multitasking and increased productivity on their devices similarly apps could previously assume that they'd never be resized and that's simply no longer the case apps cannot opt out of this Behavior with all these different devices where your apps can run and all the ways users interact with those apps across these devices it's time for a mindset shift here's how you should think about adaptive apps adaptive apps adjust layouts based on conditions such as changes to the size of the window in which your app is running running like the window size changing when your app enters split screen mode changes to the device posture like users placing a foldable into tabletop mode or users adjusting the screen density or changing the font size adaptive apps dynamically adjust their layouts by swapping components showing or hiding content based on the available window size instead of just simply stretching UI elements for example you should swap the bottom navigation bar for a vertical navigation rail on larger window sizes these conditions provide a granular and meaningful way to understand the context so that apps can adjust the layout to optimize for the users's experience accordingly for adaptive apps the available window size is the guiding star with features like free form windowing and split screen mode on Android tablets your app won't always occupy the entire display so the key to ensuring that your app consistently functions across devices in any Window mode is to use the available window size to determine your layout adaptive apps also seamlessly respond to changes like a foldable device opening or closing a device rotating or the size of an app window changing in split screen mode and while doing so adaptive apps retain their state and offer seamless continuity to users when conditions change expectations are changing for Android app development adaptability for your app to any UI size unlocks the seamless experiences that users demand today we expect apps to be used across more device surfaces and window sizes and apps will need to become more adaptive in turn building adaptive app also sets you up for long-term success let's discover how so with those use cases in mind we are releasing a new composed library that helps you build adaptive layouts which is now in beta this Library includes layouts and components that adapt as users expect when switching between small and larger window sizes let's take a look at this in action with sap who have integrated the composed material 3 adaptive Library into the sap mobile start app this app provides a native mobile entry point into Sap's business applications and content first let's take a look at the navigation UI for for optimal ergonomics users expect the navigation UI to adapt to the window size swapping between a navigation bar and navigation rail as the window size changes the new navigation Suite scaffold component automatically switches between a navigation bar and navigation rail depending on the overall window size sap was already swapping these components with custom logic by using the new scaffold they were able to replace that custom code with the logic provided by the composed material 3 adaptive Library without changing visible Behavior to users sa told us they were able to delete their custom logic and switch to navigation Street scaffold within 5 minutes their previous logic was using window siiz classes to decide which navigation UI to show and the new component does the exact same thing with the configurable default Behavior because apps apps [Applause] not apps may not always take the full screen the default behavior is based on using window siiz classes as UI breakpoints window size classes partition the raw available window size into buckets each window has a width and height and the possible widths and Heights of a window are distributed in to three window size class buckets each let's take a look for a WID for width a window that is less than 600 dips wide falls into the compact width window size Class A window that is at least 600 dips wide and less than 840 dips wide falls into the medium width window size class and a window that is 840 dips or larger falls into the expanded width window size class similarly there are also separate breakpoints that Define compact medium and expanded height window size classes but available width is usually more important than available height due to the ubiquity of vertical scrolling the default behavior in composed material 3 adaptive for navigation Suite scaffold is that if the window is compact width or compact height show a navigation bar otherwise on medium or larger width window size classes or medium or larger height window size classes show a navigation rail let's look at another example of how the Adaptive layouts can help the new library also has list detail pain scaffold and supporting paint scaffold which help you implement canonical layouts that we recommend in many cases like list detail and supporting pain looking at list detail paint scaffold specifically this component handles presenting a list and detail view together in a way that adapts to how much available Windows space there is if there is room to display both say in expanded width window size class then both the list content and the detail content will will be displayed together side by side if the window shrinks say down to compact with Windows size class and there's not enough room to show both then only one or the other will be shown the component also supports swapping between the list and detail panes with animation when there's only room to show one sap has integrated the supporting paint scaffold in the mobile start app for the Todo screen this allows saap to display the comments next to the main content on expanded width Windows to allow users more context during the approval workflows the new component in composed material Tre adaptive helped sap build an experience that adapts to the available window size the available window size is not a constant as users rotate fold and resize your apps with supporting pain scaffold sap was able to provide users a delightful experience that preserves continuity across de different devices and device states building UI with the material 3 adaptive library is a full dedicated talk that dives into the specifics of these new apis so please make sure to check that out even if you aren't using the new material 3 adaptive Library quite yet we recommend all apps to use the principles and the apis that the new library is using under the hood that make it adaptive when working on the composed material 3 adaptive Library one of the main principles we isolated was conditions the inputs driving how your layout should adapt conditions are meant to simplify how top level layouts behave there's nothing particularly special about conditions in the architectural sense they're just pieces of state that we can use to drive Behavior what is important about these conditions is defining exactly which inputs to use and when the window size and the fold posture are the main conditions that we see users expect apps to react to there are some possible pieces of state that could be used as inputs but we do not use them as conditions because we are seeing them cause an extremely confusing buggy user experience physical display size device orientation window orientation and device type are all not inputs that users are expecting apps to react to let's talk a bit more about all those inputs that are related to size and go over for some examples of what users expect when using apps the physical display size is a piece of information that you can query from the system however in almost every situation this is not a useful API for you to use your app is not guaranteed to fill up the entire display size as you saw earlier users want to use your apps in Multi Window mode to multitask and you can't prevent this your users will put your app into Multi Window mode and expect it to work if you are using the physical display size and you're assuming that your app is filling up that entire space that assumption will be wrong if you make layout decisions based on that assumption your layouts will be wrong when your app's window doesn't match up with the display size and speaking of the display size it might have been true in years past that there was a single display size to worry about but that is no longer true foldables and flippables often have two physical displays Each of which has a different display size in other parameters the number of available physical displays also isn't a constant many devices can be plugged into an external monitor adding a new physical display at runtime what really matters to your app is your current Windows size this is also what users expect if they resize your app they expected to care about the Windows size that they are intentionally giving it and this means that your window size will change and you also can't prevent this when your app gets resized that causes a configuration change to occur by default each configuration change recreates your activity when this happens all the in-memory state in your old activity is lost as a new activity is created it is critical that your app save its state correctly either in saved instance state or by retaining it in a view model if you lose State then users see a completely broken experience with your app not only did the app dap not only did the app not adapt well with layouts when they resized it the app lost their place and their work as a user it is aggravating to be almost finished with some form and then I rotated the device and now I am forced to go back through and fill it all out again to calculate the window size both for the first time and each time it changes you should use the window metric calculator compute current window metric method from jetpack Window Manager this is the same method that the new composed material 3 adaptive Library uses to determine the window size to decide how to adapt its components once you are using the window size you have a couple of options to decide what layouts to use you could look at the width the height or the orientation but we don't recommend using the orientation instead we recommend using the window size classes that are based on the width and the height let's take a closer look at why it has been common to have two different layouts a landscape layout and a portrait layout for your app depending on the orientation of your window however this can lead to awkward behavior for window sizes that are very close to a 1ex one aspect ratio for example if we're showing two panes in landscape and one paint in portrait then a window that is just barely landscaped will show two pains if the user then makes the window slightly bigger by expanding the height of the window the window will become portrait and therefore we will show one one pain but this is counterintuitive resizing the app to give it more space resulted in taking away information so instead of using landscape or portrait to decide how many panes to show you should instead use window size classes the default Behavior recommended by material design and implemented in material 3 adaptive is to have one pain when the window is compact or medium width and two paints when the window is an expanded width and because the amount of PS displayed is based only on the WID We Get Much More predictable Behavior as the window resizes let's talk more about pains and how they compare to another term that is hard to Define screens in common usage you probably refer to different parts of your app as different screens you might have a setting screen a profile screen a home screen a list screen and a detail screen the term screen here however has a drawback it implies that your screen is filling up the entire physical screen on an expanded window size you have enough room to display both your list screen and detail screen together so now neither screen is filling up the entire physical screen so you have multiple screens on screen together to avoid this confusion see if you can avoid the screen term Al together think about grouping your content into different panes each pane is still a set of content that you can navigate to on smaller window sizes you'll probably see one pan at a time and navigating somewhere will just show one p at each destination if the content in a pane can nicely adapt to fill up the entire window there's nothing wrong with always showing one pane that fills up the entire window even at larger window sizes but the content in your panes relate to each other so you can often do better than that your list pane displays a list of content and your detail plane displays more information about a specific piece of that content if you have enough space to show both the list pan and detail pain together and if you don't have enough space you can fall back to only showing one pain at a time when the window size changes you should hide or show a pain when reacting to how much space you now have but which pain you show or hide should depend at which destination you are at this is compatible with one of the principles of navigation that creates consistent predictable user experience how many pains you are showing at a destination may change but the destination you're at doesn't you should always have some specific pain that remains visible regardless of how many times the window size shrinks or expands grouping content into paines instead of just screens also unlocks more flexible opportunities to rearrange panes including by user interaction one request that we've seen from users is the ability to resize PES when in a two pan layout sometimes a user really wants to focus on the cont content in one pan over the other the solution here should be a pretty familiar interaction supporting expanding PES with a dragable handle that'll resize and hide one of the two panes let's see this in action with Google Calendar the Google Calendar app Now features a new supporting side pane on expanded width window sizes this feature allows users to see relevant information side by side users can view details of a specific event while retaining the context of their daily schedule Orie pending tasks while checking their weekly commitments and with an easy drag the size of the supporting pane can be adjusted as the user Desires in Android 15 this feature is being added to activity embedding and the support will also be added to material 3 adaptive as well so everything we covered was just covered was related to the primary condition for adapting the overall window size as a summary don't use the physical display size don't use the device orientation and don't use the window orientation to drive top level layouts instead use window size classes there are other conditions that also influence how your apps layouts should adapt to a specific situation the posture of foldables is another condition that you can use to provide an app that adapts retrieving the information about the folding features is another piece of information that jetpack Window Manager provides with this information you can adap layouts in tabletop mode to provide interactable UI on the bottom half of the window and consumable UI on the top half of the window these are the main two conditions that the material 3 adaptive Library reacts to today and makes available through its lower level apis the available window size class and the fold posture of the device as we see devices continue to become more varied and adaptive themselves we expect that the types of conditions that users expect apps to adapt to will grow let's see how all these areas come together when looking at an app that is using the camera while being resized when in this mode it should become immediately clear that the orientation and aspect ratio of the camera sensor is in general completely independent of the orientation and aspect ratio of your app's window and its UI as the window resizes and posture changes layout should adapt to the available space components shouldn't just stretch some should swap out for components that make better use of the space available so we have talked about some decisions to avoid but we see many apps following more anti patterns let's take a look at some of these now you may have been wondering about locking the orientation of your activity and if doing that will prevent window size changes the answer is no entering multi- Window mode will resize your activity folding and unfolding a device will still resize your activity resizing an app in free form windowing mode will still resize your activity additionally apps should not be logging orientation in the first place if a user wants to lck the orientation of their device they can choose to do so using the system level auto rotate feature if the users have system level auto rotate enabled and they rotate their devices they expect expect apps to rotate apps logging orientation is a usability and accessibility issue for that reason apps that lock orientation will be pillar boxed on large screen devices pillar box apps see decreased discoverability on play brow surfaces across foldables tablets and Chromebooks starting in Android 14 users can override the app behavior and stretch these filler boox apps to fill the landscape screen to make better use of the large display but here's the thing stretching isn't the same as optimizing and it's far from the Polish experience that users desire as users want to use their device and their apps differently they will rotate fold resize many times all of these change the available window space for your app and users expect apps to adapt to that space another Ana pattern that we see apps having issues with is unnecessarily requiring features Play Store will prevent users from downloading your app on devices that don't support all of the features that your app requires this system made a lot of sense with single purpose apps I mean it wouldn't really make sense to install a camera app on a device that did not support a camera but apps can do so much more these days and features may only bequ required in specific us JS many permissions like camera and location can result in your app requiring a feature even if that feature is not essential to your app score experience this completely blocks your app from being installed on devices that don't satisfy the feature requirements which can include phones tablets foldables Chromebooks and even cars running Android automotive OS to avoid that Mark these features in the Manifest as not being required don't require a camera or extra camera features don't require a physical GPS don't require the device to support both portrait and landscape orientations and lastly don't require a touchcreen let's talk about another crucial aspect now how users interact with your app across different device types let's dive into input where we will cover supporting input methods like keyboard trackpad and stylus if you have developed an app for phones chances are touch input was your main focus but think about it your users are using your apps across device types from foldables to tablets to desktop devices like Chromebooks larger scenes invite productivity and creative expression users turn to keyboards mice and styluses for precision expression and accessibility the use of keyboards with tablets has been increasing as it allows users to be even more productive allowing them to get things done more efficiently and on Chromebooks 90% of app interactions are with the keyboard so the takeaway make sure your app supports input methods Beyond touch like keyboard trackpad Mouse and stylus jetpack andos over the past year has been enhanced to provide you with more interactions out of the box keyboard interactions like tab navigation and interactions like mouse or trackpad click select scroll are now available out of the box with jetpack compos 1. 7 in Android 15 we are revamping the system shortcuts helper which provides you with an easy standardized way to make your apps keyboard shortcuts more discoverable to users by overwriting the on provide keyboard shortcuts method you can publish your supported shortcuts with individual keyboard shortcut info objects which will then be easily discoverable by users in the system shortcuts helper let's see an example with a Chrome App Android productivity users familiar with desktop Chrome shortcuts like opening a new tab or a window can make an easy transition to Android tablets with recent updates to the Chrome Android app supported shortcuts now aligned to and support desktop shortcuts that were previously missing these additions don't only make basic navigation easier but also provides easier access to Chrome Caesar such as the bookmark manager let's move to styluses now styluses with Android tablets and foldables allows users to express themselves without any limits be it capturing those fleeting ideas quickly or adding those artistic touches stylus users on Android can remain more productive with new support for stylus handwriting in text Fields with jetpack compos 1. 7 stylus handwriting is supported out of the box with basic text field Android is also enabled build your app to help users be more productive with the notes Ro in Android 15 when you register your app as a notet taking app users can open your app right from the lock screen or run it as a floating window for enhanced note taking make sure to check out the increased user productivity with large screens and accessories to learn more about input methods so we have talked about layouts and we have talked about input how about making sure it all comes together and works well over to Alex to talk about testing your adaptive apps we've heard your feedback that supporting additional window sizes leads to increased testing and maintenance costs to know what's most important to test our large screen quality guidelines and test checklist are a great place to start they're designed to simplify the testing process outlining the critical user Journeys you must test to ensure a great experience across different devices part of lowering the cost of maintaining adaptive layouts is compose itself compose makes it easier to build layouts that adapt without needing to duplicate and maintain different layout files like you would with views we are also adding new testing tools that help for testing UI on a variety of device configurations including window sizes device configuration override is a new testing API in compose 1.
7 that allows locally overriding pieces of device configuration previously things like local font size Dark theme or light theme and arbitrary sizes for UI were harder to test because these are all device level parameters you can try to change these parameters at runtime but this requires setup and tear down around your tests that you have to orchestrate carefully to not impact other tests you could also launch multiple emulators with different configurations but now this requires running tests multiple times across a suite of devices for complete coverage either way it makes a complete group of tests hard to coordinate instead device configuration override is a composable API that allows specifying an overy that is applied locally just for the content under test contained inside of it for example device configuration override. layout Direction allows overriding layout Direction locally for the content under test so that you can verify that content is in cut off or shown incorrectly with right to left compared to left to right device configuration override. font scale allows setting the font scale locally for the content under test by passing two to font scale you can check how the content behaves when nonlinear font scaling is applied in Android 14 and above and my favorite is device configuration override.
Force size this override takes a DP size for a width and a height the content inside the override will be given that much space to render regardless of how much space is actually available for example you can run tests when your UI is rendering at 840 DP wide by 700 DP tall even if that test is running on a phone emulator normally this wouldn't work since the U would be cut off if you tried to use something like modifier. required size and then assertions for whether something would be displayed could fail instead the trick here is that Force size locally modifies the effective density which makes the content inside think it has enough space to render with this approach you can test your UI in different amounts of available space on a single emulator with a single run of your test Suite so you don't have to run your tests multiple times in different emulators and maybe filter tests based on which environment you are running in you can also parameterize tests to run multiple times in different amounts of space or with different font sizes or any of the other device configuration overrides integration tests that use semantic information to control the UI are great to verify and prevent regressions for the functionality of your UI but they aren't verifying what your layouts look like visually for that we are launching official support for host side screenshot tests of your composed layouts with the Android Gradle plugin host screen host side screenshot tests provide a fast and scalable way to verify that the UI visually hasn't changed and these can be used to test that UI is looking the way you'd expect across many different window sizes and if you're already familiar with creating composed previews it's just as easy to create new hosts side screenshot tests for your app and finally some updates on manual testing tools When developing use multi- preview annotations to check what your UI is looking like across multiple different sizes font scales and more the UI checks for composed preves also highlight common usability issues that might be present in your previews such as buttons or text Fields being too wide we also have a suite of emulators including the pixel fold and pixel tablet to help you test your app's behavior in all the different ways that it's being used the resizable emulator is a single emulator that can swap between a phone size screen a foldable size screen and tablet size screen and there's also the desktop emulator which makes it easier to test free form windowing Mouse hover and keyboard shortcuts and we also know that emulators aren't perfect for every case there's often no replacement debugging an issue on a real device for that we are integrating Remote device streaming into Android Studio Android device streaming powered by Firebase allows you to securely connect to remote physical Android devices including a selection of the latest pixel and Samsung devices which are hosted in Google data centers while connected you can install and debug apps run ADB Commands rotate and fold and make sure your app is working great across a variety of real devices try Android device streaming today at no cost in the latest version of Android Studio jellyfish after you sign in and connect your IDE to the Firebase Cloud project with testing there's no single method that's perfect for everything you should be verifying but that doesn't mean you shouldn't test at all we have a new guide on developer. android.
