WHISKEYCON

Android Internals

Instructor

Jonathan Levin

Training Dates

25 - 29 March 2016

Description

This course, modeled after Jonathan's Levin's seminal works of the same name, discusses the architecture of Android - covering both its design and its implementation. We examine the features of the Android Operating System, and highlight exactly what it inherits from its Linux core, as well as how it diverges with its idiosyncrasies and "Android-isms". Every nook and cranny of the architecture is explored, with modules discussing specific subsystems, such as the Dalvik Virtual Machine, Android RunTime (ART), the Binder IPC mechanism, Hardware Abstraction Layer (HAL), the Media Framework and Android Power Management.

Objectives

  • Describe the architecture of the Android operating system
  • Describe the similarities between Linux and Android
  • Describe the differences between Linux and Android
  • Understand core architectural differences from Froyo (2.2) to Marshmallow (6.0) and Nougat (7.1)
  • Describe the functions and architecture of the Android Kernel
  • Reverse Engineer Android Apps
  • Understand Android Security, its evolution over history, and its weaknesses
  • Monitor, trace and intercept inter process communication on Android
  • Understand the frameworks of Android, and interception points
  • Understand both the framework layers and the native layer of Android
  • Gain deep knowledge of the DEX, ART and OAT formats
  • Learn to use innovative free tools, such as Dextra, bindump, and jtrace

Exercises

This course allocates time for hands-on practice, and plenty of instructor led demos. The hands-on exercises include:

  • Reverse Engineering an Android App
  • Native level debugging of Activities
  • System call and kernel level tracing

Topics

Day 1

Introduction - The Android Architecture

Introduction to the Android architecture. We discuss the design and implementation of the various subsystems, at a modular "black box" level, without going into the source code level (yet).

  • Android features
  • Android vs. Linux vs. Embedded Linux
  • Filesystem layout and directories
  • The Runtime Environment
  • The Frameworks
  • Dalvik (Java)
  • Version differences - 1.5 through L (5.0) and M (6.0)
  • User-Mode and Kernel-Mode Architecture
  • Android Kernel modifications

Inside an Android

Exploring the Android shell environment, using ADB and the various commands. We show both simulator and real devices, and examine, in depth, the various commands availble for viewing system information and debugging.

  • ADB and the Android Shell
  • Commands in /system/bin
  • Android Filesystem layout
  • Android process memory
  • Commands in /system/xbin
  • Debugging techniques

Exercises include:

  • Installing native binaries on the device.

Day 2

Booting

System startup and initialization - Walk through the various stages, from boot loader through kernel startup to basic setup of user mode processes. We also describe the modification of the boot loaders.

  • x86 vs. ARM
  • The Boot loader, and FastBoot
  • ARM TrustZone (32-bit) and ELx (64-bit)
  • Kernel Startup
  • User mode init - /init and /init.rc
  • Services and Daemons

Application Lifecycle

We walk through a simple Android application - working through the Android SDK, the Dalvik Java environment, and the basic frameworks. This module is optional, in case the target audience are new to Android, and is not meant to be comprehensive - only to cover the key concepts, and map them to the underlying native APIs that implement them. Otherwise, it can be skipped. Topics include:

  • Basic application structure - Packages and Manifests
  • Components
    • Activities
    • Services
    • Broadcast Receivers
    • Content Providers
  • Installation and Removal
  • Package Management
  • Application startup and the Zygote

Exercises include:

  • Reversing an Android Package (APK)

The NDK

For developers who need to escape the Virtual Machine, Android offers the Native SDK - or the NDK. For performance critical regions, going into native mode is a necessity, not a luxury. We explain the basic ideas of the NDK, its bridging from Java, and sample from its APIs.

  • Why go Native? Introduction to NDK
  • Bionic - Android's LibC
  • Accessing Native Calls - JNI
  • Building an Android native binary (exectuable)
  • Building an Android native shared library
  • NDK Samples

Exercises include:

  • Compiling and building a sample NDK application
  • Advanced system call tracing with jtrace

Day 3

Native Services

Describing the native services started by init - we walk through each and every one of the AOSP services, as well as some common vendor ones. The list includes:

  • adbd
  • servicemanager
  • healthd (K +)
  • logd (L +)
  • lmkd (L +)
  • keystore
  • keymaster (M +)
  • zygote/app_process
  • debuggerd

Android IPC

Android's IPC mechanism is based on Binder, with some UNIX domain sockets on the side. We detail the inner workings of Binder, including:

  • IPC and RPC basics
  • The servicemanager
  • Binder by layers:
    • Java: AIDL
    • Frameworks: android.os.Binder
    • Native level: libBinder, BBinder, BpBinder
    • Kernel interface: ProcessState and IPCThreadState
    • Kernel implementation: ioctl(2) codes and protocol
  • Other communication mechanisms: Sockets and socketpairs

Exercises include:

  • Debugging and Tracing Binder IPC.

The Input Architecture

Android has a complex stack to manage various input sources, such as the touch screen, sensors, and external devices. This module traverses that stack, covering each layer in turn:

  • The Linux Kernel Input Model
  • The EventHub
  • The InputReader
  • The InputDispatcher
  • The Activity Views

Exercises include:

  • Debugging and Tracing Binder IPC.
  • Injecting Events
  • Monitoring and Capturing Input Events

Day 4

Dalvik

Google would have you believe that Dalvik, the virtual machine architecture, is no longer relevant now that L and later versions use the Android RunTime (ART) by default. This couldn't be further from the truth. In this module, we discuss:

  • The Dalvik VM architecture
  • The DEX file format
  • The DEX OpCode and instruction format
  • Optimizing DEX
  • Reversing DEX
  • Obfuscating DEX
  • The JNI model

Exercises include:

  • Using dexdump and dextra to reverse a Dalvik APK's classes.dex all the way to Java source

ART

Since its inception as a "preview" in KitKat (4.4), ART has become the de facto standard for Android's application runtime. It has, however, changed with every version, up to and including the N Preview:

  • The concepts behind ART
  • Advantages over Dalvik
  • the ART and OAT file format
    • L: ART 009/012, OAT: 039/045
    • M: ART 017, OAT: 064
    • N (7.1): ART 030, OAT: 083
  • Reversing ART
  • ART Memory Management
  • The JNI implementation

Android's Hardware Abstraction Layer

The Android Hardware Abstraction Layer is a key component in Android. It enables the operating system to be adapted to different devices. The basic devices of a phone or tablet - the leds, power, vibrator and others - are abstracted by libhardware into virtual hardware devices, and we discuss them here.

  • The need for a hardware abstraction layer
  • The basic devices
    • Camera
    • Flashlight
    • GPS
    • Leds
    • Sensors
    • Vibrator
  • The User Event Daemon
  • HAL stubs
  • The last stretch - from HAL to the driver

Android Media and Graphics

Introduction to the Android Media Framework - SurfaceFlinger, AudioFliger and StageFright, and ways to interface with them. Topics Include:

  • MediaFlinger and the View subsystem
  • AudioFlinger
  • The Frame Buffer - Playback and Capture
  • SurfaceFlinger
  • MinUI
  • The DRM framework

Day 5

Connectivity (Optional)

An examination of Android's connectivity mechanisms:

  • The Network Stack (+ netd and ConnectivityManager)
  • BlueTooth
  • RILd
  • VPNs
  • The USB Stack

Android Kernel Tweaks

Android is, at the kernel level, almost identical to the common Linux kernel. There are, however, idiosyncrasies and modifications, which optimize the kernel to the mobile environments - low memory conditions, power management considerations, and more. In this chapter, we dig deep into these modifications, by examining each in detail, at the source code level. Specifically, we discuss:

  • ASHmem - Android's Anonymous Shared Memory mechanism
  • PMem - Physical contiguous memory for non scatter/gather capable hardware
  • Low memory killer - Allowing customized Linux OOM behavior from user space
  • Wakelocks - and the enhancements to Power Management
  • The RAM Console and pstore - Used in Android to save Panic data across reboot
  • Timed GPIO/Output
  • Kernel level debugging and tracing

Security

  • Java level modifiers: Private and Public
  • Java level Permissions
  • UNIX/native level Permissions
  • Digital Signatures (code signing, etc)
  • Key storage and encryption
  • Service Hooks
  • Protected APIs
  • Kernel mode security
    • The Linux Capabilities model
    • Linux Security Modules (LSM) and SELinux
    • Kernel buffer overflow protection - ExecShield and randomization
  • Extensions and improvements in Nougat (e.g. seccomp-bpf)
  • Why it all fails - Android Exploitation

Course Requirements

Student Requirements

Experienced Android Developers or Implementors; Security Researchers with a deep desire to understand the internals of the world's most popular mobile operating system.

This course is NOT intended for user mode developers who wish to develop GUI applications or use the Android Java SDKs - but can be a great recommended followup for those who already do

Software Requirements

The Android SDK, NDK, and full sources of Android - all of which freely obtainable, from developer.android.com and android.googlesource.com, respectively

Hardware Requirements

Though you can get by with an emulator, we strongly recommend bringing a rooted device, of at least Lollipop or higher. You will also need a Linux host - though we will gladly supply a VM