You are viewing outdated content for BUG. If you have a BUG Y.T. edition or 2.0 series device, please visit our updated wiki: http://wiki.buglabs.net

Building Android image for Bug 2.0

From BUG Wiki

Jump to: navigation, search

1 Content
2 Build and install guide
3 Development documentation

Content

This page explains how to build and install Android 2.2 (Froyo) for the Bug2.0 hardware. Furthermore it explains parts of the Android build system so that interested individuals can make modifications to it.

Build and install guide

Getting the source

NOTE: As of September 2010 the official instructions on android.com are mostly targeted at users of the SDKs. Some changes to the directory structure have taken place in the official repositories. The Buglab's version of Android (which still pulls most sources from android.git.kernel.org) follows the rules of the latest development.

NOTE: Android community is quite scattered and fragmented and there exist forks all over the net. The Buglabs' version tries to be as close as possible to the official sources. The official Android tool 'repo' makes it very easy to track sources from different origins.

Get the dependencies mentioned here http://source.android.com/source/download.html

sudo apt-get install git-core gnupg flex bison gperf build-essential zip curl zlib1g-dev gcc-multilib g++-multilib libc6-dev-i386 lib32ncurses5-dev ia32-libs x11proto-core-dev libx11-dev lib32readline5-dev lib32z-dev

Java5 is not needed. Current development trunk of official Android is Java6 compatible. The respective patches have been applied to the Buglabs version of Android as well.

Follow official instructions to install repo

Read the instructions on Initializing a repo client but do not use it with the official repository URL. Instead use this URL:
- git@github.com:buglabs/android-buglabs-manifest.git

cd into/your/path
mkdir bin
export PATH=$PATH:`pwd`/bin
curl http://android.git.kernel.org/repo >bin/repo
chmod a+x bin/repo
repo init -u git@github.com:buglabs/android-buglabs-manifest.git

Run 'repo sync' as said in the official documentation.

repo sync

NOTE: It is likely that the download of the source is interrupted. In such a case just run 'repo sync' again. It will continue from the last step.

Well done! You are now ready to build Android.

Configuration and build

Everytime you want to compile something using the Android build system it is neccessary to chose a target first. This process sets up some environment variables which are then used throughout the build system.

At first one has to get access to the build system's command at all. This is achieved by doing the following:

Enter the source's top-level directory
Run 'source build/envsetup.sh'

Now you can configure your build for a target. For starters do this:

Run 'lunch'
- Choose between either of the three variants being available for the bug20 device:

bug20-eng
bug20-debug
bug20-userdebug

The 'lunch' command presents its options as a list with each list item being prepended with a number. For subsequent builds (or automation) you can easily select a configuration by running 'lunch' with one of those numbers as its argument. E.g. to select 'bug20-userdebug' you can simply run 'lunch 5'.

NOTE: More options are available when using the 'choosecombo' command.

Run 'make' and wait a while (a complete build may take an hour).

Installation

The installation of Android is currently being done by using a µSD card that is being placed into the first µSD card slot of the bug. A card with 512 MiB was found out to be sufficient for a typical installation. For Android it plays no role how you partitioned your SD card. However many OMAP3-based devices follow a scheme where you have a small FAT boot partition and a ext2/3 partition for the root filesystem. The remainder of this documentation expects this to be the case.

SD card setup

Use Image:Omap3-mkcard.sh to prepare your SD card for booting. It creates one FAT partition for the uImage (kernel) and one ext3 partition for storing Android's root fs.

Android to SD card

The build process automatically compiles the kernel and the userland root fs. The results of the compilation can be found in the directory:

$ANDROID_DIR/out/target/product/bug20

There you find:

kernel: The uImage file of the kernel.
data: The Android data fs
system: The Android system fs
root: The Android root fs

The mentioned filesystems will also exist as NAND flashable YAFFS2 image files. At this point we are not interested in those. Instead we copy the directories and the kernel to the SD card.

Copy kernel to / on the FAT partion and name it uImage.
Assuming that the ext3 partition of the SD card is mounted to $EXT3 do

cp -rp root/*  $EXT3
cp -rp system $EXT3
cp -rp data $EXT3

Note: The order of the filesystem copy operations is important because root brings its own empty system and data directories.

At this point you have a bootable SD card. The following steps describe how to set up the uBoot bootloader for that.

Setting up the bootloader

Put the SD card into the first slot, make sure you have connected the debug board and a USB cable to it and have a terminal application ready to access the bootloader. Then power up the bug and access the bootloader prompt.

Set up a variable for the boot arguments:

setenv androidargs setenv bootargs console=ttyS2,115200n8 root=/dev/mmcblk0p2 rw init=/init rootwait rootdelay=1 kgdboc=ttyS2,115200n8 usbcore.autosuspend=-1 printk.time=1

Set up a variable for the boot procedure:

setenv androidboot 'mmc init;fatload mmc 0:1 0x80000000 /uImage;run androidargs;bootm 0x80000000'

Optionally make running 'androidboot' the default command when the device starts by invoking:

setenv bootcmd run androidboot

Make your changes persistent by running:

saveenv

To manually boot Android from the SD card run:

run androidboot

Miscellaneous

The following sections contain background information to all kinds of known issues and their workarounds. It is advised to read this before filing bug reports.

Suggested PIMs

The Android port works best when the LCD/Touchscreen PIM is connected and used. Android expects a touchscreen to be connected to the device and certain applications crash when they cannot find one. Nevertheless the port also works with the video PIM's DVI and VGA output. The video PIM's driver sets the module into DVI mode and there is no support for switching in Android yet. So the out-of-the-box experience is best with a display device connected to the DVI output.

Hardware support

Currently the Android port makes proper use of the following hardware:

CPU (armv7a binaries)
Touchscreen
Wifi
built-in sound SoC
SD/MMC controller (some issues with the 2nd controller though)
DSP (supposedly hardware accelerated built of OMX including dspbridge is build, installed and could be seen to be used while playing a MP3)

Not or not properly supported are:

GSM
GPS
Video PIM (working without touchscreen and automatically changing to VGA mode)
other PIMs
hardware buttons

Hardware issues and workarounds

With certain pre-production bug2.0 boards and LCD PIMs it is possible that the PIM is not properly powered up. A workaround for this is to:

boot the device with the video PIM connected to it (make sure the kernel does not complain about EEPROM trouble)
set the video PIM into VGA mode (regardless of whether there is something connected to the connector or not) by issueing:

echo "vga" > /sys/bus/bmi/devices/bmi-dev-1/vmode

replace the video PIM with the LCD PIM now
wait until you see something that looks like the Android desktop on the LCD (to make sure you're not interrupting the application installation process)
run reboot from the terminal

First boot

During its first boot Android extracts all system applications to the data partition. This can take a while, so be patient. What you are supposed to see while it extracts thing is the animated Android logo. When no main screen appears after a while you can trouble shoot with

dmesg

Or:

logcat -d

logcat is Android's internal logging infrastructure. You don't have any syslog support but every system component and application of Android is logging into their internal logging infrastructure.

Note: Via the serial console Android seems to be not react to 'Ctrl-C' which is very inconvenient. Be carefull not to start processes which can only be ended via this key combination.

Touchscreen calibration

During the build a pre-made calibration file for the touchscreen is automatically installed. It is possible that this calibration info fits badly to your screen. In order to redo the calibration you need to remove the calibration file from the image.

You can either do it from within Android (via the serial console) or by mounting the SD card in your computer. The file to remove is:

$ROOT/system/etc/tslib/pointercal

After the removal restart Android. Before the animated Android logo appears tslib's calibration program will be started and requests you to touch certain points of the screen. When the procedure is finished a new pointercal file is created and being used by Android's input system.

Using a USB keyboard

Having a USB keyboard plugged into the USB socket of the bugdock allows controlling the Android UI easily. So far all Android devices out there have some physical buttons and as such the Android desktop can hardly be used with a touchscreen only.

Note: Having the keyboard plugged in at boot time sometimes prevents Android from seeing it. If that is the case, just unplug and plug it in again.

F1 - gets you out of the lock-screen, activates menus (hit twice when in lock-screen)
HOME - ends programs, brings you back to desktop

Warning from WifiSettings program

The WifiSettings program might occassionally warn about the scanning operation taking too long and asks you whether it should shut down the whole thing. Do not do that, everything is fine and is probably just a bit outside of the deadline of what Android expects.

Power management and state of drivers

Android's power management locks the screen after a certain time of inactivity. After another while it would power down the screen's backlight and do not react to touchscreen input anymore. With the current state of the drivers this however is a bit problematic on the bug2.0:

the backlight is not turned off, so you still see the lock screen
there is no usable key on the bug itself to wake up Android

This is why a USB keyboard and pressing its F1 is neccessary.

Building the kernel

The kernel is built as part of the Android build and it is not neccessary to deal with it manually. However in case you want to do that, here are the instructions for manually compiling it.

The kernel sources are in the Android directory under kernel.
run make ARCH=arm omap3_blandroid_defconfig
run make ARCH=arm CROSS_COMPILE=$ANDROID_DIR/prebuilt/linux-x86/toolchain/arm-eabi-4.4.0/bin/arm-eabi-

$ANDROID_DIR refers to the directory where you checked out the Android sources to. The Android sources contain prebuilt toolchains of which we chose the gcc 4.4.0 one.

Development documentation

This part of the page describes a bit of the Android build environment and explains the configurations and customizations done to get it to work on the Bug 2.0 . Please note that this guide does not want to copy the official Android documentation.

Android build system

The Android build system uses a set of makefiles at its heart. Unlike in many other projects (e.g. Buildroot/OpenWRT or similar) it avoids calling Make recursively as it considers this harmful. Unfortunately making Make (actually it is dependent on GNU's make) work non-recursively comes at two immensive costs:

one has to use uncommon conventions to achieve that
for compiling a third party program one must write a new makefile for it

The latter has the side effect that even if you are familiar of how a certain software package build system works, you have to learn how it was integrated as part of Android,

When introducing a new machine into the Android build you will have to write new makefiles. However these files are never named Makefile but have names like BoardConfig.mk, Android.mk etc. Those names designate special meaning in the build process.

Inside the Android source directory you will find the whole build system architecture inside the directory build. Whenever you are unsure about the meaning of a certain variable it is a good idea to grep through this directory and try to understand from the code that uses the variable on how it is supposed to be used. Unfortunately documentation on things like this is very sparse. This documentation will explain all the things used for the Bug 2.0 Android port. :)

Makefile names

Android.mk: This is a makefile for an application or library. You will find many of them in the subdirectories system/core and framework. It is always a good idea to look at those for understanding how certain things are solved. Usually the board and product makefiles set certain variables and according to that a certain software package is compiled at all or enables/disables certain features. Every Android.mk file can introduce random named variables which you can then use in the board and product makefiles. This is a bit unfortunate because that makes tracing a variables' meaning and its possible values a bit difficult.

Products

Product related makefiles are usually put into a directory called device/<vendor>/products. In Android releases before Froyo this was usually vendor. Keep this in mind when looking at the sourcecode for rowboat, CyanogenMod or other Android-derived projects.

AndroidProducts.mk: This makefile has the sole purpose of specifying other makefiles which should be considered product definitions. The names of those can be freely chosen. They have to be added to a variable called PRODUCT_MAKEFILES.

<product makefile>: A product makefile defines properties for a product. One can define the included software packages by adding values to the variable PRODUCT_PACKAGES. The actual values are the package names from the Android.mk files. Furthermore one can define the variables PRODUCT_NAME, PRODUCT_BRAND, PRODUCT_DEVICE. This should give enough flexibility to make a certain Android build correspond to a companies' naming scheme for products.

Devices

Device related makefiles are to be put into a directory called device/<vendor>/<devicename>/.

BoardConfig.mk: This makefile serves as the main way to configure the build for a specific board. The variables used for the bug 2.0 Android port will be discussed in detail further down.

Projects that support lots of different devices have started to use common directories for things that are shared by different vendors, devices, board revisions etc.

Partial builds

Whenever you changed the source code for some library or program in order to test something it, you are not required to completely rebuild the system image from scratch just to get that change on your device. There is an easier way: Assuming you already ran

build/envsetup.sh

and chose your target configuration, e.g.

lunch 5

then you can enter the subdirectory with the changed sourcefile (may it C, C++ or Java) and go up one directory level until you find an Android.mk file. When found run

mm

and look carefully what files this build copies into the out directory. When the partial build finished copy the files you are interested in to your device's SD card.

Partitions

The Android OS is normally installed into internal flash memory where it spans over at least three partitions. There is

a root partition which contains the /init program, its configuration init.rc and some empty system directories (dev, sys, proc)
a system partition which contains all the basic programs (dalvik, zygote, sh, ...) and all the user-visible application (mail client, web browser, ...)
a data partition that contains user and program data

Normally the boot process mounts all those partitions from flash memory. When using a single SD card however the partitions will be merged into one directory tree.

Note: Independent from the approach taken, the root partition will show up as /, the system partition as /system and the data one as /data in the filesystem.

bug 2.0 Android configuration

For the bug 2.0 hardware a directory called device/buglabs/bug20 was created. It contains the bord configuration and all the files that are needed to complement the build. Besides the board configuration there is also a product definition which is also called 'bug20'. It resides in the directory device/buglabs/products.

The board configuration in the file BoardConfig.mk mentions a bunch of variables which are now explained.

Variable(s)	Explanation
TARGET_NO_BOOTLOADER	Configures the build system to not build a boot image.
TARGET_NO_KERNEL	Configures the build system, that a kernel is needed. Note: Kernel building is not part of the official Android project. Refer to 'bug 2.0 Android customizations' for a thorough explanation.
TARGET_PREBUILT_KERNEL	Defines this variable to be definitely empty. Having a value here would make the kernel build to be skipped.
TARGET_KERNEL_DIR	Denotes the directory (below $ANDROID_DIR) where to find the kernel sources.
TARGET_KERNEL_TARGET	Denotes which makefile target is to be build in order to receive a kernel binary. Since bug 2.0 uses uBoot we use uImage.
TARGET_KERNEL_DEFCONFIG	Denotes the defconfig being used to configure the kernel.
TARGET_PROVIDES_INIT_RC	Denotes that we are providing a custom copy of the init.rc file. This file describes what to do on boot. More about it further down.
TARGET_USE_GENERIC_AUDIO	Denotes that now custom Audio provider is being used.
TARGET_CPU_ABI, TARGET_CPU_ABI2, TARGET_ARCH_VARIANT	Variables setting the CPU architecture. Being used by various makefiles for optimizations. Will also find its way into a properties file.
BOARD_USES_TSLIB	Enables the tslib build (Check tslib's Android.mk).
BOARD_USES_ALSA_AUDIO, BUILD_WITH_ALSA_UTILS	Defines that audio output is being done through ALSA and that the necessary userland tools are to be installed.
BOARD_WPA_SUPPLICANT_DRIVER, WPA_SUPPLICANT_VERSION	Defines the use of wpa_supplicant 0.6 (as opposed to 0.5) and sets the driver the supplicant should use. The 'WEXT' values comes from a bit of makefile trickery done in wpa_supplicant's Android.mk (Which is worth having a look at it to see what is common in the Android build environment).
WIFI_DRIVER_MODULE_PATH, WIFI_DRIVER_MODULE_NAME	These variables turn into preprocessor definitions for libhardware_legacy which is used to control for (un-)loading the Wifi kernel module and firmware.
PRODUCT_COPY_FILES	Every entry in this variable follows a source:destination pattern. The first section of destination determines whether a file ends up on the root, system or data partition.
ADDITIONAL_DEFAULT_PROPERTIES	Values (in Java properties file syntax) added to this variable end up as globally reachable system properties. The one defined here (wifi.interface) is important for the Wifi systems of Android to work with the right network interface.
HARDWARE_OMX	Enables the hardware accelerated part of the Texas Instruments OMX library. Setting this variable to true is important because otherwise this library won't be build and installed which then causes a segfault in libstagefrighthw. Note: Apparently the hardware implementation of OMX is licensed under the LGPL. Google and Android normally do not permit copyleft licenses in the userland.

bug 2.0 Android customizations

This section deals with modifications done to the stock Android 2.2 environment. This should help upgrading to future Android releases.

Java6 support

The current Android development tree already contains patches to make Android compatible with a Java6 JDK. Since it was impossible(!) to download a Java5 JDK it was decided to apply these patches to the buglabs version of Android. The changeset in question is 7fcee4413fcdca8e88d1ed1b8d3d9876e606f496 and it applied to the build subproject. When upgrading to a later Android version this customization does not need to be ported.

Kernel build

Stock Android does not include support for building the kernel itself (This is probably largely because of the fact that most Android devices depend on some proprietary kernel modules which are installed as-is). However the CyanogenMod community's Android port has this functionality. The modification consists mainly of adding the file build/core/kernel.mk and an inclusion to it from build/core/Makefile (main changeset 3842ea2ed963cd0c227e27c7b3c2b28a1b07ae1e). Some later changes only add some more flexibility e.g. allowing to specify the kernel's makefile target.

It is important to note that the kernel.mk approach breaks with the no recursive make rule, as it calls the kernel's build system. However it was deemed that it makes more sense to break the rule instead of rewriting the Linux kernel's build system.

For future Android versions it might only be required to include that kernel.mk into the build system again.

tslib

Stock Android's input system expects a touchscreen's kernel driver to report the exact value range of the physical device. However in case of the TSC2004 the driver just reports a theoretical value range (0-4096) but the values reported by the device are actually between 0 to 250. So what is needed is a way to calibrate the input system to the touchscreen device. The Android-x86 project contains a very non-invasive approach to it: The input system reads a tslib pointercal file and uses that to transform the physical values into the display value space.

In order to obtain a pointercal file two approaches have been taken in Buglabs' Android.

a pre-made file is being installed as part of the build proces
tslib's ts_calibrate program is being run when the boot process does not find a pointercal file

For the latter to work a small script called calibrate.sh was written. Since the Android shell lacks an if or test (aka [) program a solution using cat was found. Additionally calling the calibration program was supposed to stop the boot process until it is done. Android's init provides the function exec for that, however this function is not implemented in stock Android. Because of that the whole touchscreen calibration system consists of the following customizations:

adding tslib as a project under external
implemented exec in the init program (changeset d3e1f3ee81724fadb10f766c7bb4973d203ec272 in system/core)
adding the shell script calibrate.sh which calls ts_calibrate if necessary
patching the input system to the use pointercal matrix (taken from Android-x86, changeset 4b9d58e1030f4b4a5a7fb5bce7264c74dce222fd in framework/core)

For future Android versions one has to check first whether a built-in solution for touchscreen calibration has been developed. With Android being used in ever more diverse devices there is a growing demand for a clean approach to that. If no official solution has been developed than the above customizations have to be ported. One should also have a look at how Android-x86 did it.

Miscellaneous Android topics

The following section describes various things related to the Android system development workflow and crucial files.

Repo and git

The small tool repo is used to keep together all the parts of the Android source directory. Inside that one you will find a hidden folder called .repo. This in turn contains a manifest.xml symlink pointing to a file inside the git repository whose URL you used initially to set up the whole source directory. Whenever it is necessary to change the locations/URLs, names or global revisions of the parts that make up Android you need to modify .repo/manifest/default.xml. This XML files' syntax is pretty straight-forward. Additionally it also contains some comments about where those git repositories are taken from.

When changes are made to the files of a specific project it is important to note that repo does not put the local git repo onto a proper branch. So to say git branch will always return

* (no branch)

first. Before making modification it is therefore advised to switch to a named branch first by running:

git checkout -b froyo

Furthermore the origin and default branch is also not set for the individual local git repositories. That means when pushing changes back it is necessary to specify those, e.g.

git push github froyo

The name of the actual origin is taken from the <remote> tag that corresponds to the local git repository. If unsure which one that is for a certain project just run:

git remote

Dealing with patches to projects hosted by the official Android project

Whenever it is necessary to deviate from the official Android project the respective git repository has to be migrated. The workflow to do that can look like this:

assuming you already changed to a sane branch (here: froyo) like described above
patch project A which has korg as its only origin, commit to local repository
create a new repository at github resembling the name of the project given in default.xml
add the new repositories' URL to the local repository as a remote by running:

git remote add github <GITURL>

push the whole git repository and your change

git push github froyo

modify default.xml to retrieve that project from github instead of korg, commit and push that change
other developers should repo sync now

system property mechanism

Android takes Java's system property mechanism to a whole new level. It allows system properties to be available in read and write fashion throughout the system. That includes Dalvik-, C- and shell programs. Via the shell the program setprop allows setting a property. Via getprop one either receives a pretty printed list of all system properties and their values (no argument) or the value of a specific system property.

Setting the value of certain internal system properties can have side-effects. One example is explained in the init.rc section.

init and init.rc

This file is the input to Android's init program. The sources for it can be found in system/core/init. This directory not only contains the program but also a readme.txt explaining the RC-file's syntax. However only by reading the code (builtin.c is the most interesting when it comes to the individual commands) you'll find out about caveats.

The init.rc system is very powerful and allows the definition of services. However the choice to call the things in init.rc services leads to a big confusion because the Android userland also works using services. Those are a completely different matter and attempts to find out more about the boot services via a web search are futile.

Each boot service in Android has a specific name attached to it. By setting the value of the system property ctl.start to one of those boot service names makes init try to start it. The result of the attempt can be found as a value in the system property init.svc.<servicename> a few moments after setting ctl.start.

In order to stop a running boot service one can set the value of ctl.stop to the name of the service to stop.

logging

Throughout the OS Android uses a single logging system. Patches to third party programs usually modify the program's logging mechanism to call Android's instead. Each log attempt is accompanied by a so-called log tag. This tag is usually globally defined for C programs. In Java sources there is usually a static variable TAG that is used throughout that file. The tag is important because it allows you to set the logging level for those logging system users at runtime.

The system property to use for this is called log.tag.<TAGNAME>. The possible values are described in the official documentation.

In C/C++ sources you should look for a

#define LOG_TAG "Something"

that is defined before including any header files. If you want to modify the loglevel of that compilation unit (log tags can be different in each!), use the above mentioned system properties with the log tag of the source file. Few sources are missing a log tag but still use the logging system. If you are interested in those outputs it is advisable to add a log tag yourself and recompile and install that part of the system.

Wifi driver and firmware

In Android 2.2 handling the Wifi kernel module is done by libhardware_legacy (sourcecode in hardware/libhardware_legacy/wifi). Given the name of the library it is possible that the location for this changes in later Android versions.

From the sourcecode wifi.c one can easily see that the variables defined in BoardConfig.mk end up directly as preprocessor macros in this code.

The firmware loading mechanism can be delegated to a specific program through libhardware_legacy. However if no such program has been specified (as it is done in Buglabs' Android) the firmware loading is done by init. This program's device.c reveals that the default location for firmware files is /etc/firmware. There is however a catch: /etc is a symlink to /system/etc because of a line creating that symlink in the default init.rc. That explains why firmware files have to be put into /system/etc/firmware finally.

Retrieved from "http://www.bugcommunity.com/wiki/index.php/Building_Android_image_for_Bug_2.0"