Merging Android patches into Linux kernel

I am trying to port Android to the Beaglebone black and do a basic kernel boot. This is the step by step log of my journey.

First I clone the mainline kernel and boot up my BBB. I use the omap2plus_defconfig and am335x-boneblack.dtb file to boot-up the BBB. This will act as my reference kernel.

Next I clone the Android kernel as follows:

git clone https://android.googlesource.com/kernel/common

Next we find match the kernel version numbers. So in my reference kernel I check the tags of the kernel version number. My partial output is as follows:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

v4.13 v4.13-rc1 v4.13-rc2 v4.13-rc3 v4.13-rc4 v4.13-rc5 v4.13-rc6 v4.13-rc7 v4.14 v4.14-rc1 v4.14-rc2 v4.14-rc3 v4.14-rc4 v4.14-rc5 v4.14-rc6 v4.14-rc7 v4.14-rc8 v4.15-rc1

Next we check the branches of the Android kernel. We do a git branch -a. The partial output is as follows:

1 2 3 4 5 6 7 8 9 10 11 12

remotes/origin/android-3.18-n-release remotes/origin/android-3.18-o-mr1 remotes/origin/android-3.18-o-release remotes/origin/android-4.14 remotes/origin/android-4.4 remotes/origin/android-4.4-eas-test remotes/origin/android-4.4-llvm remotes/origin/android-4.4-n remotes/origin/android-4.4-n-release remotes/origin/android-4.4-o remotes/origin/android-4.4-o-mr1 remotes/origin/android-4.4-o-release

I have decided to use the 4.14 kernel. So I decide to checkout the 4.14 in both the Reference kernel and Android kernel.

In my Android tree I do a:

1	git checkout remotes/origin/android-4.14 -b android_4.14_linux_merge

Git log generally shows:

"Merge 4.14.14 into android-4.14"

In the Linux kernel we do the same:

1	git checkout v4.14 -b linux_4.14_android_merge

Git log generally shows:
"Linux 4.14"

Next we start the merging process.

We go to the Android kernel directory and check for the point where Linux 4.14 LTS was used to add the Android patches on top. For this we do the following:

git log --pretty=oneline --grep="Linux 4.14"

 We get the following output:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40

9b68347c3554e886487ea5587cde21926d950f15 (HEAD -> android_4.14_linux_merge, origin/android-4.14) Merge 4.14.14 into android-4.14 9c0bf9847171edd597a13adf3ddc879e96d947cd (origin/upstream-linux-4.14.y) Linux 4.14.14 918914133ea13a07ec7967b25f0992c1f8928411 Merge 4.14.13 into android-4.14 b8447222eb207d5a5ec20a0f357065963dabdcd0 Linux 4.14.13 81e7d4589439a5b5dd07b7ecb7b02dbea7f6f795 Merge 4.14.12 into android-4.14 8d577afdee3540808302d9dc7a0a7be96c91178f Linux 4.14.12 51caa5cc8092b11fd91a7ffc12428bff47ddf518 Merge 4.14.11 into android-4.14 7af3494ec5325187effc30bef12ae72318cb2108 Merge 4.14.10 into android-4.14 0cf36be1cdf63598e98f094d4fb5e65dab164d14 Merge 4.14.9 into android-4.14 7237d3f322a7b005f751eb0e8da6e766ae6d9dca Merge 4.14.8 into android-4.14 5adbbb16a574d794c3b1573b8bcc0f51b3035985 Merge 4.14.7 into android-4.14 20f3b53781e9102b365f66cfa883aee95e932bb3 Merge 4.14.6 into android-4.14 0730eb4486b992c6a6193b48d54ffb463e4cec73 Merge 4.14.5 into android-4.14 0730eb4486b992c6a6193b48d54ffb463e4cec73 Merge 4.14.5 into android-4.14 c5c36272cda5e0fddadd1cff144d436514e509f2 Merge 4.14.4 into android-4.14 50cb0d5ca6df48129265cf7bde4e738fe509050c Merge 4.14.3 into android-4.14 fbbc906cb44d660eee3ce8c6c30eeccc79ecb144 Merge 4.14.2 into android-4.14 dc6bfa18652919ae193a3bcd05c6fcc49327bdd0 Merge 4.14.1 into android-4.14 0d59679df5b53755c00ea0292df696f97bfc950d Linux 4.14.11 b8ce8232fcc37fe7a97db79ea0a5f32098c25e72 Linux 4.14.10 dad5c1402c570cd07a80113784bc20a7f930c8ae Linux 4.14.9 7b3775017f4e6b87dfd2c7f63d1eaf057948f31d Linux 4.14.8 3afae8437c3cbc22966762e80e81818f5a90eb06 Linux 4.14.7 5fd159e1ee6a87a72626139813034f24f047d0e6 Linux 4.14.6 64138f0adb25ca8f34baa57af33260b05efe2874 Linux 4.14.5 51a2a68fde2035887c0d74aee1c9569c691dfd61 Linux 4.14.4 191314edb326764c4481b09ccf7d00159abe4679 Linux 4.14.3 f9f0b03dedc19a6363a305d119efcb48667a3027 Linux 4.14.2 780a781dd6f1af9dfac15b8eeba1cb678c9fc380 Linux 4.14.1 bebc6082da0a9f5d47a1ea2edc099bf671058bd4 Linux 4.14 39dae59d66acd86d1de24294bd2f343fd5e7a625 Linux 4.14-rc8 0b07194bb55ed836c2cc7c22e866b87a14681984 Linux 4.14-rc7 bb176f67090ca54869fc1262c913aa69d2ede070 Linux 4.14-rc6 33d930e59a98fa10a0db9f56c7fa2f21a4aef9b9 Linux 4.14-rc5 8a5776a5f49812d29fe4b2d0a2d71675c3facf3f Linux 4.14-rc4 9e66317d3c92ddaab330c125dfe9d06eee268aff Linux 4.14-rc3 e19b205be43d11bff638cad4487008c48d21c103 Linux 4.14-rc2 2bd6bf03f4c1c59381d62c61d03f6cc3fe71f66e Linux 4.14-rc1 f9773b22a27a4234f436c9570afd62d905e00a13 Merge tag 'nfs-rdma-for-4.14-1' of git://git.linux-nfs.org/projects/anna/linux-nfs into linux-next 2b76da95638010a70435f8455913133acc26e93f Merge branch 'nvme-4.14' of git://git.infradead.org/nvme into fo

I find Linux 4.14 with the commit hash of "bebc6082da0a9f5d47a1ea2edc099bf671058bd4" which is the same hash which I see in the reference kernel when I do a git log.

Now I extract all the patches from 4.14 Linux commit hash to the HEAD which is all the Android patches added to the 4.14 LTS version. I do the following:

git diff bebc608 HEAD > 4.14_android.patch

Next we first check and then apply the 4.14_android.patch to the Reference Linux kernel which we have checked out at tag 4.14 and branched it. I 'cd' to the reference kernel directory and do the following:

1 2	git apply --check ../4.14_android.patch git apply ../4.14_android.patch

First I check the patch application to see if there are any errors and then apply without the check.

Next I build the kernel. Here I merge the .config files of the board i.e. omap2plus_defconfig, android-base.config and android-recommended.config. I issue the following command:

LOADADDR=0x80008000 ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- scripts/kconfig/merge_config.sh arch/arm/configs/omap2plus_defconfig kernel/configs/android-base.config kernel/configs/android-recommended.config

A .config file gets generated which I will use to build, flash and verify if the build is working fine using my Debian rootfs. Please note that there might be an issue with the systemd trying to bring up ttyO0 which I am trying to get it resolved. This works fine in the Reference kernel 4.14 and also after merging with just the omapplus2_defconfig but not after the merging of .config files of android and omapplus2.

Please note that the terminal and dns might not work. The solution is as follows:

For the terminal login to work make sure CONFIG_VT=y and CONFIG_FHANDLE=y. 

For the DNS to work we need to disable PARANOID_NETWORKING. If you still want to enable PARANOID_NETWORKING then you need to add the following groups with their GIDs:

AID_INET: 3003
AID_NET_RAW: 3004
AID_NET_ADMIN: 3005

My sample addition of users to groups is as follows:

AID_INET:x:3003:debian,root,avahi,avahi-autoipd,_apt,systemd-timesync,systemd-resolve,systemd-bus-proxy
AID_NET_RAW:x:3004:debian,root,avahi,avahi-autoipd,_apt,systemd-timesync,systemd-resolve,systemd-bus-proxy
AID_NET_ADMIN:x:3005:debian,root,avahi,avahi-autoipd,_apt,systemd-bus-proxy,systemd-timesync,systemd-resolve

Even though some of the network utilities work I was unable to make apt resolve the addresses. I still need to tinker with the options to make it work. I will update it here when I am able to get it working.

Android AOSP Nougat Build and Jack "Out of Memory" issue resolution.

I had downloaded Android Nougat before and had failed miserably with Jack. It used to always give me a "Out of memory" error. I have an Intel Quad Core Q8200 2.33GHz laptop with 4GB of RAM.

Looking around I found a post by Chris Simmonds in his blog which said which says to increase the memory  using the JACK_SERVER_VM_ARGUMENTS environment variable. I set the following value in my ~/.jack_settings i.e. max memory to 3GB:

1	JACK_SERVER_VM_ARGUMENTS="-Dfile.encoding=UTF-8 -XX:+TieredCompilation -Xmx3g"

 
While compilation I still had problems with JACK Server having communication problems. I further reduced the JACK compile jobs to by setting the SERVER_NB_COMPILE=1 in ~/.jack_settings.

Please see reference 3 where the Google documentation for JACK is updated and it details on different parameters JACK.

I am not sure whether the problem was resolved by installing
ibcurl4-openssl-dev and nss-updatedb packages but it is worth to give a try.

After this I was able to completely build Nougat and boot it up using my emulator. The final aim is to understand how to boot Nougat on the Beaglebone.

Reference:

1. Chris simmonds post: http://www.2net.co.uk/blog/jack-server.html
2. Google groups post: https://groups.google.com/forum/#!topic/android-building/8SQ0-4zZDo8
3. Google Gerrit Jack Documentation change: https://android-review.googlesource.com/c/platform/prebuilts/sdk/+/233481




Creating a bootscript file for u-boot

I have created a boot script file to boot my Beaglebone via tftp. I did not want to change the default boot configuration in the headers as it would add changes to my u-boot code which would deviate from the mainline.

First we create a file called boot.cmd. I have the following in my boot.cmd file which can be reused.

1 2 3 4 5 6 7 8 9 10

setenv bootargs console=${console} root=/dev/mmcblk0p2 rootfstype=ext4 rw rootwait setenv fdtaddr 0x8C008000 setenv loadaddr 0x80008000 run findfdt setenv tftp_fdt tftp ${fdtaddr} ${fdtfile} setenv tftp_uimage tftp ${loadaddr} uImage setenv sd_fdt fatload mmc 0:1 ${fdtaddr} ${fdtfile} setenv sd_uimage fatload mmc 0:1 ${loadaddr} uImage setenv tftpboot 'run tftp_fdt; run tftp_uimage; bootm ${loadaddr} - ${fdtaddr}' setenv sdboot 'run sd_fdt; run sd_uimage; bootm ${loadaddr} - ${fdtaddr}'

We change this to boot.scr file by doing the following:

1	mkimage -C none -A arm -T script -d boot.cmd boot.scr

We keep this boot.scr in the first partition. We reset/reboot/boot the board and then we do the following:

1 2 3 4 5

fatload mmc 0:1 0x80000000 boot.scr source 0x80000000 setenv serverip <yourtftp ip address> setenv ipaddr <yourbeagleboneip> run tftpboot

Basically the above would load the boot.scr file in 0x80000000 location and run the script from that location by doing a run <memory_address>.

I will improve on this script a little bit and will keep updating this blog post.

Beaglebone UART connection using PL2303 USB to UART adapter

To connect to the Beaglebone UART for console messages I connect to my PC with a PL2303 USB to UART adapter.

The product page is present here: https://www.adafruit.com/product/954

We have the wire color coding and pin-out to the Beaglebone's J1 header6 (6 pin header) as follows:

Red: +5V (DANGER: Do not connect to any of the UART headers. Please keep it unused for UART. You may use it for something else but please read the specifications before doing it)
Black: Gnd (Pin 1 which is marked with white dot)
Green:  Tx from USB port to Rx of BeagleBone UART (Pin 4. Net Name: B_UART0_RX)
White: Rx into USB port from Tx of BeagleBone UART (Pin 5. Net Name: B_UART0_TX)


A reference image to easily verify:

After connection you can verify the console messages using a program like screen with a configuration 115200n81.