NOTICE: This content was originally posted to Google+, then imported here. Some formatting may be lost, links may be dead, and images may be missing.
Preface: What's in the box?
The three boxes on Android are toolbox, busybox, and toybox. These boxes provide implementations for various basic unix commands - similar to those GNU Core Utils provides on various Linux distributions.
toolbox was until recently the standard Android implementation. busybox is pretty much the standard box outside of Android, GPL-licensed, and has been ported to Android by many. toybox is a BSD-licensed alternative to busybox created by a former busybox maintainer. toybox first appeared on Android in M, and is slowly replacing the toolbox implementation for various commands.
These commands are mostly used by root apps, device scripts, and adb shell / terminal emulator users.
Consistency is key
Those who have been following me for a while know that I am not a fan of busybox. This is mainly because many users install it in such a way that it overrides the default commands.
Many root apps need to execute commands provided by these boxes. While the base command is generally the same, the accepted parameters, exact working, and output, will differ between implementations, and different versions of those implementations.
For example, around 4.3-ish days, there were at least three different versions of busybox common, with different levels of SELinux support, and inconsistent behavior for common commands.
Of course, a developer can call toolbox explicitly, and work around a user using an overriding install of busybox. Exact toolbox implementation can be matched with currently running Android version, and is thus easier to work with for developers. Apps with special needs usually come with their own copy of busybox so they can predict exactly what a command does - but this is overkill for most apps.
Mmmmm?
Since toybox's introduction in M, calling toolbox explicitly stopped being reliable, as commands were being removed from toolbox as the default implementations shifted to toybox.
Unlike toolbox though, toybox provides a list of supported commands, so there was still a way to avoid the dreaded busybox, and get to a more predictable (even if not generally consistent) implementation, be it toolbox or toybox.
At this point, you might be starting to suspect that supporting all this from a root app supporting years old Android versions up to the most recent is a painful affair.
Mount Doom
(or: why apps fail to remount /system read/write)
And so, we finally arrive at the perils of the mount command. Even on M, toybox had a mount implementation. In many cases, this implementation would just segfault at you without doing any work. No matter, toolbox still had its mount command, and that still worked.
Unfortunately, on N Preview, toybox mount has replaced toolbox mount as the default implementation, and even worse, toolbox mount has been removed, so you cannot fall back to it. While toybox mount seems to have gotten rid of its saga of segfaults, it still doesn't actually work all that well. Not nearly as well as the toolbox implementation it has replaced, at least.
One of it's issues is that it doesn't understand relatime - and that's not even a joke. As most mounts are relatime by default, when a remount is performed, first it checks the current mount flags, finds relatime, and then panics.
In SuperSU v2.70, this has been countered by setting the (by default) read-only mounts to use noatime, which toybox mount does understand.
With this fix applied, the following remount commands will work:
((/system/bin/)(toybox )mount -o rw,remount /system
((/system/bin/)(toybox )mount -o rw,remount /path/to/device /system
Unfortunately, this does not cover all the popular forms of the mount command as used by various apps, and as such, many root apps will fail to remount /system, and thus will not be able to modify it. These apps will need an update, as I don't see an easy way to work around this.
Let's start with the /path/to/device parameter. The actual (g)libc/bionic mount(2) syscall that is ultimately called generally ignores this parameter completely in case of a remount, but various mount utilities do not.
For example, on Android 2.x the parameter had to be both present and correct, while on 3.x and early 4.x, it only needed to be present. On later versions, it can be present or omitted. As a shortcut, many apps actually use:
mount -o rw,remount /system /system
With toybox, if you specify the /path/to/device parameter, it has to be correct again as well. An interesting note on that is that toybox's mount command's output differs from toolbox as well, and may actually resolve symlinks in its output - so you actually need to examine /proc/mounts instead to get the correct /path/to/device (otherwise you'll get a busy error).
As toolbox prefixes will now also no longer work to perform the remount, the following commonly used mount constructs by root apps will fail:
(/system/bin/)toolbox mount -o rw,remount /system
(/system/bin/)toolbox mount -o rw,remount /path/to/device /system
(/system/bin/)mount -o rw,remount /wrong/path/to/device /system
If you're a root app dev, you know what you need to fix. If you're a user, this is the reason many root apps will fail manipulating /system on N Preview right now.
Excellent write up Señor Fire!
Perfect explanation :)
+Chainfire I'm a naive user. How easy is this fix, and what are your expectations for adoption?
Brilliant Composition! The timings cool because, I've actually just finished reinstalling busybox a few hours ago and wondering, why doesn't Google simply include the binaries by default like nearly every other Unix based system? This also goes along with my feeling that the ability to run commands as the root user should be included in developer options and bundled with improved security developed directly by Google with an access manager. Failsafe bootloaders are yet another thing that should have happened years ago. It seems the large companies have forgotten all they've learned creating standard computers for nearly 50 years and started over completely with mobile devices. Anybody who owns a Windows PC can run commands with system level privileges easily without any modification. Why can't one do the same on a mobile device I own? I've spend quite a bit of time modding my Nexus's software to obtain what I should have the first time I turn it on. Now I'm not mad or anything because I love computers and enjoy it but, my point stands.
It's getting too complicated for a guy who just wants to root his Nexus 6P and be able to install updates without re rerooting or going though other hassles.
Another version of Android, Users = :) Devs = :( too many versions. Thanks for the work for us to keep root.
great read :good:
Just curious. You say you don't like busybox because it isn't consistent or reliable on how it is applied. Also I have been reading it must now be installed to /su/xbin. The only one that I know of that does that (I'm a convenient recovery flashable package) is osm0sis's version found here . .
http://forum.xda-developers.com/showpost.php?p=64228091&postcount=420
Is this infact the correct implementation from your perspective?
+Norman Hirsch It was a pain trying to root fresh Marshmallow. It was a custom kernel or no root. I still use a custom kernel to disable encryption. Encryption is great but, it makes my data too delicate when I'm modding.
Oooooo
Skipsoft made root easy but keeping it updated each month is a pain.
So can we use toybox instead of busybox? ... And it it compatable with apps that use toybox?
+Anthony DelliGatti most root app devs will pick this up and fix it, no doubt. It's not complicated, just annoying. This is why I document it here.
+Norman Hirsch this information is mostly for the interested and the devs - as a root user, you may only notice that some root apps may not work until they're updated by the authors.
+Christian Hayden it's the best I know of at this time
But, thse days when a mobile device has similar or even more computing power than the average mid-level laptop, plenty storage and plenty randomly accessible marshmallows :D there is room for a full blown GNU stack beneath Android.
I tweaked my devices to be almost as my computers but, at times, I really feel the differences...
A real libc mount with libuuid, libselinux and the rest.
Yeah... I dream of the real GNU beef...
Never liked boxes... not even boxes inside 16Mhz, 2Mb ram and 64Mb flash boxes...
And now, with Gigs of ram, several gigahertz multi-core SoCs... ugh... ;))
+Pașca Alexandru That would be fantastic. But we can only hope for it, at this time.
+Chainfire - Excellent write-up. Thanks for pointing this out - I believe I will not install busybox on my N6 this go around, now being armed with this knowledge.
+Chase K I think because like most embedded devices, /system space is at a premium, sometimes with only ~100-200M remaining even as delivered. Each discrete component (ls, ps, top, ifconfig, etc) takes space requiring a binary (compiled/linked) so that something like busybox starts to look pretty brilliant where all you have is a single binary that has basic (and I do mean basic) functionality included, but that allows you to use commands like mount, ls etc, that are just soft-links to the busybox binary and no extra space is required.
Oops: Looks like this thread has run it's course, but as usual , I got here late ..
+Mickey Stein Better late than never?. You make a good point especially considering the hardware early versions of Android were developed for. Now of course we could easily integrate it into our 128GB devices but, I'm sure thats extremely low on Google's priority list.
+Chase K Not necessarily. It wasn't very long ago (6 months?) that the Nexus 6 was running out of room on /system because, well, the OS had grown, and there was a lot less than 100M left - in fact, IIRC, the original MM build left even less than 35 MB, with some users reporting that recovery tools were reporting as low as 16 MB or less. This made editing files with the older /system based su to gain root privileges hairy, as mounting /system RW was breaking the OS.
+Chainfire I have an issue with mounting..
"mount -o rw,remount -t ext4 /system" works perfectly when I have rooted by Nexus 5x, as you mentioned here.
However, I have some 5x devices on android N which I don't want to root.
I have made a custom boot image to run some specific services, and I try to run the same mount command in the init context as root user, and I get permission denied:
mount: '/dev/block/platform/soc.0/f9824900.sdhci/by-name/system'->'/system': Permission denied
This is the output of the id command when I try to run it from the same context in the init.rc service:
uid=0(root) gid=0(root) groups=0(root) context=u:r:init:s0
..which is the same output I get on a rooted 5x.
I have copied the sepolicy from your root image but it doesn't seem to help.
Anything I am missing here?
I have also edited the fstab in the bootimage to add the noatime option to the system partition.
You r hilarious
+John L Galt سلام عليكم ورحمة الله وبركاته
انا عندي مشكله في الاكس مود
What about /proc unreadable? It can work only through su now?
That also makes busybox a bitch to remove. As it overrides tool/toybox
Also fun, on Android N (Google Pixel) mount -o remount,rw produces 'Device or resource busy' while mount -o rw,remount works. Order of those options should not matter (vs GNU Linux).
As a user with su permission i am not able remount system partition as read write.
Any help would be appreciated.
Android 7.0 Kernel 3.18