If you do a lot of flashing of ROMs, kernels, and patches, one thing you’ll notice quickly is how large your backup folder becomes and how frequently you have to remove older backups. The question you may have asked is: why should this be the case? After all, data from one Nandroid to the next is not entirely different. APKs do not change from one ROM to the next, and neither do most of the binaries that make up our ROMs. Now, developer Koush of ClockworkMod has potentially fixed this.
The new ClockworkMod format simply backs up files from ROM to ROM into the same directory: blobs. Individual backups you create are links to files in the blobs directory. Files are identified by their SHA256 fingerprint, so if you backup a ROM with a version of Facebook and it’s updated in the next ROM, the original would not be overwritten. If they were the same, the file would be skipped from backing up 20+ megs of duplicated data, and a little link to the blob file would exist in the new backup.
Another advantage to this is that doing a fingerprint of a file and looking it up in a blobs directory takes significantly less time and processor power than compressing that file and writing it to storage.
When a backup directory is deleted, the blobs entries associated with that particular backup are removed at the next backup unless they’re in use elsewhere. This prevents you from having old ROM data you no longer want laying around clogging up your backups.
With backups for some newer phones reaching gigabyte levels and relatively little changing from one backup to the next, this could save a ton of space, backup time, and allow a wide date range of backups on your phone without stealing much space away.
The only issue I see with this is that these are not standalone nandroid backups, meaning if you just copy an individual backup directory to your computer that was made, you’re copying links to files on your phone. Deleting the backup from the phone will result in the blobs entries being killed, and moving the backup directory from your computer back to the phone would result in a lot of files not being found as the associated blobs were removed in maintenance. If that’s right, that means all your eggs are in one basket, and if you want to copy a small egg off of the phone you’re going to end up hauling a henhouse to your computer to get it.
Or perhaps there will be another way to do it. It’s an interesting concept, but I’m not sure how it will play out and whether it will be a pain or a godsend in the recovery world.
For those wondering, ClockworkMod 6 will be fully backwards compatible, so no need to worry about your old nandroid backups not being able to be restored. It’s unclear whether it can make nandroids that are compatible with existing recoveries, but you can expect existing recoveries to incorporate this into their code if it works out well.
It’s also unclear when exactly CWM 6+ will make its way to each device, but very soon is what I’m betting.
[Google+]
This is interesting. I like the concept, but I value being able to move Nandroids around individually.
I wish CWM would just compress stuff. I know it takes longer, but an extra few minutes for a few hundred extra megabytes of space is worth it, IMO.
Unix file systems usually* allow multiple hard links to the same file, so if CWM uses hard links to link to a file in an earlier backup, there is no difference between copying an old backup directory containing a specific file and copying a newer backup directory with a link to that file, you get the exact same file either way.
There are backup systems which take advantage of this, so I suspect CWM is following along.
* I can’t think of a file system in wide use that does not allow multiple hard links
unless I’m mistaken CWM generally stores backups on a fat32 partitioned SD card, so for the majority of users there are not going to be hard links – probably just a text file with filenames.
You are correct, sir. FAT32 on the microSD card it is.
I’ll give this a whirl if they bring it to the EVO 3D, but like I said above, I prefer self-contained Nandroids. We shall see.
I will give it a whirl when it comes out, but also not sure I want non-self-contained nandroids.
Actually, I don’t think you can call these nandroids. Not entirely sure who writes the definition, but something about a contained backup seems like it would be implied.
As nandroids are generally one large chunk files of partitions or a big ol’ zip file, these would not be it.
I wonder if there will be an application to move off a more normal nandroid to the computer… something that pieces together the information you need.
ooh, something I did not think about – fat 32. each one of the files is going to be written individually. That probably means ~300 APKs with 300 data elements and then (perhaps) thousands of entries for things like config files, libraries, shell scripts, etc. Let’s say the whole thing comes in at 1500 files.
FAT 32 cluster size for 32-64 GB is 16K, meaning a file of 1 byte is allocated 16K, a file of 15K is allocated 16K, and a file of 17K is allocated 32K. Let’s pull an average and say that each file written will “waste” 8K of that 16K. That’s 12 megabytes of wasted space right there.
Ok, 12 meg, nevermind, I was off by a factor of 10 on my “ooh ooh”.
frackin coffee needs to finish
Thanks for writing this! I was really confused that doing (what I thought) was a Nandroid backup then a Factory Wipe (and flashing CM9) wasn’t actually killing all the app directories and data etc.
I don’t know for sure if my “all the stuff is there still” issue is due to CM9 having Clockwork 6+ baked in, or other issues…