Virtual Guru's Blog – Home of Virtualization Workshops

August 31, 2010

12minutes: Which compression is best for sharing Chromium OS images ?

Filed under: 12minutes, ChromeOS — Tags: , — natiku @ 6:47 pm

12 Minutes: Chromium OS Which compression is best for sharing Chromium OS images with my friends?

Motivation – Most my fiends have strong download size limits

Well, in Europe is common then you have from one hand relatively fast DSL connection, but from other hand you have strong monthly transfer limits in size of single gigabytes. In this situation each megabyte counts and if I wont to share my Chromium OS images with friends which don’t have time to compile itself (or even don’t know how to do it) i need to focus on possible size savings.

Various compression of default Chromium OS image

I use for it this comparison my Chromium OS development system based on Intel Atom Used $250 HW for compiling and evaluation of Chromium OS and my previous experience with OS compressed images like Mapping possible Space and Time savings by using SMP aware (threaded) compression Utilities ? or Compressed ZFS Stream sizes in various compressing programs and OVF

Also some Intel Atom platforms compression benchmarks can be useful:

Intel Atom N550 vs. N450 : 7Zip Benchmarks, Dual Core vs. Single Core PineTrail Fight

NOTE: Chromium OS encrypt sensitive user information!

Compressing a file after encryption is inefficient. The ciphertext produced by a good encryption algorithm should have an almost statistically uniform distribution of characters. As a consequence, a compression algorithm should be unable to find redundant patterns in such text and there will be little, if any, data compression. In fact, if a data compression algorithm is able to significantly compress encrypted text, then this indicates a high level of redundancy in the ciphertext which, in turn, is evidence of poor encryption.

So already used (after login and browsing) Chromium OS image can be much larger then default one you get right after build process.

Measuring various compression of default Chromium OS image

Uncompressed size of default right after Chromium OS images is 1802MB, what minimal compressed image I can get ?

dd if=chromiumos_image.bin bs=16M | pv -B 16M -s 1802M | gzip \
   -9 >chromiumos_image.bin.gz
Time: 08m38s Size: 234MB

dd if=chromiumos_image.bin bs=64M | pv -B 64M -s 1802M | zip \
   -9 -q >chromiumos_image.bin.zip
Time: 08m51s Size: 234MB

dd if=chromiumos_image.bin bs=16M | pv -B 16M -s 1802M | bzip2 \
   -9 >chromiumos_image.bin.bz2
Time: 11m09s Size: 207MB

dd if=chromiumos_image.bin bs=64M | pv -B 64M -s 1802M | 7z a \
  -mx9 -mmt -ms=on -md=64m -si -bd chromiumos_image.bin.7z
Time: 1h24m05s Size: 143MB

Notes:

– 7z Utra 8 threads on AMD 4 core 2Ghz CPU take 6m12s
– Latest 7z versions use on Atom CPU hyper threading by default,
it will be interesting to make test with 1 enabled thread and with
maximum free memory because I have only 1GB RAM, 7z Utra
mode is known to be memory intensive (CLI mode with minimum services)

Retesting 7z in CLI when I have 750MB or more free memory

Text mode on Atom 230/1GB RAM and also after upgrade to D510/2GB RAM.

/usr/bin/time -v 7z a -mx9 chromiumos_image.bin.7z chromiumos_image.bin
Atom  230/1GB RAM = Time: 25m35ss CPU usage: 149% Size: 143MB
Atom D510/1GB RAM = Time: 12m55ss CPU usage: 355% Size: 143MB
Atom D510/2GB RAM = Time: 12m55ss CPU usage: 355% Size: 143MB

/usr/bin/time -v 7z a -mx8 chromiumos_image.bin.7z chromiumos_image.bin
Atom  230/1GB RAM = Time: 23m57ss CPU usage: 155% Size: 147MB
Atom D510/1GB RAM = Time: 11m25ss CPU usage: 384% Size: 147MB
Atom D510/2GB RAM = Time: 11m10ss CPU usage: 378% Size: 147MB

/usr/bin/time -v 7z a -mx7 chromiumos_image.bin.7z chromiumos_image.bin
Atom  230/1GB RAM = Time: 23m54ss CPU usage: 138% Size: 147MB
Atom D510/1GB RAM = Time: 11m28ss CPU usage: 379% Size: 147MB
Atom D510/2GB RAM = Time: 11m00ss CPU usage: 379% Size: 147MB

/usr/bin/time -v 7z a -mx6 chromiumos_image.bin.7z chromiumos_image.bin
Atom  230/1GB RAM = Time: 18m22ss CPU usage: 162% Size: 153MB
Atom D510/1GB RAM = Time: 05m22ss CPU usage: 356% Size: 153MB
Atom D510/2GB RAM = Time: 04m57ss CPU usage: 366% Size: 153MB

/usr/bin/time -v 7z a -mx5 chromiumos_image.bin.7z chromiumos_image.bin
Atom  230/1GB RAM = Time: 18m22ss CPU usage: 148% Size: 153MB
Atom D510/1GB RAM = Time: 05m13ss CPU usage: 367% Size: 153MB
Atom D510/2GB RAM = Time: 04m51ss CPU usage: 368% Size: 153MB

/usr/bin/time -v 7z a -mx4 chromiumos_image.bin.7z chromiumos_image.bin
Atom  230/1GB RAM = Time: 07m33ss CPU usage: 095% Size: 186MB
Atom D510/1GB RAM = Time: 04m37ss CPU usage: 095% Size: 186MB
Atom D510/2GB RAM = Time: 04m22ss CPU usage: 097% Size: 186MB

So on my Atom machine is best to use:
– For night unattended builds is best to use 7z -mx5
– For stuff generated on demand bz2.

7zip (7z) can generate traditional formats like BZIP2 using threading – ALL CPU CORES

Parallel BZ2 on ATOM D510/2G RAM

/usr/bin/time -v bzip2 -5 -k chromiumos_image.bin
Atom D510/2GB RAM = Time: 07m39ss CPU usage: 099% Size: 212MB

/usr/bin/time -v 7z a -mx5 -mmt=4 chromiumos_image.bin.bz2 chromiumos_image.bin
Atom D510/2GB RAM = Time: 04m65ss CPU usage: 378% Size: 209MB

/usr/bin/time -v 7z a -mx5 -mmt=8 chromiumos_image.bin.bz2 chromiumos_image.bin
Atom D510/2GB RAM = Time: 04m49ss CPU usage: 380% Size: 209MB

Compression of Chromium OS used image

Most space will be occupied by Web cache – many images with very low compression ratio.

1. Boot default (empty) Chromium OS image
2. Create a new user
3. Browser web a lot (webs with news - lot of pictures)
4. ~ 107MB used
  Ctrl-Alt-T
  shell
  sync
  sudo -s -h /home
5. Shutdown system
 sudo init 6
6. Get image = first 1802 MB
 dd if=/dev/sdb of=chromiumos_image_used.bin bs=1MB count=1802
7. Compress it
 /usr/bin/time -v 7z a -mx5 -mmt=4 chromiumos_image_used.bin.bz2 chromiumos_image_used.bin
 Atom D510/2GB RAM = Time: 05m13ss CPU usage: 371% Size: 257MB
Advertisements

Leave a Comment »

No comments yet.

RSS feed for comments on this post.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Blog at WordPress.com.

%d bloggers like this: