Compilation speed test

In the following article We are going to test what is the optimal N for make -jN.

Testing environment

Debian GNU/Linux 6.0 amd64 (cat /proc/cpuinfo):

vendor_id       : GenuineIntel
cpu family      : 6
model           : 15
model name      : Intel(R) Core(TM)2 Quad CPU    Q6600  @ 2.40GHz
stepping        : 11
cpu MHz         : 2400.000
cache size      : 4096 KB

Cpufreq is enabled, so cores are dynamically (and separately) scaled 1600 - 2400MHz using ondemand governor.

Total amount of free RAM: 3958MB

Compiler is custom made (crosstool-ng) cross-compiler, which generates armv5te code.

System is running the custom-made kernel (uname -a):

Linux mbw 2.6.38.3 #8 SMP PREEMPT Wed Apr 20 22:13:07 CEST 2011 x86_64 GNU/Linux

We are cross-compiling linux kernel 2.6.38.5 configured as follows:

make ARCH=arm versatile_defconfig
make ARCH=arm menuconfig 
Kernel Features  --->
  [*] Use the ARM EABI to compile the kernel

Other options left in their default states.

System is not doing anything else while compiling (only standard system services are running).

For the first time build is run manually:

time make ARCH=arm CROSS_COMPILE=arm-none-linux-gnueabi- 2>&1 | tee run1.txt

All accessed files are now in cache.

The test

The sequence of following commands is run:

for n in 1 2 3 4 5 6 7 8; do
  for i in 1 2 3 4; do
    make ARCH=arm CROSS_COMPILE=arm-none-linux-gnueabi- clean
    /usr/bin/time make ARCH=arm CROSS_COMPILE=arm-none-linux-gnueabi- -j$n 2>&1 | tee run${n}_${i}.txt
  done
done

We compile the kernel using 1 - 8 (8 is two times number of CPU cores) simultaneous jobs 4 times each. Execution times are measured using time command. Then average is taken from 4 measurements.

The Results

N cpu time [s] system time [s] real time [s] cpu usage [%]
1419.3537.95462.6698
2463.8241.75257.65196
3449.4441.48168.89290
4426.5940.06121.61383
5429.8439.63122.37384
6432.0739.89122.84384
7435.1439.66123.36385
8437.4940.07124.56383
  • N - is number of jobs executed simultaneously (-j N);
  • cpu time - the time spend by processor doing computations for started process and it's child processes;
  • system time - time spend by processor doing system calls (for measured processes);
  • real time - time measured from start to end command - this is the time we had to wait for kernel to be compiled (aka. coffee break time);
  • cpu usage - percents of cpu usage (max possible is 400%).

Conclusion

Best results are achieved when number of simultaneously started processes equals number of processor cores (or processors). No improvement is observed while starting more processes.

ostatnio zmienione: 2011/06/29 18:49