Bash scripting help - compile kernel

[Patrick]

I hate to do this, but I am trying to figure out how to bypass the make menuconfig manual entry before running the Linux compile.



I am trying to add this to our Linux benchmarking but it would be helpful if I could bypass this step and have the .configure file made via defaults. I have pretty much everything else scripted at this point and have begun profiling boxes.

If anyone has ideas, it would be greatly appreciated.

 

[Patrick]

BTW if anyone wants a starting point (not fancy yet):

#!/bin/bash

mkdir kernel
cd kernel

wget https://cdn.kernel.org/pub/linux/kernel/v4.x/linux-4.4.2.tar.xz

tar xf linux-4.4.2.tar.xz

sudo apt-get install -y git fakeroot build-essential ncurses-dev xz-utils libssl-dev bc
cd /home/ubuntu/kernel/linux-4.4.2
make menuconfig
time make -j16

Notes with the above:

• Directory structure assumes Ubuntu LiveCD.
• The -j16 should have the 16 replaced by number of threads in the system.

 

[TuxDude]

Just replace 'make menuconfig' with 'make defconfig'

 

[mstone]

Take a look at the top level README for various configuration options, like make defconfig.

Not only do I remember configuring the kernel before menuconfig was written, I remember when it fit on a 5.25 floppy. Man, has it gotten big.

 

[Patrick]

@TuxDude THANK YOU!!!!!!! (add about 100 more '!' to the end of that)

 

[Patrick]

@mstone and @TuxDude do you know if defconfig is equivalent to going to menuconfig and just doing save?

I just ran it on two machines and they ran the compile in about 1/7th the time as when I used menuconfig and just saved.

 

[TuxDude]

Can make the final 'make' command automatically use the right number of threads as well, replace the final line with:

time make -j`grep --count ^processor /proc/cpuinfo`


And I'm afraid I'm not sure what menuconfig will spit out if you just open it and hit save - a quick look at the documentation does not describe what that would do. Doc's do say that defconfig will use the default options for the current architecture.

Edit: Why not compare the two configs?

rm .config
make defconfig
mv .config config.defconfig
make menuconfig
<save and exit>
diff -u config.defconfig .config

 

[Patrick]

Yea that makes sense. For the thread count in the current version I was just using numcpu which was a bit easier. In the main L-B script we have the variable already defined.

It seems like the differences are quite large.

Maybe I just save the .config file I want and download it.

 

[mstone]

Did you do a make clean in between runs? it'll be a lot quicker if it doesn't have to do much. For your purposes I'd think a make allmodconfig would be ideal. But if you want to just use a standard .config then make olddefconfig would work.

 

[TuxDude]

Ya - just saving a copy of the .config the way you want it makes things easy. That would also make it very easy if you want to copy the config from an ubuntu kernel, or a redhat one, etc. Or you can try some of the other configuration commands, as @mstone mentioned in his post. Maybe 'make allmodconfig' (I see in the diff you attached a LOT of the changes are configuring modules), or if you just want a long repeatable compile, maybe 'make allyesconfig'

 

[Patrick]

I am going to try make allyesconfig but I think just saving a standard config is a cleaner option.

@mstone I am doing this on a LiveCD so just rebooting so I can get a consistent baseline.

 

[Patrick]

Ok I have that down to something that is consistent off the Ubuntu LiveCD now.

 

[sean]

You could also build the config the running kernel is using. Going off memory, the running config, assuming you haven't done major package updates, is at /boot/config-$(uname -r). Copy that to to .config in the source directory and run make oldconfig.

 

[Patrick]

You could also build the config the running kernel is using. Going off memory, the running config, assuming you haven't done major package updates, is at /boot/config-$(uname -r). Copy that to to .config in the source directory and run make oldconfig.

@sean that is a great idea. I ended up just saving a .config and use wget. That way it at least stays consistent as people use different versions. Likely will keep the work done today unless there is a big reason not to.

 

[OBasel]

Your scripts are ugly IMO.

 

[Patriot]

Your scripts are ugly IMO.

It will get cleaned up and shoved in linuxbench as a module before long.... now if you call that ugly... thems fightin words.

 

[Patrick]

Your scripts are ugly IMO.

Sometimes "functional" is good enough.

I did start a python version of this yesterday that even iterates and will run multiple times (without repeating the download.)

Being a python neophyte I did not realize that raw_input() + a curl scriptURL | python was a recipe for disaster. Unlike with bash, you need to python <(curl scriptURL) if you have user input in the script.

 

[KioskAdmin]

I did start a python version of this yesterday that even iterates and will run multiple times (without repeating the download.)

Can we try that?

 

[Patrick]

Sure:

python <(curl http://files.linux-bench.com/lb/PyKCB.py)

I still need to figure out how to parse the result of each "time make" function.