su - on sles 9 doesn't respect limits.

This can end up causing some good security holes.

If you put in a ulimit for higher files for a user named oracle (just an example) in /etc/security/limits.conf:

oracle soft nofile 2048
oracle hard nofile 65535


then run ulimit -a >~limits in cron. Your file limit is 1024 (the default on SLES).

Why? I'll tell you.

Crond does a su - to you user. ON SLES SU - DOESN"T RESPECT THE LIMITS IF THEY ARE HIGHER!!! I haven't tried lower yet.

If you login interactively your limits are fine. If you su - oracle as root, your limits are f-d up (back to the system default 1024).

This doesn't happen on RH/Fedora and as far as I can tell Debian/Ubuntu.

updating one package on sles 9

online_update --url 'my_local_update_server' --force -S patch-10903

or you can do what I do and write script that checks architecture and rpm -ivh http://installserver/sample-i386.rpm from a web server. The script is useable on RH or SLES for onesy twosey patches.

SLES still sucks

but this makes it suck less:

automatic update at your command-

It still doesn't do the right thing with a kernel (it upgrades instead of installs leaving modules broken and your currently running machine in bad need of a reboot), so it is dangerous in some ways.

Just FYI here is the magic:

ssh -n $HOSTNAME "which online_update && online_update -gVu http://servername/YOU/ && online_update -iV"

This executes the command "which online_update" and if that is successful runs online_update to download packages from your You server (yast2 can help you make one, that works) to the local box, then if that is successful online_updates from the packages on the local box. No other combination of switches appears to work update a machine via online_update. SLES needs to download then install.

RHEL/CentOS does the right thing with the kernel and only requires a "which yum && yum -y update" and you can run your own repositories if you use yum, like I do, so it is still better (if you use up2date, that is okay too- but yum does better with the repositories.

The reason I run "which commandname" is to avoid trying to yum a SLES box and online_update a CentOS.

I can feed the script file a list of servers and it will go patch the lot. You can save the output and have a list of patched boxes.

The benchmarks...

I promised some comparisions of SLES9 and why it has slow I/O. I haven't cleared releasing the application yet (it is a small peice of C code that opens as many files as you throw as an argument and then writes to those files). In place of that code, a nice workabel substitute is substitue a one large file write: "time dd if=/dev/zero of=/tmp/testfile bs=16k count=65536 ". You can also try reads, but that is more divergent based on caching the filesystem, if you bench reads, reboot between benchmarks (or otherwise flush all cache and buffers).

I've benchmarked this on HP servers (and a couple desktops). I've tried different filesystems, different kernel versions and different I/O subsystems (SCSI, SATA, ATA). The numbers pretty much go about the same way (except kernel version as you will see). Apparently Redhat backported the patch or knows about the bug and fixes it in their kernel.

Here are some benchmarks all on the same HP DL140 hardware (2.8 Xeon, 1G ram, sata drive, 11211 Bogomips):

Hardware	OS	FS Type	time a.out 1000	user	sys	Notes
dl140	SLES 9	reiser	5m15.042s	0m38.427s	0m8.303s	unresponsive after a few seconds and well after test ls will hang
dl140	SLES 9	ext3	5m31.042s	0m38.427s	0m8.303s	unresponsive after a few seconds and well after test ls will hang
dl140	SLES 9	reiser	3m53.546s	0m44.687s	0m3.052s	2.6.9 kernel unresponsive
dl140	SLES 9	reiser	2m51.070s	0m44.687s	0m3.052s	2.6.16.1 vanilla kernel responsive.
dl140	FC 5	ext3	1m52.354s	0m44.515s	0m7.124s	responsive.
dl140	FC 5	ext3	1m52.354s	0m44.515s	0m7.124s	run 5 instances still responsive

Centos/RHEL4 perform similarly to FC 5. You can see reiser is slightly faster than ext3 on the SUSE test, but it doesn't matter as they are blown away by a good kernel. The interesting thing is SuSE/Novell didn't really want to hear about this when I tried to open a ticket. I'll be trying again. I have benchmarks from DL380's and a reproduceable method, that doesn't rely on the C program, just dd (you can also produce the bug with sort and some other ways).

The nice thing here is we can double our performance by going to a new distribution.

The dismal thing is a 500$ desktop (1.7 P4 Celeron 512 Mb of ram, ata drive) with FC 5 was able to perform on par with a DL385 (~10,000$) dual Opteron, 8GB of ram, 6 drive SCSI raid array, on the first run. And able to beat the Opteron with multiple runs. That means this silicon garden is poorly optimized and utilized.

SLES9 kernel compile

I'm still getting to the bottom of why I/O on SLES 9 is so pathetic (future article with benchmarks on various HP server hardware). But I'm testing many kernel compiles and there wasn't a good recipe for SLES9 and vanilla 2.6 kernels. I've tested this with 2.6.16.1, 2.6.9, 2.6.12.6.

Sles 9 kernel build:

make sure you have the kernel source on the machine and gcc.

yast2 -i kernel-source gcc

then get the linux kernel and untar it (replace the kernel.org/foo.kernel stuff with a real path to a kernel):

wget http://kernel.org/foo.kernel.bz
tar -jxvf linux-2.6.6.1.tar.bz2

then move the kernel to /usr/src/

mv kernel-foo /usr/src/

remove old kernel build sim link:
rm /usr/src/linux

make new symlink to the new kernel:
ln -s /usr/src/kernel-foo /usr/src/linux

If you want to build a kernel with similar options to a SLES Kernel (building one with the old config), then you need to get and old .config file.
If you are running a plain SLES kernel:

zcat /proc/config.gz > /usr/src/linux/.config

or you can copy the .config file from a SLES kernel source:
cp /usr/src/linux-2.6.5-7.244/.config /usr/src/linux/.config

Now you have one more interactive part:
make oldconfig

That will ask questions about new options in the kernel that are not covered by the old .config only. You may be able to take all defaults.

I will break down the next steps with commentary, but the rest does not necessarily need intervention, so I usually stack them on a command line (see below):

Clean up configs:
make clean

Make a bootable linux image:
make bzImage

Compile loadable kernel modules (these are the same as drivers usually):
make modules

Install said drivers:
make modules_install

This makes a module dependency map and if it isn't done right you might not boot. The 2.6.16.1 will need to be replaced with the kernel version.
depmod -ae -F System.map 2.6.16.1

Install everything:
make install

So a line to do all of the post interactive stuff with a 2.6.12.6 kernel would look like:
make oldconfig && make clean && make bzImage && make modules && make modules_install && depmod -ae -F System.map 2.6.12.6 && make install

And then you might have to wait for a while. If you use grub as a bootloader, you shouldn't have to do anything else to run your new kernel except reboot. This whole process is so much easier than 2.0, 2.2 or 2.4 kernels. It still isn't as easy as apt-get upgrade kernel or yum update kernel or emerge or whatever, but such is the price you pay for SuSE.

SLES 9 IO problem

There are some aphids in the garden right now.

I have an interesting problem with SLES9. I know it isn't my favorite linux distribution either, but it is pretty good for business that like support and it is a great java platform.

The interesting problem is with I/O. On a default install on various platforms, if you run a high I/O job the system becomes unresponsive. I believe it is a problem with the kernel version, because none of the other Linux distributions I've tried have it.

SLES9 is stuck in a heavily patched 2.6.5 kernel, and if you look through the kernel changelog, you'll see plenty of virtual memory and I/O improvements.

Today, I think I'll try a vanilla 2.6.12 or later kernel- because if my suspicions are right, somewhere between 2.6.5 and 2.6.12 the bug is fixed.