root device discrepency

[bow-wow]

I just got a new server, and I can't enable disk quotas because my root partition is mounted as /dev2/root (which does not exist). fstab says my root partition is /dev/hda3 but that does not seem to matter. I've found out that /dev2/root is a temporary device created by initrd, but I don't understand why it is created or what I can do to make disk quotas work. On my old server the root partition was mounted as /dev/hda3. Why is this one different? What can I do about it?

#/etc/fstab:
/dev/hda3 / ext3 errors=remount-ro 0 1
proc /proc proc defaults 0 0
/dev/hda2 none swap sw 0 0
/dev/hda1 /boot ext3 defaults 0 2

#/proc/mounts
/dev2/root / ext3 rw 0 0
proc /proc proc rw 0 0
devpts /dev/pts devpts rw 0 0
/dev/hda1 /boot ext3 rw 0 0

[knightfoo]

Try this:

# mount -o remount /dev/hda3 /

You might find that /dev2/root is really /dev/hda3 with a different name (same major/minor numbers).

It is probably different than your old server because the older Debian installer used a kernel that did not need an initrd. The newer installer used a standard Debian kernel, which uses an initrd.

-knightfoo

[bow-wow]

Thanks for your reply Knightfoo,
Unfortunately I've tried # mount -o remount /dev/hda3 / with no luck. I tried it again, just to be sure, but even though the command seems successful, mount and /proc/mounts still show:

/dev2/root on / type ext3 (rw)
proc on /proc type proc (rw)
devpts on /dev/pts type devpts (rw)
/dev/hda1 on /boot type ext3 (rw)


Also interesting: my development server, also Debian 3.0r2, has a nearly identical configuration with standard Debian kernel and on that box, the root filesystem is mounted as /dev/hda3.

Any other ideas?

[bow-wow]

I learned some interesting stuff, but unfortunately I'm no closer to solving this problem.

Now I know that the reason I need initrd is because ext3 support is not built into the standard Debian kernel, so an initrd must be used to mount the root filesystem, which is ext3.

Also, the reason I don't have this problem on my dev box is because I don't use initrd-- I am using the 2.4.18-bf2.4 kernel which does have ext3 compiled in and doesn't need initrd.

Alas, I checked my old server and it has the same exact kernel as the new server, 2.4.18-1-k7, and it is also using an initrd. BUT like I said before it has /dev/hda3 mounted on root. There is something about the initrd on the new server that is causing my root filesystem device to appear as /dev2/root after the system has booted. I just wish I knew what it was.

[bow-wow]

I've just converted my dev box to use the standard initrd requiring kernel insteat of the bf kernel. I apt-get install'ed kernel-image-2.4.18-1-686 (the closest I can get to my prod box, since I don't have amd hardware). Apt noticed that I didn't have an initrd and set me up. After all that I edited grub to add the new kernel image and it's initrd, matching almost exactly (aside from 686 vs k7) the config on my new SB server. I rebooted with the new kernel + initrd and found guess what.... my root partition is mounted as /dev/hda3.

At this point I'm convinced that something is not right on my new server. It is the only one exhibiting this /dev2/root weirdness.

[bow-wow]

I would really appreciate it if other SB debian server owners could show me what is in their /proc/mounts (same as what is returned by "mount" command). Also when was the server ordered/setup? I want to see if this is just my server or others as well.

[bow-wow]

I know there are other debian server people out there, unfortunately the debian forum isn't frequented much by anyone.

I've been hanging out on the debian IRC channel trying to get this problem solved but it looks like I've stumped everyone there too. I guess this is where I throw in the towel and open a trouble ticket. I hope SB can solve this problem.

[JonTrainer]

Here are my results on my PowerServer. I purchased it in January of 2004.

# mount
/dev/hda3 on / type ext3 (rw,errors=remount-ro)
proc on /proc type proc (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
/dev/hda1 on /boot type ext3 (rw)
tmpfs on /dev/shm type tmpfs (rw)

[bow-wow]

Thanks for the info Jon.

After some delay (due to the fact SB techs were for some reason unable to connect to my server, despite my firewall being set to allow them) they finally logged in and look at the issue last night. Unfortunately the guy that looked at it said he wasn't sure what was causing this problem and he is forwarding the issue to someone who knows debian better.

[greyskull]

I'm showing the same /dev2/root. I think I remember seeing this when I enabled some kernel option a few months back on one of my home boxes, but its late. What I do know, is that my lspci showed a VIA82CXXX controller, and the 2.4.18-k7 installed by default doesn't have the VIA82CXXX IDE driver built in the kernel or as a module. I'm not sure if that has any impact on the /dev2/root issue, but... If you do a 'dmesg | grep VIA' you'll probably see the driver complaining like mine.

I did an 'hdparm -tT /dev/hda' and saw some pretty miserable transfer speeds, so I checked to see if dma was enabled ('hdparm -d /dev/hda') which it isn't. I also can't seem to set it, probably due to the driver. Also, block transfers were off ('hdparm -m /dev/hda'). So, no DMA and ~ 7MB/sec, yuck... transfer was in the high 5MB/s before setting multcount to 16.

I'm about to do a kernel rebuild (to 2.4.26), and enable VIA82CXXX, which should fix things up and maybe I'll get a /dev/hda3 in the process.

[bow-wow]

Thanks for the info greyskull. I forwarded it on to SB support who are now working on this problem for me. It's not looking too good, as far as I can tell. The last message I got was a few days ago and they said they were escalating the issue to their lead engineer.

Also, hdparm -Tt on my server does indeed reveal miserable disk reads.

/dev/hda:
Timing buffer-cache reads: 128 MB in 0.38 seconds =336.84 MB/sec
Timing buffered disk reads: 64 MB in 10.17 seconds = 6.29 MB/sec

dmesg says this about the VIA chipset:
PCI: Using IRQ router VIA [1106/3147] at 00:11.0
VP_IDE: Unknown VIA SouthBridge, contact Vojtech Pavlik <vojtech@suse.cz>

[greyskull]

It turns out the VIA South Bridge on my box (VT8237) isn't well supported until 2.4.20+. You can get updated kernel-images from backports.org or from testing. Once I went to 2.4.26-1-k7 I had 56MB/s transfer, which rocks.

I think the /dev2/root is because /etc/mtab is linked into /proc/mounts. ls -al your /etc/mtab on the dev box and then on your box at SB. I'm not sure is you can just break that link depending on how the kernel was built. The debian-devel list has a looong thread on it. I saw some info on using quota with a symlinked mtab here (try a -a when starting the qutoa daemon):

http://www.roedie.nl/lids-faq/html-multiple/user-quota.html

[greyskull]

You can just rm /dev/mtab and reboot, and voila:

one:~# df
Filesystem Size Used Avail Use% Mounted on
/dev/hda3 75G 2.9G 68G 5% /
tmpfs 505M 0 505M 0% /dev/shm
/dev/hda1 76M 14M 63M 18% /boot

[bow-wow]

Thanks a million greyskull. You were exactly right. /etc/mtab was a symlink to /proc/mounts and removing that symlink and rebooting resulted in a new mtab being created, and caused mount, df, etc to correctly show /dev/hda3 mounted on / . That allowed me to solve the initial problem, enabling disk quotas. Yea!

Thank you!!! :beer:


Now I just need to solve the slower than crap disk access. I guess I might have to bite the bullet and upgrade to a newer kernel.

[Jeremy Howard]

Has anyone solved the DMA performance problem with these Debian servers? I'm very surprised that Serverbeach supply them in this degraded state.

If anyone has solved it, can you tell us exactly what you did? I've tried compiling a new kernel with the VIA module, but I can't get it to boot (I guess I forgot to compile something in - but without console access it is nearly impossible to diagnose :( ).

[greyskull]

You might just want to play it safe and apt-get install kernel-image-2.4.26-1-k7, as opposed to going the cutom route. After dpkg finishes its stuff (you'll have to Y the postinst script initrd warning) make sure your /boot/grub/menu.lst is correct, which it should be. Mine looks like:

title Debian GNU/Linux, kernel 2.4.26-1-k7
root (hd0,0)
kernel /vmlinuz-2.4.26-1-k7 root=/dev/hda3 ro nomce
initrd /initrd.img-2.4.26-1-k7
savedefault
boot

The bottom line is the 2.4.18 kernel is going to perform horribly with the latest VIA chipsets, and you have to go 2.4.20+. Maybe they'll hook you up so you don't have to burn a ticket if it goes south. I wasn't so lucky on my first attempt either..

[Jeremy Howard]

Heh. I've just come back to report that upgrading to 2.4.26 works beautifully - but I see Greyskull has beaten me to it!

RAID rebuild now taking 10x faster than with 2.4.18!

Hey Serverbeach - I really think you should fix this problem with new Debian installs; it's a major performance problem.

[Jeremy Howard]

Got a message back from Serverbeach support - apparently they are aware of the VIA DMA problem already and are looking into it. Might be a good idea to inform customers too.

[bow-wow]

I've also upgraded to 2.4.26, but I do wish I'd known about the DMA problem sooner. Until I had this /dev2/root problem I had no idea my disk access was so pitifully slow.

[alee]

The disk speed seems okay on my server:

# hdparm -tT /dev/hda

/dev/hda:
Timing buffer-cache reads: 1240 MB in 2.01 seconds = 617.93 MB/sec
Timing buffered disk reads: 172 MB in 3.00 seconds = 57.30 MB/sec

hdparm -tT /dev/md0

/dev/md0:
Timing buffer-cache reads: 1256 MB in 2.01 seconds = 626.22 MB/sec
Timing buffered disk reads: 164 MB in 3.03 seconds = 54.13 MB/sec

I compiled a kernel myself and converted whole root filesystem to raid1.

It's running 2.6.7 now:
# uname -r
2.6.7-k7

Haven't tried 2.4.26, but I think 2.6.7 may be solved your problem too.

[bow-wow]

Originally posted by alee
Haven't tried 2.4.26, but I think 2.6.7 may be solved your problem too. Yes, I'm sure it would. The problem is the default install from serverbeach for these servers gets kernel-image-2.4.18 which has no support for this chipset and keeps you at disk read rates of ~6MB/sec unless you upgrade to a newer kernel with support for this VIA chipset.

[knightfoo]

Originally posted by bow-wow
Yes, I'm sure it would. The problem is the default install from serverbeach for these servers gets kernel-image-2.4.18 which has no support for this chipset and keeps you at disk read rates of ~6MB/sec unless you upgrade to a newer kernel with support for this VIA chipset.

Hmm .. that is pretty inconvenient. The 2.4.18 kernel image is the latest available for Debian Woody, and I know how much people freak out when we start doing something non-standard. I think installing Debian Testing (Sarge) is out of the question since I get the impression that most people don't want to use testing/unstable. It is much easier to upgrade to testing/unstable than it is to downgrade to stable.

Maybe we can wedge the IDE chipset drivers into the standard kernel-image package without breaking too much stuff. :)

-knightfoo

[alee]

That's what backports.org stand for.