RapidReboot: monitor apache too?

[isoHunt]

I have a feature request: I know SB ping servers on a timer, and if pings don't go through it is "rapidrebooted".

My problem is that my server sometimes locks up (for reasons I'm still trying to determine, I think it's certain MySQL queries), with all the daemons dead (apache, ssh, etc.) but pings still go through. So the server remain locked up w/o getting rebooted.

So a nice feature would be to either:

1. Monitor daemons (apache) rather than pings to determine if a server is "up"

2. Give users the ability to automate rapidreboot. So in essense let users monitor and rapidreboot their own servers. Perhaps this can already be done with some PHP tricks, but the my.serverbeach.com site is HTTPS and needs certificates and POST submission and all that, and I'm not sure if automated rapidreboot control in PHP would be easy.

[Tino Didriksen]

Have a cron job test connections to the local services, and if one doesn't answer force a restart of the service.

Or find and install one of the countless service monitoring programs.

ServerBeach cannot rely on whether the machine replied to HTTP requests to see if it's online, since some servers don't even serve HTTP...the only common denominator that all servers do run is ping.

[awsolutions]

Agreed with tino! I have a couple servers that don't run http! You cannot gauge by that. You will need to find some software that will run a cron and do it for you.

This is a cheap dedicated unmanaged server company - I am happy with having the big fat pipe to the internet and all those GB's! -

[isoHunt]

cron won't work coz when the server locks up, all daemons are "hanged", ssh etc., which would include cron. Therefore the brute force solution would be to force reboot from an external source.

I understand not everyone runs HTTP, but it would be nice to extend the great RapidReboot functionality 1 step further to allow any authorized, external computer to autonomously rapidreboot the server. I know how to monitor services on the server, but I lack the ability to automatically (and easily) reboot the server when certain services are down.

[Vogue]

Why try to rig a bizarre work-around for a problem that shouldn't be there in the first place?

Instead of trying to reboot the server every time it locks up... figure out WHY it locks up. Since you're running SSH and Apache, I'm going to assume it's Linux. (Although it could be windows...)... but there's no reason why you shouldn't have an uptime of months. There's rarely reason to reboot. I would find out why it locks, it could be a hardware problem. Hardware problems are just about the only reasons I've ever had Linux totally lock up. Things will crash... but rarely does the entire system lock.

Just a thought...
ben

[awsolutions]

Hello,

Usually, if you had good hardware and such a linux box will never lock up completely. Only child proccesses will "lock up". In that case a cron daemon would work and can be set up to restart the child process.

[isoHunt]

yes it's linux, ok i'll try cron

i filed a hardware check support ticket but no response yet

it's prob. some mysql query, i'm now continuously checking "hanged" mysql processes and kill them

[awsolutions]

Iso,

How do you know they are hung? Mysql entirely stops responding? I ask, because mysql child proccess are suppose to hang around by default fro something like 3000 seconds after a query unless the programmer has specified the mysql_close() function in the script.

Hope this helps!

[isoHunt]

the queries the site runs are not simple, but should not run for more than a minute. I just have a script that "kill" any Query process running for more than a minute.

There r Mysql timeout directives, but I couldn't find a "query timeout"

the problem is i don't think it's supposedly hanged queries which hang the server, as such queries should have been killed and the last server uptime was 12 hours. Perhaps i should do a log of every single query ran, and look for the last entry before the server hanged.

btw, i'm a semi linux noob so spare me ;)

[awsolutions]

Query timeouts are set either in the my.conf file or the mysqladmin utility not in scripts themselves.

[Nestor]

MySQL & PHP, it's best to close the link once you've finished using it in your scripts:

$link = mysql_connect(,,tothedb);

...

mysql_close($link);

Not sure why you would think of leaving a connection open but it's just good practice to close what you have opened.

[isoHunt]

for timeout directives in Mysql 4.0.13, there are

slave net timeout
wait timeout
connect timeout
interactive timeout
net read timeout
net write timeout

but no query timeout. i know it's in my.conf

i do close mysqlconnect's, it is in case some "runaway" process don't terminate for over a minute