On a second thought, you could use VirtualBox to achieve the two server setup I described
without the need of an actual physical server. See: http://www.virtualbox.org/
|
Lets say that a libc upgrade for some reason was the result of a crash because some libraries
where using an old version along with a new one. Then the simple act of the system crashing
and rebooting would fix it. Major compatibility changes with libc would be listed under
another package name, so that's not an issue either.
If such events do happen then it will be a result of a bug, you can't protect yourself
unknown bugs if you upgrade regularly.
It seems to me that you need to look at your hardware setup rather than your software one.
You insist on very little downtime but want every service to use the latest libraries. As you
have discovered dbus is a service that can suffer from a reload, and that in itself is a
major design flaw in dbus and is (from my experience) rarely seen in any other services.
You could benefit from a two server setup, this would half the load and in the event of a
failure or a software upgrade you can route all traffic to the other server. Then simply
introduce the server back when you have either resolved the issue that caused it to crash or
when you know that your server is functioning correctly after a software upgrade by the use
of a test suite. Once a successful software upgrade on one server, you would continue to do
the same on the other.
|
I understand but on a very important production server you can't introduce instability,
especially when tools like Uptrack from ksplice.com mean that a server may be up for several
months at a time. From what I've seen there's no immediate issue with something like
libc upgrading but at some undetermined point later on a critical service
crashing isn't welcome, especially if you then spend ages trying to diagnose it.
Also it's not possible to -HUP quite a few Debian-based services without knocking over key
services, the d-bus example being one.
|
Libc won't break your system if you don't reboot after an upgrade, nothing I have heard
before in the Debian world anyway. The only disadvantage is that loaded services before the
upgrade will still be using the old library, not necessarily a bad thing unless it is a major
security fix.
You can either reload each service or reboot when most convenient.
|
But libc is used by almost every package on a Debian-based system and running
some services on older libraries and some on new at the same time is a recipe for
instability. Surely ? My experience of Gentoo tells me that at least libc can
universally break an entire system after an upgrade without a reboot :) I learned the
"Gentoo" way :)
|
This say's it all:
library is fundamentally broken in this regard
Mine worked fine on my machine with only gnome-network-manager needing to be start again.
The conclusion to this maybe that no packages, expect from kernel packages, require a system
reboot unless the library itself is broken.
|
I think this is where my info on d-bus partially came from: https://wiki.ubuntu.com/DbusRestartI almost tried restarting dbus on a server then to see, and I'm VERY glad I didn't. I tried
it on a desktop and the whole machine failed and needed powered off and on again !
|
/etc/init.d/dbus restart
You may have to experiment to see if any other services fail after this command is issued and
create a bash script that will restart dbus and start any services that fail. Like this:
/etc/init.d/dbus restart
/etc/init.d/failed-service start
|
That's interesting. Do you have any links saying that libc and dbus don't need a reboot ?
It's not what I've found, searching for info on this. How do you reload d-bus on a Debian
machine ? I'm not sure.
|
After taking another look it seems that dbus and libc shouldn't need a reboot after upgrade.
What happens if you try and reload dbus after an upgrade?
|
Thank you for the script. That will be useful.
On Debian and Ubuntu there is actually a mechanism for saying which packages need a reboot
after being updated but it seems that the maintainers of some packages don't always follow
protocol and add it correctly and therefore it's unreliable.
Basically if this exists then when you login a note is triggered
/var/run/reboot-required.pkgs. Ksplice sidesteps this with this script in
/usr/share/update-notifier/notify-reboot-required.
#!/bin/sh
# Ksplice: don't notify about reboots required by kernel upgrades
if echo "$DPKG_MAINTSCRIPT_PACKAGE" | grep -q "^linux-image-2[.]6[.]"; then
# Add another line to the list of reboots we've saved.
echo "$DPKG_MAINTSCRIPT_PACKAGE" $(date) >> /var/cache/uptrack/reboots_saved
elif echo "$DPKG_MAINTSCRIPT_PACKAGE" | grep -q "^libssl0.9.8"; then
# Work around LP #244250, a bug in libssl configure script that
# sends a reboot notification unnecessarily.
true
else
if [ -x /usr/share/update-notifier/notify-reboot-required.ksplice-orig ]; then
/usr/share/update-notifier/notify-reboot-required.ksplice-orig "$@"
fi
fi
I really just need a list I can trust which means fewer reboots for my production servers.
|
I suppose the critical piece of information we are missing here is a list of packages that
require a reboot. Unfortunately, as far as I know, this information doesn't exist outwith
what we learn as we go being a system admin. I have written up the following python script that, with a predefined list of packages that
require a reboot, will warn you of such before the next upgrade. I would suggest you go on a debian forum and ask what packages you should look out for and
add them to the script as you go. Script: http://pastebin.com/GA4hjDTS
|
Thanks, I'll clarify my question a bit further.
My understanding is: I never want to run an unstable server by not rebooting when it's
necessary (by running old and new libraries simultaneously that make it unstable or because
the system is insecure because new patches haven't been properly applied) and I also only
want to reboot when it's absolutely necessary because they are production servers.
An example might be when lsof (through debian-goodies) can't find an
/etc/init.d start/stop script to restart services that still have old libraries
attached to them. I know that d-bus and libc need a reboot under
those circumstances or the system will be unstable but not all daemons/services need a reboot
and are important to maintaining stability so I'd like a comprehensive way of being able to
tell or a list of absolutely critical updates that need a reboot.
|
If debian-goodies checks for use of old libraries that applications maybe using
and Ksplice deals with the kernel upgrades, What other bases do you want to cover? What do
you mean by "other package upgrades"?
|
