[Users] Hosts not responding, kinda urgent

[Gregor at HostGIS]

Hey guys. I have 4 VEs, out of 13 on this server, which suddenly stopped 
being networked. No pings, no HTTP, no nothing -- except from the HN 
itself. The VEs cannot ping out, no DNS, nothing.

I know that no changes were made to the HN, as I'm the only one who 
works on it. And it's kinda urgent as these are live. :)

I have tried comparing the 4 that don't work against the 9 which do 
work, and am coming up totally blank here.

* The VEs are running. All VEs use veths.

* Double-checked the "route -n" and "ifconfig" output inside the VE.

* iptables -- Turned off, I'm positive. "iptables -L" shows nothing but 
ACCEPTs.

* The "route -n" output on the HN is perfectly ordinary; proper veths 
and host-IP entries, no null-routes or other such shenanigans.

* I compared "sysctl -a | grep vethXX | grep ipv4" outputs, and 
identical except for the vethXX itself.

* Restarted one of the affected VEs. No effect at all.

So I'm at a loss here. If the routing and veths are OK, and the sysctl 
output is identical, why would 4 of them suddenly have ceased 
functioning, and how can I fix them?

Following is an example of "sysctl -a | grep ipv4.conf | grep veth16"

net.ipv4.conf.veth16/0.forwarding = 1
net.ipv4.conf.veth16/0.mc_forwarding = 0
net.ipv4.conf.veth16/0.accept_redirects = 1
net.ipv4.conf.veth16/0.secure_redirects = 1
net.ipv4.conf.veth16/0.shared_media = 1
net.ipv4.conf.veth16/0.rp_filter = 1
net.ipv4.conf.veth16/0.send_redirects = 1
net.ipv4.conf.veth16/0.accept_source_route = 0
net.ipv4.conf.veth16/0.proxy_arp = 1
net.ipv4.conf.veth16/0.medium_id = 0
net.ipv4.conf.veth16/0.bootp_relay = 0
net.ipv4.conf.veth16/0.log_martians = 0
net.ipv4.conf.veth16/0.tag = 0
net.ipv4.conf.veth16/0.arp_filter = 0
net.ipv4.conf.veth16/0.arp_announce = 0
net.ipv4.conf.veth16/0.arp_ignore = 0
net.ipv4.conf.veth16/0.arp_accept = 0
net.ipv4.conf.veth16/0.disable_xfrm = 0
net.ipv4.conf.veth16/0.disable_policy = 0
net.ipv4.conf.veth16/0.force_igmp_version = 0
net.ipv4.conf.veth16/0.promote_secondaries = 0

The "route -n" line for its veth:

Destination     Gateway         Genmask         Flags Metric Ref    Use 
Iface
216.93.173.151  0.0.0.0         255.255.255.255 UH    0      0        0 
veth16.0

Any thoughts on further diagnostics?

If venets are better, and may solve this, perhaps specific instructions 
on converting one to a venet and trying again?

-- 
HostGIS, Open Source solutions for the global GIS community
Greg Allensworth - SysAdmin, Programmer, GIS Person, Security
    Network+   Server+   A+   Security+   Linux+
    PHP   PostgreSQL   MySQL   DHTML/JavaScript/AJAX

"No one cares if you can back up — only if you can recover."