[Users] Dual networks. Routing issue?

[Einar S. Idsø]

Hi,

We currently run a setup with a number of nodes. All of them are
connected to two different networks: Frontend ("the internet") and
backend (10.0.0.x). These two networks are connected to via separate
switches from separate ethernet ports. In this setup, all nodes can
communicate with eachother via both the frontend and backend. All
virtual hosts have IPs in the frontend-range, and can communicate with
eachother. However, I would like some of the virtual hosts to also be
able to communicate via the backend network, but so far I have been
unable to make this work.

I have tried to run vzctl with --ipadd 10.0.0.x. Upon entering the
virtual host, I see that ifconfig correctly reports venet0:1 with the
given IP (venet0:0 has the frontend IP). Trying to ping back and forth
between hosts and nodes on the backend network, this is what I've
found:
1. Virtual hosts can ping their local node. This is regardless of
whether the virtual host has had an IP in the backend network added.
2. Virtual hosts can ping other virtual hosts on the local node.
Similarly to (1), the pinging host does not need to have an IP on the
backend network.
3. Virtual hosts can not ping non-local nodes or non-local virtual hosts.
4. Nodes can ping non-local virtual hosts.

(Please recall that the frontend network works flawlessly.These issues
only concern the backend.)

My guess is that this is a routing issue, but to be honest I don't
know enough about routing to work out what's needed. Here's the output
from route on a node with two virtual hsots (I've masked the frontend
IP range as 9.9.9.x):

# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
9.9.9.173  0.0.0.0         255.255.255.255 UH    0      0        0 venet0
10.0.0.179      0.0.0.0         255.255.255.255 UH    0      0        0 venet0
10.0.0.173      0.0.0.0         255.255.255.255 UH    0      0        0 venet0
9.9.9.179  0.0.0.0         255.255.255.255 UH    0      0        0 venet0
9.9.9.128  0.0.0.0         255.255.255.192 U     0      0        0 eth0
10.0.0.0        0.0.0.0         255.255.255.0   U     0      0        0 eth1
169.254.0.0     0.0.0.0         255.255.0.0     U     1002   0        0 eth0
169.254.0.0     0.0.0.0         255.255.0.0     U     1003   0        0 eth1
0.0.0.0         9.9.9.129  0.0.0.0         UG    0      0        0 eth0

The node IP is 9.9.9.138/10.0.0.138.  The two virtual hosts are
9.9.9.179/10.0.0.179 and 9.9.9.173/10.0.0.173.

Here's the same output from the virtual host on x.x.x.179:
# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
169.254.0.0     0.0.0.0         255.255.0.0     U     1002   0        0 venet0
0.0.0.0         0.0.0.0         0.0.0.0         U     0      0        0 venet0

Can anyone advice?

Cheers,
Einar