Welcome to Server Circle. It's a friendly site and all levels of experience are welcome. Be aware that we use cookies for your login.
Server Circle - Ask questions about Servers and get answers from experts.
Beta (0.81 sec)
Routing with dual networks and KVM-hosts

I have a firewall (shorewall) with two networks, one with public ip-addresses and one behind a nat with private addresses. A KVM-server (Ubuntu 10.04) attached to both networks via two bridged network cards. A VPS (a KVM-virtual machine) on the KVM-host has two virtual network interfaces attached to each of the networks. I want to be able to reach the server each path from the local private network and via the public ip from Internet.

link to a picture

http://ubuntuone.com/0FGG30aIUK14CmCJLGcmXa

With this configuration I can reach the VPS using the private network and private address and using the public address from Internet, but not using the public address from the private network.

Gateway (gate)

aw@gate:~$ route

Kernel IP routing table

Destination Gateway Genmask Flags Metric Ref Use Iface

99.99.99.0 * 255.255.255.240 U 0 0 0 eth0

99.99.98.128 * 255.255.255.128 U 0 0 0 eth2

192.168.1.0 * 255.255.255.0 U 0 0 0 eth1

default 99.99.99.129 0.0.0.0 UG 100 0 0 eth2

KVM-host (magne)

aw@magne:~$ ifconfig

br0 Link encap:Ethernet HWaddr 12:25:90:39:80:44

inet addr:192.168.1.78 Bcast:192.168.1.255 Mask:255.255.255.0

inet6 addr: 2001:470:28:1ac:225:90ff:fe39:8044/64 Scope:Global

inet6 addr: fe80::225:90ff:fe39:8044/64 Scope:Link

UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1

RX packets:81437 errors:0 dropped:0 overruns:0 frame:0

TX packets:75378 errors:0 dropped:0 overruns:0 carrier:0

collisions:0 txqueuelen:0

RX bytes:10458872 (10.4 MB) TX bytes:29451307 (29.4 MB)

br1 Link encap:Ethernet HWaddr 12:25:90:39:80:45

inet6 addr: fe80::225:90ff:fe39:8045/64 Scope:Link

UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1

RX packets:477 errors:0 dropped:0 overruns:0 frame:0

TX packets:6 errors:0 dropped:0 overruns:0 carrier:0

collisions:0 txqueuelen:0

RX bytes:45443 (45.4 KB) TX bytes:468 (468.0 B)

aw@magne:~$ route

Kernel IP routing table

Destination Gateway Genmask Flags Metric Ref Use Iface

192.168.1.0 * 255.255.255.0 U 0 0 0 br0

192.168.122.0 * 255.255.255.0 U 0 0 0 virbr0

default 192.168.1.1 0.0.0.0 UG 100 0 0 br0

have also tried with this

aw@magne:~$ route

Kernel IP routing table

Destination Gateway Genmask Flags Metric Ref Use Iface

99.99.99.0 99.99.99.1 255.255.255.240 UG 100 0 0 br1

192.168.1.0 * 255.255.255.0 U 0 0 0 br0

192.168.122.0 * 255.255.255.0 U 0 0 0 virbr0

default 192.168.1.1 0.0.0.0 UG 100 0 0 br0

I don't think the routing table on the bridged KVM-host does matter?

VPS (virtual machine, ask)

aw@ask:~$ ifconfig

eth0 Link encap:Ethernet HWaddr 12:54:00:e1:b3:d7

inet addr:192.168.1.185 Bcast:192.168.1.255 Mask:255.255.255.0

inet6 addr: 2001:470:28:1ac:5054:ff:fee1:b3d7/64 Scope:Global

inet6 addr: fe80::5054:ff:fee1:b3d7/64 Scope:Link

UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1

RX packets:5549 errors:0 dropped:0 overruns:0 frame:0

TX packets:1941 errors:0 dropped:0 overruns:0 carrier:0

collisions:0 txqueuelen:1000

RX bytes:675590 (675.5 KB) TX bytes:214175 (214.1 KB)

eth1 Link encap:Ethernet HWaddr 12:54:00:2a:3a:dc

inet addr:99.99.99.14 Bcast:99.99.99.15 Mask:255.255.255.240

inet6 addr: fe80::5054:ff:fe2a:3adc/64 Scope:Link

UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1

RX packets:2122 errors:0 dropped:0 overruns:0 frame:0

TX packets:1720 errors:0 dropped:0 overruns:0 carrier:0

collisions:0 txqueuelen:1000

RX bytes:232251 (232.2 KB) TX bytes:198017 (198.0 KB)

Eth0 are bridged with br0 in VirtManager and eth1 with br1

aw@ask:~$ route

Kernel IP routing table

Destination Gateway Genmask Flags Metric Ref Use Iface

99.99.99.0 * 255.255.255.240 U 0 0 0 eth1

192.168.1.0 * 255.255.255.0 U 0 0 0 eth0

default 99.99.99.1 0.0.0.0 UG 10 0 0 eth1

default 192.168.1.1 0.0.0.0 UG 100 0 0 eth0

aw@ask:~$

Machine on the Internet (embla)

aw@embla:~$ ifconfig

eth2 Link encap:Ethernet HWaddr 12:30:48:88:31:b8

inet addr:88.88.88.78 Bcast:88.88.88.79 Mask:255.255.255.248

inet6 addr: fe80::230:48ff:fe88:31b8/64 Scope:Link

UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1

RX packets:5593116 errors:0 dropped:0 overruns:0 frame:0

TX packets:3947742 errors:0 dropped:0 overruns:0 carrier:0

collisions:0 txqueuelen:1000

RX bytes:4262717553 (4.2 GB) TX bytes:491960763 (491.9 MB)

Interrupt:18

aw@embla:~$ route

Kernel IP routing table

Destination Gateway Genmask Flags Metric Ref Use Iface

88.88.88.72 * 255.255.255.248 U 0 0 0 eth2

default 88.88.88.73 0.0.0.0 UG 100 0 0 eth2

aw@embla:~$

aw@embla:~$ ssh 99.99.99.14

aw@99.99.99.14's password:

(working)

Klient on the local network (lian)

aw@lian:~$ route

Kernel IP routing table

Destination Gateway Genmask Flags Metric Ref Use Iface

192.168.1.0 * 255.255.255.0 U 1 0 0 eth0

default 192.168.1.1 0.0.0.0 UG 0 0 0 eth0

aw@lian:~$ ssh 192.168.1.185

aw@192.168.1.185's password:

aw@lian:~$ ssh 99.99.99.14 (hangs)

I think lian and embla are correct and the error could be found in either gate/magne/ask. If ask has only one virtual ethernet with for example br1 it works both from Internet (embla) and from the local network (lian). I have fiddled with metrics for the dual default route in ask without any luck.



Any ideas?

Asked by:
aw
50 points
 Report Abuse
 Share Page - Category: Linux Servers - Tags: Routing with dual networks and KVM-hosts
 Enter your response
Please use Pastie.org to paste lengthy code or to fix formatting issues with code
  • Responses in reverse (1)

My first observation is that you have two gateways configured on ask - you should only have one (on its public-side interface) - that being the route you want all external traffic to take out of the system.

That is unlikely to solve the problem, I suspect, as I would think the issue you describe is being caused by the NAT rules on gate.

I would think gate is performing NAT translation on traffic from your internal (192.168-net) network out to the internet but not to the 99-net interface (this may be easily configurable in your firewall interface, but I haven't used shorewall in approaching 10 years).

You will need to either remove the internal (192.168-net) interface from ask or configure gate to NAT the requests coming from 192.168-net to 99-net in order for the communication to work between lian and ask's public 99-net address.

Hope this helps,

Jim

Response by:
jimrippon
873 points


  • Related Questions
About Us : Contact Us : Etiquette : Terms : CDN Failover : ShorterURL : CDN Fallback : © 2013 Server Circle