at work we rent a dedicated server from OVH; except unexplained openvpn throttling all is working pretty well for the price we pay. besides primary IPv4 address OVH can provide few additional ‘failover’ IPv4 addresses and /64 IPv6 subnet. in our setup some of IPv4s and IPv6s are routed to a KVM VM. below – description of the configuration details.
IPv4 setup
it seems that the additional [called failover] IPv4 addresses are re-routed by the datacenter via our primary IPv4 address. because i have some private traffic flowing between KVM guest, the host server and few LXC containers i did not want to bridge any of the guests directly to the data-center facing eth0 interface. instead i’ve created two ‘host only’ networks:
- br0 – connecting LXC machines with the dummy0 virtual interface on the physical server
- br1 – connecting KVM VM with the dummy1 virtual interface on the physical server
the logical setup looks as follows:
addresses:
- 192.95.0.10/24 is the primary IPv4 assigned by OVH, it’s using 192.95.0.254 as a default gateway
- 192.50.128.15/32 and 142.4.206.19/32 are additional IPv4 addresses provided by OVH. their routing expects to reach those addresses via 192.95.0.10 which means i could bind them e.g. to the loopback interface of the physical server, or – as in my case – i can route them via internal network to KVM VM
- on br0:
- 192.168.0.1/24 is bound to the dummy0 – virtual interface on the physical machine
- 192.168.0.2/24, 192.168.0.3/24 – are bound to LXC guest machines. those machines can communicate between each other and reach the internet resources via routing and SNAT handled by the physical server
- on br1:
- 10.0.0.1/24 is bound to the dummy1 – another virtual interface on the physical machine
- 10.0.0.2/24 is bound to the eth0 of KVM guest. to the same interface i’ve bound 192.50.128.15 and 142.4.206.19; to make that guest reachable from the internet i’ve also set up static routing and forwarding on the physical machine telling that both addresses are reachable via 10.0.0.2 on br1
configuration of the physical machine – in /etc/network/interfaces:
auto eth0
iface eth0 inet static
address 192.95.0.10/24
gateway 192.95.0.254
# dummy interface used to build bridge between LXC guests and the physical machine
auto dummy0
iface dummy0 inet manual
auto br0
iface br0 inet static
address 192.168.0.1/24
bridge_stp off
bridge_fd 0
bridge_ports dummy0
# dummy interface used to build bridge between KVM guest and the physical machine
auto dummy1
iface dummy1 inet manual
auto br1
iface br1 inet static
address 10.0.0.1/30
bridge_stp off
bridge_fd 0
bridge_ports dummy0
# i re-route further the 'failover' IPv4 addresses to the KVM guest
post-up /sbin/ip r a 142.4.206.19/32 via 10.0.0.2 dev br1
post-up /sbin/ip r a 198.50.128.15/32 via 10.0.0.2 dev br1
additionally in the startup script /etc/rc.local routing is enabled:
echo 1 > /proc/sys/net/ipv4/ip_forward
since i want all outgoing internet traffic from KVM to have source ip of 142.4.206.19 – i’ve added a nat rule on the KVM guest in /etc/rc.local:
iptables -t nat -A POSTROUTING -o eth0 -d 10.0.0.0/8 -j RETURN iptables -t nat -A POSTROUTING -o eth0 -d 192.168.0.0/16 -j RETURN iptables -t nat -A POSTROUTING -o eth0 -j SNAT --to 142.4.206.172
at this stage i can already reach the physical server [ 192.95.0.10 ], and KVM VM [ 142.4.206.19, 198.50.128.15 ] from the internet.
IPv6 setup
configuration of the IPv6 is less obvious – OVH does not re-route some subnet via primary address assigned to us. they just provide /64 network reachable directly via the datacenter-facing eth0. my options of providing v6 connectivity to the KVM VM:
- bridge the VM directly to the datacenter-facing network – initially i wanted to avoid doing it. i wanted to keep the private part of my internal traffic – well – private.
- ask OVH for additional v6 subnet re-routed via our server – unsurprisingly that did not work – they only provide /64
- use arp-proxy mechanism for IPv6
the last option worked pretty well. i did not have to set up any additional interfaces; i’ve just added more addressing and configuration rules. new setup:
new addresses:
- 2607:5300:12:2f0a::0/64 is the IPv6 network designated for my use by OVH
- 2607:5300:12:2fff:ff:ff:ff:ff/64 is the default gateway for my subnet provided by OVH
- 2607:5300:12:2f0a::1/64 is an arbitrarily chosen IPv6 address out of ‘my’ v6 subnet. i intentionally bound 2607:5300:60:2F0a::1/126 [rather than /64] to eth0 of the physical server
- 2607:5300:12:2f0a::11/124 is the v6 address i’ve assigned to the dummy1 interface – part of the br1 bridge between KVM and the physical server
- 2607:5300:12:2f0a::12/124 is the v6 address assigned to the KVM guest
it’s all fine and dandy but we have to be able to tell the datacenter’s router that 2607:5300:12:2f0a::12 can be reached; just setting above addresses up is not enough. the mentioned arp-proxy mechanism solves the problem. additional lines in the /etc/network/interfaces of the physical machine:
iface eth0 inet6 static
address 2607:5300:12:2f0a::1/126
# for some reason 'normal' way of specifying the default route did not work. maybe because of the intentional mask mismatch?
post-up /sbin/ip -f inet6 route add 2607:5300:12:2Fff:ff:ff:ff:ff dev eth0
post-up /sbin/ip -f inet6 route add default via 2607:5300:12:2fff:ff:ff:ff:ff
iface br1 inet6 static
address 2607:5300:12:2f0a::11/124
# use arp proxy - a trick that lets me make some of the IPv6 addresses available to KVM guest despite the fact
# that it's not directly bridged to the OVH-facing network
post-up /sbin/ip -6 neigh add proxy 2607:5300:12:2f0a::12 dev eth0
post-up /sbin/ip -6 neigh add proxy 2607:5300:12:2f0a::13 dev eth0
additional commands in the startup script:
echo 1 > /proc/sys/net/ipv6/conf/all/proxy_ndp echo 1 > /proc/sys/net/ipv6/conf/all/forwarding echo 1 > /proc/sys/net/ipv6/conf/default/forwarding echo 0 > /proc/sys/net/ipv6/conf/all/autoconf echo 0 > /proc/sys/net/ipv6/conf/all/accept_ra echo 0 > /proc/sys/net/ipv6/conf/all/accept_redirects echo 0 > /proc/sys/net/ipv6/conf/all/router_solicitations
kvm guest just requires this in /etc/network/interfaces :
iface eth0 inet6 static
address 2607:5300:12:2f0a::12/124
gateway 2607:5300:12:2f0a::11
what i described is simplified. on the top of that i have in production:
- v4 and v6 packet filtering
- SNAT / routing for traffic from LXC guests done on the physical server
- openvpn tunnel to reach our other sites
- gre tunnel to make openvpn work at the speed of tens of megabits rather than kilobits
edit: 2014-08-16 – i’ve enabled ipv6 on our internet facing DNS servers [and advertised v6 addresses in the glue records], web server, mail server, client-facing proxies and… nothing happened. we’ve started to see some traffic and so far – no complains. nice!