Nexus

From Hackerspace Brussels
Jump to: navigation, search

Contents

[edit] Project Nexus

[edit] Goal

"Configure a dedicated server so that it can host several virtual machines connected on the same virtual network, routed on the IPv6 Internet."

Goals

  • IPs are assigned by the Nexus with DHCPv4 and DHCPv6
  • Adding a new machine should be easy

[edit] Why

The hackerspace needs a new server infrastructure, and we decided to host it on a dedicated server.

[edit] Steps

[edit] Notes

  • In this document, we will use the demonstration IPv6 network 2001:db8:1:2::/64.
  • Our dom0 has the public IPv6 2001:db8:1:2::1, and our domUs will use the network 2001:db8:1:2:100::/72.
  • Similarly, the dom0 has the public IPv4 address 203.0.113.234, and our domUs will use the private network range 10.20.30.0/24.
  • The dom0 will have a LVM virtual group nexus with enough space on it to create new logical volumes on it.

These volumes will be used as the domUs' hard drives.

[edit] Installing Linux

This won't be described here. However, the following steps were executed on Ubuntu quantal (12.10).

Note that as it was automatically installed by OVH on a Kimsufi server, the custom OVH kernel does not support Xen. It had to be changed:

aptitude install linux-server

[edit] Installing Xen

Installing Xen on Debian or Ubuntu is easy with the package manager:

aptitude install xen-hypervisor xen-tools xen-utils

However, the system keeps booting on the first entry, which is not the one we want. However, it can be changed:

mv /etc/grub.d/{06,23}_OVHkernel
mv /etc/grub.d/{10,25}_linux
update-grub

Reboot to use the correct entry:

reboot

Make sure Xen is functional:

# xm list
Name                                        ID   Mem VCPUs      State   Time(s)
Domain-0                                     0 15852     4     r-----      7.6

[edit] Prepare the network

We are going to make a virtual network bridged between all the virtual machines (domUs), but not bridged to the external network, so that the hosting machine (dom0) will route and firewall the traffic.

Nexus Network.svg

[edit] Internal bridge

To create the xenbr0 bridge, we can't use eth0, the unique network interface of the server. We must create a virtual network interface. That's where the kernel module dummy can help us.

modprobe dummy
echo dummy >> /etc/modules

This will have created a virtual network interface named dummy0. We will use it as a basis for a bridge for all our domUs.

cat <<EOF >> /etc/network/interfaces
# Xen Backend
auto xenbr0
iface xenbr0 inet static
    bridge_ports dummy0
    address 10.20.30.1
    netmask 255.255.255.0
    network 10.20.30.0
    broadcast 10.20.30.255
iface xenbr0 inet6 static
    address 2001:db8:1:2:100::1
    netmask 72
EOF
ifup xenbr0

[edit] IP forwarding

Our dom0 will act as a router in our configuration. Therefore, we have to configure it for packet forwarding on both IPv4 and IPv6.

cat <<EOF >> /etc/sysctl.conf
net.ipv4.conf.all.forwarding=1
net.ipv6.conf.all.forwarding=1
net.ipv6.conf.all.proxy_ndp=1
EOF
sysctl -w net.ipv4.conf.all.forwarding=1
sysctl -w net.ipv6.conf.all.forwarding=1
sysctl -w net.ipv6.conf.all.proxy_ndp=1

We also need to tell iptables to perform network address translation (NAT) for our private network:

iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE

More iptables configuration won't be discussed in this document. There is plenty documentation everywhere on the Internet. However, here is a nice configuration to restore the rules at boot time:

cat <<EOF > /etc/network/if-pre-up.d/iptablesload
#!/bin/sh
iptables-restore < /etc/iptables.rules
exit 0
EOF
cat <<EOF > /etc/network/if-post-down.d/iptablessave
#!/bin/sh
iptables-save -c > /etc/iptables.rules
if [ -f /etc/iptables.downrules ]; then
   iptables-restore < /etc/iptables.downrules
fi
exit 0
EOF
chmod +x /etc/network/if-pre-up.d/iptablesload /etc/network/if-post-down.d/iptablessave

[edit] DHCP and router advertisement

As we (obviously) want automatic network configuration on our domUs, we need a DHCP server. Additionnally, as IPv6 DHCP doesn't include router information, we need a router advertisement daemon like radvd.

  • Install DHCP and radvd daemons:
aptitude install isc-dhcp-server radvd
  • Configure DHCP for IPv4:
cat <<EOF > /etc/dhcp/dhcpd.conf 
ddns-update-style none;
option domain-name "hackerspace.be";
option domain-name-servers google-public-dns-a.google.com, google-public-dns-b.google.com;
default-lease-time 600;
max-lease-time 7200;
log-facility local7;
subnet 10.20.30.0 netmask 255.255.255.0 {
  option routers 10.20.30.1;
  range 10.20.30.128 10.20.30.254;
}
EOF

10.20.30.0/25 is reserved for host configuration, based on MAC address.

  • Configure DHCP for IPv6:
cat <<EOF > /etc/dhcp/dhcpd6.conf 
ddns-update-style none;
option domain-name "hackerspace.be";
option domain-name-servers google-public-dns-a.google.com, google-public-dns-b.google.com;
default-lease-time 600;
max-lease-time 7200;
log-facility local7;
subnet6 2001:db8:1:2:100::/72 {
  range6 2001:db8:1:2:180::/73;
}
EOF

2001:db8:1:2:180::/73 is reserved for host configuration, based on MAC address.

  • Tell DHCP to listen only on xenbr0:
patch -p0 <<EOF
--- /etc/default/isc-dhcp-server	2013-01-18 01:47:25.000000000 +0100
+++ /etc/default/isc-dhcp-server	2013-01-18 01:54:19.000000000 +0100
@@ -18,4 +18,4 @@
 
 # On what interfaces should the DHCP server (dhcpd) serve DHCP requests?
 #	Separate multiple interfaces with spaces, e.g. "eth0 eth1".
-INTERFACES=""
+INTERFACES="xenbr0"
EOF
  • Configure radvd:
cat <<EOF > /etc/radvd.conf
interface xenbr0
{
    AdvSendAdvert on;
    prefix 2001:db8:1:2:100::/72
    {
        AdvOnLink on;
        AdvAutonomous on;
    };
};
EOF

This will send router advertisements on interface xenbr0, telling that the network prefix is 2001:db8:1:2:100::/72 and that the router is 2001:db8:1:2:100::1 (as this is the IP of xenbr0).

  • At last, restart the services to use this configuration:
service radvd restart
service isc-dhcp-server restart
service isc-dhcp-server6 restart

[edit] Neighbor solicitations

IPv6 routing is done by answering to neighbor solicitations. Our router will have to answer our ISP's requests for known IPs.

The problem is that, in Linux, the list of IPs for which the router will answer is statically configured.

The project npd6 (Neighbor Proxy Daemon IPv6) will answer to *all* the requests for a given IPv6 range.

  • Add the npd6 launchpad:
add-apt-repository ppa:sgroarke/npd6
  • Install it
aptitude update
aptitude install npd6
  • Configure it:
cat <<EOF > /etc/npd6.conf
prefix=2001:db8:1:2:
interface=eth0
listtype = none
listlogging = off
collectTargets = 100
linkOption = false
ignoreLocal = true
routerNA = true
maxHops = 255
pollErrorLimit = 20
EOF

Here, we will answer 'yes' for *all* IPv6 addresses in the range 2001:db8:1:2::/64

  • Start it:
service npd6 start
  • Add it to the system startup
update-rc.d npd6 defaults

[edit] Create a domU

[edit] Create a LVM virtual disk

If there is no LVM physical volume yet, you can create it.

  • Create the physical volume:
pvcreate /dev/sda4
  • Create the virtual group named nexus:
vgcreate nexus /dev/sda4

Now you can create a new logical volume:

lvcreate -L 4G -n probe0 nexus

[edit] Get the boot files

Ubuntu provides light netboot images for installation. We are going to use them to install our dom0.

Change the mirror and the version if needed ;)

mkdir -p /var/lib/xen/images/ubuntu-netboot
cd /var/lib/xen/images/ubuntu-netboot
wget http://ftp.belnet.be/ubuntu.com/ubuntu/dists/quantal/main/installer-amd64/current/images/netboot/xen/vmlinuz
wget http://ftp.belnet.be/ubuntu.com/ubuntu/dists/quantal/main/installer-amd64/current/images/netboot/xen/initrd.gz

[edit] Create the domU

Now we can create a configuration file for the domU probe0:

cat <<EOF >/etc/xen/probe0.cfg
name = "probe0"
memory = 256
disk = ['phy:/dev/nexus/probe0,xvda,w']
vif = ['mac=0a:00:00:00:00:01']
kernel = "/var/lib/xen/images/ubuntu-netboot/vmlinuz"
ramdisk = "/var/lib/xen/images/ubuntu-netboot/initrd.gz"
extra = "debian-installer/exit/always_halt=true -- console=hvc0"
EOF

Boot it to install the system:

xm create /etc/xen/probe0.cfg -c

Once the installation complete, change the configuration file:

patch -p 0 <<EOF
--- /etc/xen/probe0.cfg
+++ /etc/xen/probe0.cfg
@@ -3,5 +3,6 @@
 disk = ['phy:/dev/nexus/probe0,xvda,w']
 vif = ['mac=0a:00:00:00:00:01']
-kernel = "/var/lib/xen/images/ubuntu-netboot/vmlinuz"
-ramdisk = "/var/lib/xen/images/ubuntu-netboot/initrd.gz"
-extra = "debian-installer/exit/always_halt=true -- console=hvc0"
+#kernel = "/var/lib/xen/images/ubuntu-netboot/vmlinuz"
+#ramdisk = "/var/lib/xen/images/ubuntu-netboot/initrd.gz"
+#extra = "debian-installer/exit/always_halt=true -- console=hvc0"
+bootloader = "pygrub"
EOF

And boot this domU:

xm create /etc/xen/vmtest.cfg -c

The last thing to do is to tell the domU to ask for an IPv6 address:

echo iface eth0 inet6 dhcp >> /etc/network/interfaces