You might want your LXC containers to be accessible on your LAN, just like any other systems. In this article we're going through the steps to setup LXC networking using the Host Shared Bridge and DHCP controlled by the LAN router. A very good guide can be found at wiki.debian.org.
Host setup (br0)
The main host has a bridge interface called br0 with the default ethernet interface (eth0) attached to it.
The physical ethernet device is set to manual and the bridge is set up to obtain the lease via DHCP.
File /etc/network/interfaces
:
allow-hotplug eth0
iface eth0 inet manual
auto br0
iface br0 inet dhcp
bridge_ports eth0
bridge_fd 0
bridge_maxwait 0
LXC setup: lxc-net, lxc-usernet
If not using a dedicated LXC bridge, the specific setting may also be changed in /etc/default/lxc-net
:
USE_LXC_BRIDGE="false"
In case of unprivileged containers, the user needs to be allowed to created and attach veth interfaces to br0.
File /etc/lxc/lxc-usernet
:
lxcuser veth br0 10
If all of the containers will follow the same scheme, dnsmasq on the main host may be disabled.
systemctl disable dnsmasq
Containers
The containers' network interfaces will be attached to the br0 bridge.
The fragment of my lxc0a
container configuration file (.local/share/lxc/lxc0a/config
):
# Network configuration
lxc.net.0.type = veth
lxc.net.0.link = br0
lxc.net.0.flags = up
lxc.net.0.hwaddr = 00:00:00:00:00:0A
I called the container lxc0a
and I manually set the hardware address to 00:00:00:00:00:0a
. I use these in the DHCP server configuration in my LAN router.
Once you start the container, its interface should be attached to a bridge:
# brctl show br0
bridge name bridge id STP enabled interfaces
br0 8000.f0def1a1679f no eth0
veth1004_6GST
LAN Router
If needed, configure your DHCP server. I'm using OpenWRT and opt to configure static leases for most of my hosts.
DHCP, Firewall
If the main host's default firewall policy is DROP, then DHCP client within the containers might not be working right away. If the container does not get assigned an address, it's time to debug things. I'd suggest using tcpdump (on both - main host and the router (if possible)).
Debugging with tcpdump
tcpdump -v -i br0
This will allow you to trace the packets (DHCP request, DHCP offer and ACK). It looks simiar to this:
2:56:18.194075 IP (tos 0x10, ttl 128, id 0, offset 0, flags [none], proto UDP (17), length 328)
0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 00:00:00:00:00:0a (oui Ethernet), length 300, xid 0x6d189230, Flags [none]
Client-Ethernet-Address 00:00:00:00:00:0a (oui Ethernet)
Vendor-rfc1048 Extensions
Magic Cookie 0x63825363
DHCP-Message (53), length 1: Discover
Requested-IP (50), length 4: lxc0a.lan
Hostname (12), length 5: "lxc0a"
Parameter-Request (55), length 13:
Subnet-Mask (1), BR (28), Time-Zone (2), Default-Gateway (3)
Domain-Name (15), Domain-Name-Server (6), Unknown (119), Hostname (12)
Netbios-Name-Server (44), Netbios-Scope (47), MTU (26), Classless-Static-Route (121)
NTP (42)
Starting DHCP client
For debugging it is best to stop the dhcp client in the container and start it manually.
Stop:
killall dhclient
Run manually:
/sbin/dhclient -v
This will let you see what is happening and where things go wrong.
Firewall rules (iptables)
You need to forward packets on br0:
-A FORWARD -i br0 -j ACCEPT
The main host might need a rule allowing outgoing DHCP traffic:
-A OUTPUT -p udp --dport 67 -j ACCEPT
Depending on the scenario - you might also need to explicitely allow DHCP ports.
-I INPUT -i br0 -p udp --dport 67:68 --sport 67:68 -j ACCEPT
If using netfilter-persistent - it's best to flush all rules before restarting netfilter.
Flushing the rules can be achived by a simple script:
#!/bin/sh
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -t raw -F
iptables -t raw -X
I use it this way:
# ./iptables-flush-all; /etc/init.d/netfilter-persistent restart
Restarting netfilter-persistent (via systemctl): netfilter-persistent.service.
It is also useful for testing the new firewall rules and avoiding locking yourself out of the remote machine. Just switch the command order, and use a simple sleep
in between, like so:
# /etc/init.d/netfilter-persistent restart ; sleep 60; ./iptables-flush-all
DNS
If your router handles DHCP and DNS - the rest of you machines will be able to resolve the container address by name.
(laptop) $ host lxc0a
lxc0a has address 10.0.3.100
(laptop) $ host lxc0b
lxc0b has address 10.0.3.101