Daily Archives: November 14, 2008

Using KVM to securely host servers in a DMZ

Posted on November 14, 2008 by Chris

We host a number of web services and applications on the servers in here in the PaperCut office. We’ve always planned on hosting these on an isolated server inside a demilitarized zone (DMZ) to ensure public applications are isolated from internal servers. This usually requires separate dedicated servers, however with the recent growth in virtualization technology, we decided to see if we could accomplish the same in a virtual environment. There was not a lot of information out there so I embarked on a project to develop our own. The solution has worked very well over the past 6 months so I’ve decided to open source the configuration and control script so others in the Linux community can benefit. (One of my Friday projects when I’m not working on print accounting software!).

The crux of the script is to host a Qemu or KVM virtual machine on an independent subnet via a tun/tap interface. iptables on the host (Dom0) is used to ensure that connections can not be instigated from the VM in the DMZ to any system in the internal network. They say a picture is worth a thousand words, so here’s a diagram:

Network Setup - KVM running in a DMZ

The key items are:

  • The host (dom0) hosted the VM on a tun/tap interface.
  • The VM is on a separate subnet.
  • A firewall on dom0 (important) prevents access to the internal network.
  • A static route has been added to the router so internal network can “find” the systems in the DMZ.
  • Public ports (e.g. port 80) on the router are forwarded into the server in the DMZ.

This strategy will provide an extra layer of protection as a compromise on the server in the DMZ (say hosting your website) will not automatically mean a compromise on your internal network. There are however come caveats to this: It may be possible to “jailbreak” from the VM into the host by exploiting vulnerabilities in the hypervisor/host. For example, some exploits were found in QEMU in 2007.

The control script and its brief setup procedure should work on most modern Linux distributions.

file: dmz-vm-controller

#!/bin/sh
### BEGIN INIT INFO
# Provides:          vm-dmz-controller
# Required-Start:    $local_fs $network
# Required-Stop:     $local_fs $network
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: VM Management in a DMZ
# Description:       QEMU/KVM VM Management in a semi-secured DMZ.
### END INIT INFO

##############################################################################
#
# VM-DMZ-Controller is a wrapper script written to help with the management
# and setup of a VM running inside a secured demilitarized zone (DMZ).  The
# objective is to ensure the host/vm inside the DMZ are firewalled in a way
# that ensures connections from the DMZ to the internal network are not
# possible.
#
# Brief summary:
#
#   1. Install QEMU or KVM, and socat, iptables and tun/tap tunctl
#      (uml-utilities).
#
#   2. Create non-privileged user on your system called "vm".
#
#   3. Create a sub-directory in the VM user's home directory to host your VM
#      files.
#
#   4. Create your disk images (e.g. qemu-img) in this sub-directory.
#
#   5. Copy this script into the directory and modify configuration section
#      below.
#
#   6. Link in this script into /etc/init.d/ and configure runlevels as
#      appropriate.
#
#   7. Add a static route in your internal network default router so internal
#      systems can connect to the VM.
#
#   8. Start your VM and test.  Confirm that the VM is unable to access your
#      internal network.
#
# See here for details:
# http://www.papercut.com/blog/chris/2008/11/14/using-kvm-to-securely-host-servers-in-a-dmz/
#
#
# Copyright (c) 2008, PaperCut Software International Pty. Ltd.
# http://www.papercut.com/
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
#     * Redistributions of source code must retain the above copyright
#       notice, this list of conditions and the following disclaimer.
#     * Redistributions in binary form must reproduce the above copyright
#       notice, this list of conditions and the following disclaimer in the
#       documentation and/or other materials provided with the distribution.
#     * Neither the name of the  nor the
#       names of its contributors may be used to endorse or promote products
#       derived from this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY PAPERCUT SOFTWARE ''AS IS'' AND ANY
# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
# DISCLAIMED. IN NO EVENT SHALL  BE LIABLE FOR ANY
# DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
# (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#
###############################################################################

###############################################################################
# VM Configuration - modify below as appropriate
###############################################################################

# The name of the VM instance (should be unique if hosting multiple VMs)
#
VM_NAME=external-web-server

# The non-privileged user ID used to run the VM.
#
VM_USER=vm

# The VM kernel module to load (e.g. kvm-intel, kvm-amd, qemu). Leave blank if
# using QEmu as a kernel model is required.
#
VM_MODULE=kvm-intel

# The name of the virtual network tap to bind/host to the DMZ network on.
#
IFNAME=tap0

# The .1 gateway address that denotes the DMZ subnet.
#
DMZ_IP=192.168.100.1

# The subnet range of the internal network (the range to firewall/protect)
#
INTERNAL_SUBNET=192.168.1.0/24

# Your DMZ system may need DNS access provided by your internal network.
# Set this if required. This will leave a hole in the firewall allowing
# DNZ access (UDP source port 53).
#
INTERNAL_DNS_IP=

# The directory with disk images (and pid files, etc.) are hosted
#
VM_DIR=/home/${VM_USER}/${VM_NAME}

MONITORFILE=${VM_DIR}/.${VM_NAME}.monitor
PIDFILE=${VM_DIR}/.${VM_NAME}.pid
LOGFILE=${VM_DIR}/${VM_NAME}.log

# VM Start command-line. No need to define:
#    -pidfile, -net, or -monitor
# as these are all appended as part of this script.
# Add -cdrom and -boot d to boot and install your VM off a CD.
#
VM_START_CMD="kvm \
        -hda disk1.qcow2 \
        -m 384 \
        -vnc :0"

# The maximum time to provide the VM to conduct a graceful shutdown.
#
SHUTDOWN_TIMEOUT=20

###############################################################################
# End Configuration - DO NOT MODIFY BELOW THIS LINE
###############################################################################

start_vm() {

    echo_n "Starting VM ${VM_NAME}..."
    if isrunning; then
        echo "ALREADY RUNNING"
        exit 0
    fi

    setup_networking
    start_firewall

    if [ ! -z "${VM_MODULE}" ]; then
        modprobe "${VM_MODULE}"
    fi
    cd "${VM_DIR}"
    su "${VM_USER}" -c "${VM_START_CMD} \
                -net nic -net tap,ifname=${IFNAME},script=no \

                -pidfile ${PIDFILE} \
                -monitor unix:${MONITORFILE},server,nowait \
                >> ${LOGFILE} 2>&1 &"

    for i in 0 1 2 3; do
        sleep 2
        if isrunning; then
            echo "Started ${VM_NAME} at: `date`" >> ${LOGFILE}
            echo "started."
            exit 0
        else
            echo_n "."
        fi
    done

    echo "ERROR"
    exit 1
}

stop_vm() {

    echo_n "Stopping VM ${VM_NAME}..."
    if isrunning; then
        # Send nice powerdown command
        echo "system_powerdown" | socat - UNIX-CONNECT:${MONITORFILE} \
                >/dev/null
        clean_shutdown=
        for (( i = 0 ; i <= ${SHUTDOWN_TIMEOUT} ; i++ )); do
            sleep 1
            if isrunning; then
                echo_n "."
            else
                clean_shutdown=y
                break;
            fi
        done

        if [ -z "${clean_shutdown}" ]; then
            echo_n "forcing..."
            kill -TERM "${pid}"
            sleep 2
        fi

        if isrunning; then
            echo "problem stopping!"
            exit 1
        fi

        rm ${MONITORFILE}
        rm ${PIDFILE}
        stop_firewall
        stop_networking
    fi
    echo "Stopped ${VM_NAME} at: `date`" >> ${LOGFILE}
    echo "stopped."
}

status() {

    if isrunning; then
        echo "Running (pid: ${pid})."
    else
        echo "Not Running."
    fi
}

forcekill() {

    if isrunning; then
        kill -9 "${pid}"
    else
        echo "Not running!"
    fi
}

isrunning() {

    if [ -r ${PIDFILE} ]; then
        pid=`cat ${PIDFILE} 2>/dev/null`
        if [ ! -z "${pid}" -a -d /proc/${pid} ]; then
            return 0 #Success - running
        else
            return 1 #Failure - not running
        fi
    else
        return 1 #Failure - not running
    fi
}

setup_networking() {

    tunctl -u ${VM_USER} -t ${IFNAME} >/dev/null

    ifconfig ${IFNAME} ${DMZ_IP} netmask 255.255.255.0 up >/dev/null

}

start_firewall() {

    modprobe ip_tables
    modprobe iptable_nat

    echo "1" > /proc/sys/net/ipv4/ip_forward

    #
    # Deny new connections to internal network (forwarded) and Dom0 (input)
    #
    iptables -A FORWARD -d $INTERNAL_SUBNET -i $IFNAME -p tcp --syn \
            -m limit --limit 6/h --limit-burst 5 -j LOG

    iptables -A FORWARD -d $INTERNAL_SUBNET -i $IFNAME -p tcp --syn \
            -j DROP

    iptables -A INPUT -d $INTERNAL_SUBNET -i $IFNAME -p tcp --syn \
            -m limit --limit 6/h --limit-burst 5 -j LOG

    iptables -A INPUT -d $INTERNAL_SUBNET -i $IFNAME -p tcp --syn \
            -j DROP

    # Also need to protect the DMZ side of host box.
    iptables -A INPUT -d $DMZ_IP -i $IFNAME -p tcp --syn \
            -m limit --limit 6/h --limit-burst 5 -j LOG

    iptables -A INPUT -d $DMZ_IP -i $IFNAME -p tcp --syn \
            -j DROP

    #
    # Allow DNS UDP packets to DNS server (required if on internal network)
    #
    if [ ! -z "${INTERNAL_DNS_IP}" ]; then
        iptables -A FORWARD -p udp -d $INTERNAL_DNS_IP \
            --dport 53 -i $IFNAME -j ACCEPT
    fi

    #
    # Deny UDP packets to internal network
    #
    iptables -A FORWARD -d $INTERNAL_SUBNET -i $IFNAME -p udp \
            -m limit --limit 6/h --limit-burst 5 -j LOG

    iptables -A FORWARD -d $INTERNAL_SUBNET -i $IFNAME -p udp -j DROP

    iptables -A INPUT -d $INTERNAL_SUBNET -i $IFNAME -p udp \
            -m limit --limit 6/h --limit-burst 5 -j LOG

    iptables -A INPUT -d $INTERNAL_SUBNET -i $IFNAME -p udp -j DROP

    # Don't log Windows/Samba name broadcasts as they will occure often
    iptables -A INPUT -d $DMZ_IP -i $IFNAME -p udp --dport 137 -j DROP

    iptables -A INPUT -d $DMZ_IP -i $IFNAME -p udp \
            -m limit --limit 6/h --limit-burst 5 -j LOG

    iptables -A INPUT -d $DMZ_IP -i $IFNAME -p udp -j DROP

    #
    # Deny selected ICMP to internal network
    #
    iptables -A FORWARD -d $INTERNAL_SUBNET -i $IFNAME -p icmp \
            --icmp-type echo-request -j DROP
    iptables -A FORWARD -d $INTERNAL_SUBNET -i $IFNAME -p icmp \
            --icmp-type redirect -j DROP
    iptables -A FORWARD -d $INTERNAL_SUBNET -i $IFNAME -p icmp \
            --icmp-type router-advertisement -j DROP

    iptables -A INPUT  -d $INTERNAL_SUBNET -i $IFNAME -p icmp \
            --icmp-type echo-request -j DROP
    iptables -A INPUT -d $INTERNAL_SUBNET -i $IFNAME -p icmp \
            --icmp-type redirect -j DROP
    iptables -A INPUT -d $INTERNAL_SUBNET -i $IFNAME -p icmp \
            --icmp-type router-advertisement -j DROP

    iptables -A INPUT  -d $DMZ_IP -i $IFNAME -p icmp \
            --icmp-type echo-request -j DROP
    iptables -A INPUT -d $DMZ_IP -i $IFNAME -p icmp \
            --icmp-type redirect -j DROP
    iptables -A INPUT -d $DMZ_IP -i $IFNAME -p icmp \
            --icmp-type router-advertisement -j DROP

    #
    # Deny spoofed packets from DMZ
    #
    iptables -A INPUT -s ! ${DMZ_IP}/24 -i $IFNAME -j DROP
    iptables -A FORWARD -s ! ${DMZ_IP}/24 -i $IFNAME -j DROP

}

stop_firewall() {

    #
    # Remove all rules added on the IFNAME interface
    #
    iptables -S | \
            egrep "${IFNAME}" | \
            egrep "^-A " | \
            sed "s/-A //" | \
            while read rulespec; do
                iptables -D ${rulespec}
            done
}

stop_networking() {

    tunctl -d ${IFNAME} >/dev/null

}

# Hack for POSIX echo -n support on all platforms
if [ "X`echo -n`" = "X-n" ]; then
    echo_n() { echo ${1+"$@"}"\c"; }
else
    echo_n() { echo -n ${1+"$@"}; }
fi

#
# Begin Main
#

userid=`id | sed "s/^uid=\([0-9][0-9]*\).*$/\1/"`
if test "${userid}" -ne 0; then
    echo "Error: You must be root to run this program" 1>&2
    exit 1
fi

if [ -z `which iptables` ]; then
    echo "Error: Please install iptables." 1>&2
    exit 1
fi

if [ -z `which socat` ]; then
    echo "Error: Please install socat." 1>&2
    exit 1
fi

if [ -z `which tunctl` ]; then
    echo "Error: Please install tunctl." 1>&2
    exit 1
fi

case "${1}" in
    start)
        start_vm
        ;;

    stop)
        stop_vm
        ;;

    forcekill)
        forcekill
        ;;

    restart)
        stop_vm
        sleep 1
        start_vm
        ;;

    stopfirewall)
        stop_firewall
        ;;

    startfirewall)
        start_firewall
        ;;

    status)
        status
        ;;

    *)
        echo "Usage: vm-dmz-controller start|stop|restart|status" >&2
        echo "Advanced Options: stopfirewall|startfirewall|forcekill" >&2
        exit 1
        ;;
esac
Posted in General | Leave a comment