Friday, April 17, 2009

Per-process routing

Just a quick tip before the big monthly blog post :-)

Today I was asked by a friend if it was possible to assign different network interfaces to different programs for their Internet traffic...

For example, let's say you're a student living on-campus; the university provides you with broadband Internet access via Wi-Fi, which is great, except for the fact that you cannot trust it (yes, even when you're careful to use HTTPS and so on, I'll cover that in subsequent blog posts). A general solution to that problem would be getting your very own private Internet access, but being a student, you would prefer not to waste too much money into it, so you'll most likely take the cheapest subscription. So now you have two routes to the Internet: a fast but insecure one, and another that is private but slow. How to use both on the same computer? As bandwidth-intensive applications are often also the ones that don't really require privacy, one could imagine categorizing programs in a way so as to watch Internet TV over the Wi-Fi network while corresponding over the cable.

Here's how to do it with Linux, assuming that the default route is your private connection and that your Wi-Fi interface is named ath0, has IP address 10.1.2.3 and gateway 10.0.0.1:
  1. Create a "wifi" user

    adduser wifi

  2. Mark packets coming from the wifi user

    iptables -t mangle -A OUTPUT -m owner --uid-owner wifi -j MARK --set-mark 42

  3. Apply the Wi-Fi IP address on them

    iptables -t nat -A POSTROUTING -o ath0 -m mark --mark 42 -j SNAT --to-source 10.1.2.3

  4. Route marked packets via Wi-Fi

    ip rule add fwmark 42 table 42
    ip route add default via 10.0.0.1 dev ath0 table 42

  5. Launch programs as the wifi user

    sudo -u wifi vlc

Step 1 is of course required only once; steps 2, 3 and 4 are better put together in a shell script. Regarding step 5, it is much more practical to edit your KDE menu entries for example and there specify that the program has to be run as the wifi user.

One last remark: know your localhost network architecture, some of the wifi user's traffic might still go via the default route! This will be the case if, for example, you use a local DNS cache such as NSCD or any other kind of local proxying (Privoxy, Tor, etc).

7 comments:

  1. Interesting idea. This can be useful. Thanks !

    ReplyDelete
  2. Great idea i also searching for same

    ReplyDelete
  3. I am trying to implement your suggestion with slight modifications (instead of over wifi, I'm trying to send the packets out on a tap interface over OpenVPN, and my user is called vpnuser), but something isn't right. The vpnuser packets are still routed through the physical interface, rather than the virtual tap interface.

    Here are the exact commands I used:

    iptables -t mangle -A OUTPUT -m owner --uid-owner vpnuser -j MARK --set-mark 42
    iptables -t nat -A POSTROUTING -o tap0 -m mark --mark 42 -j SNAT --to-source 192.168.10.161

    ip rule add fwmark 42 table 42
    ip route add default via 192.168.10.1 dev tap0 table 42

    where 192.168.10.161 is my OpenVPN IP address.

    If this looks right, can you suggest any good methods for debugging this which would show:

    1) Whether the packets are properly being marked by iptables for user vpnuser
    2) Whether they are ending up in table 42 at all?

    Do you have any other suggestions?

    Thanks!
    Iordan Iordanov

    ReplyDelete
  4. works great! thanks

    ReplyDelete
  5. Three+ years later, one thing has changed - reverse path filtering is now enabled by default on many systems, so the reply packets get dropped by the kernel. The fix is to run "sysctl net.ipv4.conf.ath0.rp_filter=0". Maybe this well help someone else save two hours of intense googling ;)

    ReplyDelete
  6. Thanks for the post. I was able to get it working based on this. I had to disable reverse path routing for what I was trying to accomplish. (Squid and deluge traffic via VPN, everything else local routing)

    I also added a bogus default route to table 42 so that traffic wouldn't go out the local connection if the VPN goes down.

    iptables -t mangle -I OUTPUT -m owner --uid-owner deluge -j MARK --set-mark 42
    iptables -t mangle -I OUTPUT -m owner --uid-owner squid -j MARK --set-mark 42
    iptables -t mangle -I OUTPUT -d 192.168.1.0/24 -m owner --uid-owner deluge -j RETURN
    iptables -t mangle -I OUTPUT -d 192.168.1.0/24 -m owner --uid-owner squid -j RETURN

    iptables -t nat -I POSTROUTING -o tun0 -j MASQUERADE

    IP=`netstat -rn | grep tun0 | grep ^0.0.0.0 | awk '{print $2}'`

    ip rule add fwmark 42 table 42

    ip route add default via 192.168.1.250 table 42
    ip route add 0.0.0.0/1 via $IP table 42
    ip route add 128.0.0.0/1 via $IP table 42

    ip route del 0.0.0.0/1
    ip route del 128.0.0.0/1
    echo 0 > /proc/sys/net/ipv4/conf/tun0/rp_filter

    ReplyDelete
  7. I tried your solution but something doesn't work properly.
    Connections using wifi user won't works at all, apparently they didn't receive any replies but tcpdump shows outgoing and ingoing packets.

    ReplyDelete