[Debian] OpenVPN, IPTables, NAT Issues with UDP Translation
after IP Change
Ola Lundqvist
ola at inguza.com
Fri Sep 16 08:47:22 EDT 2011
Hi Mikhael
Quoting "Michael Flaig" <mflaig at googlemail.com>:
> Hi Ola,
>
> thanks for your reply. I sent my mail to users list again because of the
> cron's coming in and I didn't get my mail back from the list.
>
> On Fri, 2011-09-16 at 07:33 +0200, Ola Lundqvist wrote:
>> Hi Michael
>>
>> To me it looks like you are trying to do something that is outside the
>> standard setup of openvz. It may be so that it works, but as you see
>> there could be problems. I have some further comments below.
>
> I agree, that my setup is somehow non-standard. Up until now it went
> pretty smooth however.
Ok.
>> Are there any specific reasons why you want to do all this within the VE
>> and not outside it?
>
> The host should not be exposed to the outside world.
I see. Tricky thing. But can't you expose the VE ip from the host in
that case? It would be a bit more complex netfilter rules however so
maybe it is too complicated.
>> On Thu, Sep 15, 2011 at 10:42:49AM +0200, Michael Flaig wrote:
>> > Hi Folks,
>> >
>> > I have a Debian host running multiple VEs and seeing trouble with udp
>> > translations (masq or snat) when the interface ip of ppp0 changes.
>>
>> Is VE102 and VE101 on the same server in the picture below?
>
> yes
Ok. thanks.
>> > A little about the Setup (Stripped down to the problem)
>> >
>> > VE0 Squeeze
>> > VE101 Firewall / NAT / PPPOE
>> > VE102 Firewall (all accept) / OpenVPN Client
>> >
>> > ________
>> > | VE 102 | Openvpn Client
>> > |________|
>> > |veth102.0
>> > ________|__________________________________
>> > bridge |veth101.0
>> > ____|____
>> > Firewall | VE 101 |_____ppp0
>> > NAT |_________|
>> >
>> >
>> > Then there is a OpenVPN gateway on the NET somewhere behind ppp0 that
>> > acts as Server and gets connected by VE102's Openvpn CLient.
>> >
>> > The Firewall Policy on VE101 is build by ferm - nothing fancy yet.
>> > * Allows forwarding of packets from openvpn client to the outside
>> > world
>> > * Does a MASQUERADE to outer-interface ppp0
>> > * Also tried SNAT, same effect.
>> > * The iptables rules are generated right with MASQ (of course) and
>> > also on my SNAT test
>> >
>> > PPP0 is brought up the debian way with /etc/network/interfaces and
>> > ifup/ifdown
>> >
>> > The Policy of VE101 looks like (input and output chains stripped):
>> > --- snip ---
>> > @def $IP_WORLD = `ip addr show ppp0 | grep inet | awk '{print $2}'`;
>> >
>> > table filter {
>> >
>> > [...]
>> >
>> > chain FORWARD {
>> > policy DROP;
>> >
>> > # connection tracking
>> > mod state state INVALID DROP;
>> > mod state state (ESTABLISHED RELATED) ACCEPT;
>> >
>> > # connections from the internal net to the internet or to other
>> > # internal nets are allowed
>> > interface $DEV_PRIVATE ACCEPT;
>> >
>> > # Reject, the rest is dropped by the above policy
>> > REJECT;
>> > }
>> > }
>> >
>> > table nat {
>> > chain POSTROUTING {
>> > # masquerade private IP addresses
>> > saddr $NET_PRIVATE outerface $DEV_WORLD MASQUERADE;
>> > # saddr $NET_PRIVATE outerface $DEV_WORLD SNAT to $IP_WORLD;
>> > }
>> > }
>> > --- snap ---
>> >
>> > Now the whole setup works at first. Tunnel comes up. All good.
>> > Until the first IP change. The Tunnel is down but I see connections
>> > incoming on the OpenVPN server.
>>
>> IP address change of what?
>
> Sorry maybe I wasn't clear on that. IP Change on ppp0 of course on VE
> 101. ppp0 is a DSL line with 24 hrs reconnect.
I see. Now I start to understand.
>> > I do a tcpdump on ppp0 and see outgoing packets that get translated to
>> > the old IP address (X.X.X.X) instead of the new IP address.
>> >
>> > 09:50:09.628341 IP X.X.X.X.20001 > Y.Y.Y.Y.20001: UDP, length 14
>> >
>> > I flushed the rules, reinstalled the rules - not helping.
>> >
>> > Now /proc/net/conntrack on VE101 tells me that the old connection is
>> > there in state ASSURED
>> >
>> > udp 17 166 src=VE102 dst=Openvpnserver sport=20001 dport=20001
>> > packets=435 bytes=23129 src=Openvpnserver dst=X.X.X.X sport=20001
>> > dport=20001 packets=92 bytes=9701 [ASSURED] mark=0 secmark=0 use=2
>> >
>> > Hm ... okay. weird. Shouldn't this be flushed on an IP change?
>> > Well as UDP is stateless and iptables makes it stateful, however source
>> > and destination port do not change, this old conntrack rule is still
>> > used but should not be there anymore IMHO.
>> >
>> > So what about deleting it?
>> >
>> > ON VE101
>> > conntrack -D -s VE102
>> > conntrack v0.9.14 (conntrack-tools): Operation failed: Connection refused
>> >
>> > strace for conntrack:
>> > socket(PF_NETLINK, SOCK_RAW, 12) = 3
>> > getsockname(3, {sa_family=AF_NETLINK, pid=0, groups=00000000}, [12]) = 0
>> > bind(3, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0
>> > getsockname(3, {sa_family=AF_NETLINK, pid=947, groups=00000000}, [12]) =
>> > 0
>> > bind(3, {sa_family=AF_NETLINK, pid=947, groups=00000000}, 12) = 0
>> > socket(PF_NETLINK, SOCK_RAW, 12) = 4
>> > getsockname(4, {sa_family=AF_NETLINK, pid=0, groups=00000000}, [12]) = 0
>> > bind(4, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0
>> > getsockname(4, {sa_family=AF_NETLINK, pid=-4269, groups=00000000}, [12])
>> > = 0
>> > bind(4, {sa_family=AF_NETLINK, pid=-4269, groups=00000000}, 12) = 0
>> > sendto(3, "\24\0\0\0\1\1\1\3\270\260qN\0\0\0\0\2\0\0\0", 20, 0,
>> > {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = -1 ECONNREFUSED
>> > (Connection refused)
>> > close(4) = 0
>> > close(3) = 0
>> > write(2, "conntrack v0.9.14 (conntrack-too"..., 37conntrack v0.9.14
>> > (conntrack-tools): ) = 37
>> > write(2, "Operation failed: Connection ref"..., 36Operation failed:
>> > Connection refused) = 36
>> > write(2, "\n", 1
>> > ) = 1
>> > exit_group(1) = ?
>> >
>> > So strace ends on a socket connect with a connection refused error.
>> > Deleting the conntrack rule is not allowed inside VE101. Maybe missing
>> > something in the VE 101 config? Could this error also be the reason for
>> > not flushing those conntrack rules?
>>
>> No as you can see both the socket creation and the bind succeeds. It is
>> sendto that fails. I can not really explain that however.
>>
>> Is VE102 an IP address in the call above?
>
> Yes the IP of VE102. Same in /proc/net/conntrack
Ok.
>> What happens if you run this outside the VE?
>
> That works, not with the VE102 IP of course as /proc/net/ip_conntrack
> does not show the connections of any VE other than VE0 - which is right
> otherwise it would be a security problem on the host.
Ok, thanks.
>> > --- snip ---
>> > [... non-relevant parts cut ...]
>> > IPTABLES="ip_tables iptable_filter iptable_mangle iptable_nat ipt_limit
>> > ipt_multiport ipt_tos ipt_TOS ipt_REJECT ipt_TCPMSS ipt_tcpmss ipt_ttl
>> > ipt_length ip_conntrack ip_conntrack_ftp ip_conntrack_irc ipt_conntrack
>> > ipt_state ipt_helper ip_nat_ftp ip_nat_irc ipt_REDIRECT xt_mac
>> > ipt_recent ipt_owner"
>> > FEATURES="ppp:on "
>> > DEVICES="c:108:0:rw "
>> > CAPABILITY="NET_ADMIN:on NET_RAW:on SYS_ADMIN:on "
>> > --- snap ---
>> >
>> > On the CAP - this VE101 is running quagga.
>> >
>> > Oh of course this problem can be fixed stop and start of VE101 - but
>> > that should not be the solution here on a 24 hr ip change :-/
>> >
>> > Any suggestions? I'm pretty much out of google hints and ideas myself.
>> > Oh there are a few forum posts about AF_NETLINK in russian though.
>> >
>> > Should conntrack-tools work at all inside a VE and if yes anyone know
>> > how to make them work?
>>
>> Hard to tell. I do not think people have specifically looked in how
>> to make it work. It is interfacing directly to the kernel (as you can
>> see by the netlink socket above) and that may have problems.
>
> Agreed, however I see 2 problems here, looking related:
> * Conntrack does not get cleared of old sessions on an ip change
> and that probably leads to the incorrect address translation.
> However that also means that there are connections left in the
> conntrack table that shouldn't be there anymore.
And this is probably a bug.
> * I cannot access/modify conntrack information with the tools
> (which i think is sad but not essential.
Which may be a bug or a request for feature.
> Should I try raising attention on the devel list?
I suggest you write a bug report. Either in Debian bugtracking system
or in openvz bugtracking system.
Best regards,
// Ola
> Cheers,
>
> Mike
>
>
>
>
--
--- Inguza Technology AB --- MSc in Information Technology ----
/ ola at inguza.com Annebergsslingan 37 \
| opal at debian.org 654 65 KARLSTAD |
| http://inguza.com/ Mobile: +46 (0)70-332 1551 |
\ gpg/f.p.: 7090 A92B 18FE 7994 0C36 4FE4 18A1 B1CF 0FE5 3DD9 /
---------------------------------------------------------------
More information about the Debian
mailing list