VPN tunnels between US and China dropping/slow

At my current place of business, we have several manufacturing plants in
China as well as the United States. All of the plants have an OVPN tunnel to
a datacenter here in Indianapolis which connect all of the plants. Our China
plants pay for the basic 3mbit/3mbit fiber internet connections. I've had a
hell of a time keeping their tunnels up. They're running on port 443 over
TCP now, but every month or so the tunnel degrades so badly I have to switch
the port. I've recently tried tunneling OVPN (UDP) over a GRE tunnel and
that has worked for a few months..but even now is degrading. The interesting
thing is that ONLY the tunnel traffic gets degraded. I've replaced all of
the equipment on both ends of all of the VPN tunnels, which changed nothing.

Currently, we're talking to Time Warner and some of our customers who have
plants in China to see what solutions they're using to get around this kind
of issue. One thing we are hearing quite often is that they're using a MPLS
based connection to Hong Kong, then going to the USA from there. We're happy
to try this, but due to cost issues we're (management mostly) considering
this a last resort option. Are there any other options maybe some of you
have to fixing this issue? Thanks

Thomas York

At my current place of business, we have several manufacturing plants
in China as well as the United States. All of the plants have an OVPN
tunnel to a datacenter here in Indianapolis which connect all of the
plants. Our China plants pay for the basic 3mbit/3mbit fiber internet
connections. I've had a hell of a time keeping their tunnels up.
They're running on port 443 over TCP now, but every month or so the
tunnel degrades so badly I have to switch the port. I've recently
tried tunneling OVPN (UDP) over a GRE tunnel and that has worked for
a few months..but even now is degrading. The interesting thing is
that ONLY the tunnel traffic gets degraded. I've replaced all of the
equipment on both ends of all of the VPN tunnels, which changed
nothing.

This is actually caused by the Chinese firewall trying to reset the VPN
connection. The reason why they are doing this is because people are
buying VPN services to get around the firewall. As of late, they have
become a lot more clever about VPN blocking.

Currently, we're talking to Time Warner and some of our customers who
have plants in China to see what solutions they're using to get
around this kind of issue. One thing we are hearing quite often is
that they're using a MPLS based connection to Hong Kong, then going
to the USA from there. We're happy to try this, but due to cost
issues we're (management mostly) considering this a last resort
option. Are there any other options maybe some of you have to fixing
this issue? Thanks

The only option is to get transport to an endpoint outside China, e.g.
in Hong Kong.

William

or just tunnel without a protocol... or spread-spectrum across more
than one endpoint set

At my current place of business, we have several manufacturing plants in
China as well as the United States. All of the plants have an OVPN tunnel to
a datacenter here in Indianapolis which connect all of the plants. Our China
plants pay for the basic 3mbit/3mbit fiber internet connections. I've had a
hell of a time keeping their tunnels up. They're running on port 443 over
TCP now, but every month or so the tunnel degrades so badly I have to switch
the port. I've recently tried tunneling OVPN (UDP) over a GRE tunnel and

Perhaps a DPI issue ? We make use of OpenVPN a lot here. When the
local ILEC started rolling out their DPI boxes, our VPN traffic was
initially identified as bit torrent traffic and was being tampered with.
Of course they said that was impossible... It took a good month before
I was able to get to the right people to actually look at the pcaps that
demonstrated the issue. I setup an openvpn tunnel between the two
impacted sites (A,B)

From A, I would do a straight up icmp ping to B. It would get to the

other side 100% clean.

At the same, time, I would do a ping inside the VPN tunnel. It would
show dropped packets.

I then used hping to generate UDP packets of the same size or bigger of
the VPN packets, but with all FF as the payload, so it didnt look like
anything to the DPI boxes. This too would get to the other side 100% of
the time. But the VPN UDP packets would experience loss. The DPI
vendor then made some patches and/or config changes to stop messing up
our traffic and we have been ok since.

Not sure what you can do on the China side to test things, but perhaps
setup an OpenVPN instance in one of those free test instances in Amazon
and see if you see the loss from there to China.

  ---Mike

Realize also that China Telecom is congested both internally and on
certain peering interfaces.

While DPI is a likely culprit, be sure to not overlook a good
old-fashioned inability to manage capacity, combined with certain
hashing algorithms...

-a

Realize also that China Telecom is congested both internally and on
certain peering interfaces.

While DPI is a likely culprit, be sure to not overlook a good
old-fashioned inability to manage capacity, combined with certain
hashing algorithms...

if you're measuring the end-to-end path you'll likely see evidenced of
the latency climbing on a near daily cycle.

my median rtt from the us east coast is 268ms sometimes it's north of
370 with essentially the same loss properties.