Anyone happen to have more information on the problems that have been
happening with the peering between Cogent and Level3. Cogent gives the
standard answer when you call support, but some more specific
information would be great.
Thanks
Dale
Might have to do with
http://isp-lists.isp-planet.com/isp-bandwidth/0212/msg00978.html
(AOL vs Cogent Peering issue)
---Mike
AOL (AS1668) stopped peering with cogent yesterday for reasons they did
not disclose publicly. Cogent sends same letter to all customers who
asked for what is going on and in the letter they say that two weeks
ago, peering to AOL was upgraded to OC48 from OC12 and now for some
reason AOL stopped peering and if somebody has questions they should call
AOL to complain .. (with phone# to their NOC provided in the letter
- not very nice thing to do it like this in my opinion).
Afterwards direct peering to AOL was lost, cogent started sending out all
their data though Level3 (AOL upstrem), making them pay for that traffic and
because of amount of data the peering links got congested easily - PAIX
cogent->level3 peering in bay area for example had > 1 sec latency
yesterday afternoon. After I spent some time yesterday investigating all
that I found that actually cogent is only using former netrail (AS4006)
peering links to send data out to that is going to AS1668 and only some
of those links. On the other hand most level3 destined data from
cogent is sent through PSI links (i.e. 16631 174 3356).. So for me
currently I put up filters to route all AS1668 and Level3 data through
savvis for routes that look like (16631 4006 3356 ... ) and it works out
fine for now.
Today I'm going to deal with incoming traffic but the problem I had is
that I could not find AOL (AS1668) looking glass, traceroute or some other
tool like that. I checked traceroute.org and was very surprised that large
network like theirs does not have publicly available looking glass listed.
So those of you who peer with them - how do you deal with network
troubleshooting when you need to check soemthing from their network side?
If you know where their looking glass is - please send me a link!
P.S. If somebody is seeing level3->cogent route that is congested I'd also
like to know as this is something I have to deal with today as well. Send
this traceroute to me privately.
If nothing else, Cogent could be using their idle 6461 transit, instead of
grandstanding by overloading their Level 3 capacity so they can blame AOL.
I pointed that out on another list too but somebody else responded that
abovenet to aol connection is congested as it is and more then likely
would not have been able to take all the extra traffic. I'm not a customer
of MFN/abovenet so I really do not know but I did not like it how cogent
deal with it all either - if somebody blames too much somebody else, more
then likely they are at fault to considerable degree...
is it really idle ?
Why wouldn't Cogent create a community string to provide its multihomed
customers with prepend 16631 (or customer asn) to Level3 peering sessions
to control inbounds better?
-Basil
There are no congestion issues between 6461 and 1668 at this time.
If someone believes there is a problem please have them contact me
off list so I can look into it.
I can make no comment on the volume of traffic, having no idea what
volume is involved.
If you consider the paths that are available for a moment, I think
most people can deduce why they are choosing the Level 3 path, and
it has nothing to do with performance.
Me thinks Cogent doesn't have a problem with congestion on the inbound
direction. Fix your reverse path.
Because they do not do custom anything.
Alex
Customers of Cogent should be/are more concerned about congestion on the
inbounds at Level3 <-> Cogent; outbound is way too easy to control.
If the site is multihomed to any decent Tier1 provider +Cogent; (701 or 2914
or 1239 or 3561 or 1, etc +16631) with or without prepend 16631 or site ASN,
a lot of, if not all of the inbound traffic will still be going through one
of those better carriers, fortunately or unfortunately.
All I really want is to be able to control my inbound from Level3 
-Basil
> Me thinks Cogent doesn't have a problem with congestion on the inbound
> direction. Fix your reverse path.Customers of Cogent should be/are more concerned about congestion on the
inbounds at Level3 <-> Cogent; outbound is way too easy to control.
Cogent has a pile of available inbound - websites tend to send traffic out,
not take traffic in.
Alex
Thing is if your connection is completely full one way, it'll effect
traffic the other way too. It should not be happening with syncronyous
connections, but practical observation is that it does! I suspect router
hardware is to blame (possibly packet cache is way full) and I'v seen
it happen only if you try to send 100% more traffic then link can handle
(just 100% traffic does not efect it - have to really push it), this
happened on 100Mb and even on Gb interface.
Somewhat true. Yet still if the inbound from one of the major players is
really saturated wouldn't that hurt Cogent customers.
-Basil
william@elan.net wrote:
Thing is if your connection is completely full one way, it'll effect traffic the other way too. It should not be happening with syncronyous connections, but practical observation is that it does! I suspect router hardware is to blame (possibly packet cache is way full) and I'v seen it happen only if you try to send 100% more traffic then link can handle (just 100% traffic does not efect it - have to really push it), this happened on 100Mb and even on Gb interface.
Or could it be that your ACK packets just simply get delayed enough for the traffic
to the other direction to suffer somewhat?
This is quite common phenomenan in asymmetric links but also exists for symmetric
ones.
Pete
> > > Me thinks Cogent doesn't have a problem with congestion on the inbound
> > > direction. Fix your reverse path.
> >
> > Customers of Cogent should be/are more concerned about congestion on the
> > inbounds at Level3 <-> Cogent; outbound is way too easy to control.
>
> Cogent has a pile of available inbound - websites tend to send traffic out,
> not take traffic in.Somewhat true. Yet still if the inbound from one of the major players is
really saturated wouldn't that hurt Cogent customers.
When you have symmetric links [same up/down] (which basically all links are)
the probability of the inbound link from others being saturated for a
company that provides mostly transit to webhosters is nill to nothing.
Alex
Thing is if your connection is completely full one way, it'll effect
traffic the other way too. It should not be happening with syncronyous
connections, but practical observation is that it does! I suspect router
hardware is to blame (possibly packet cache is way full) and I'v seen
it happen only if you try to send 100% more traffic then link can handle
(just 100% traffic does not efect it - have to really push it), this
happened on 100Mb and even on Gb interface.
If your circuit is full one way, it really makes no sense to be bothered
dealing with reverse path *before* fixing your forward path. Fix the
outbound (saturated) first, *then* look at the rest.
Alex
Of course, your right about what needs to be fixed! But situation with
cogent is such that I do not have that option. Their peering link with
level3 is congested because of all the traffic going to AOL and some of
traffic destined to me is going through same link the other way and
getting jammed as a sideeffect as well. I route aol-destined traffic from
my net to provider other then cogent - but how do I tell AOL and L3 (and
only them) that best path to me is through somebody else? Basil is right -
the best way to deal with that would be for cogent to provide special
community that would allow me to direct cogent to prepend several of their
ASN to level3 advertisements.
Of course, your right about what needs to be fixed! But situation with
cogent is such that I do not have that option. Their peering link with
level3 is congested because of all the traffic going to AOL and some of
traffic destined to me is going through same link the other way and
getting jammed as a sideeffect as well. I route aol-destined traffic from
my net to provider other then cogent - but how do I tell AOL and L3 (and
only them) that best path to me is through somebody else?
route-map cogent-ick permit 10
match community-list <aggregate-only>
set as-path prepend <iamuglypath> <iamuglypath> <iamuglypath>
route-map cogent-ick deny 20
match community-list <deaggregated-routes>
route-map happy-with-backup permit 10
match community-list <deaggregated-routes>
route-map happy-with-backup deny 20
match community-list <aggregate-only>
Basil is right - the best way to deal with that would be for cogent to
provide special community that would allow me to direct cogent to prepend
several of their ASN to level3 advertisements.
Cogent doesnot do anything custom.
Alex
In the middle of all of this talk re: Cogent and such,
they apparantly spent some $$$ somewhere and made our
test connection a whole lot better.
In a nutshell, we did this a month ago and it stank.
Bursty, high and variable latencies.. etc..
Apparently they fixed something.
Yesterday in our first test on the new stuff,
in 5 FTP sessions to various FTP sites we generated
a smooth 20-30mbps
So, both iperf and the web based speed test are both back online:
Please abuse and test it for a day or two..
It is supposedly an OC3 to Cogent..
It's 100mbps fdx on the test machine..
Nail that mother to the wall.. PLEASE.
All test results will be made public. Iperf and Web, complete with
the MTR traceroutes - just in case anyone wants to peek.
--Mike--