SaidCom disconnected by Level 3 (former Telcove property)

http://www.phillyburbs.com/pb-dyn/articlePrint.cfm?id=1310151

Is this a normal thing for Level 3 to do, cut off small, responsive
providers?

Frank

I heard from a few folks working for TelCove that they were under order to do whatever it took to disconnect customers under certian levels in my local area (not philly).

Related? Dunno.

Level3 recently stiffed us on a colo contract though, 3 weeks past the "no later than" install date and we finally canceled it. (mind you that was 7 weeks after the contract was signed).

~shrug~

..d

As a former Broadwing customer (now a Level3 customer), things like this worry me a bit… (and tend to keep my up at night).

-brandon

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Frank Bulk wrote:

http://www.phillyburbs.com/pb-dyn/articlePrint.cfm?id=1310151

Is this a normal thing for Level 3 to do, cut off small, responsive
providers?

Frank

- ------------------------
Just curious, should "small responsive providers" should be multi-homed?

regards,
/virendra

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

virendra rode // wrote:

Frank Bulk wrote:

http://www.phillyburbs.com/pb-dyn/articlePrint.cfm?id=1310151

Is this a normal thing for Level 3 to do, cut off small, responsive
providers?

Frank

------------------------
Just curious, should "small responsive providers" should be multi-homed?

regards,
/virendra

- --------------------------
Sorry my keyboard came before coffee.

I meant to say, shouldn't "small responsive providers" be multi-homed?

regards,
/virendra

All companies in all industries have a policy of stopping to provide
their services if a customer stops paying or violates the contract, I
really don't see this as a big/little provider argument.

Yes, the small provider should be multi-homed, otherwise a fiber cut or
outage can have this same effect.

To me the only part of this that is up for argument is did SaidCom
actually violate the contract and/or terms of use, and I certainly don't
have enough information beyond that one article to make that decision.
If someone else does please share with the group.

  -Scott

(As always just my 2 cents)

To me the only part of this that is up for argument is did SaidCom
actually violate the contract and/or terms of use, and I certainly don't
have enough information beyond that one article to make that decision.
If someone else does please share with the group.

I will say that I've used Level(3) as a transit provider (not the only
one, of course) at 2 different companies (both of which were / are
pretty responsive to abuse complaints, in one case because I was the one
responding to them), and I have never had an issue like this.

However, both of these were content / hosting type setups, not ISPs who
provide local Internet access, so the type of complaints (and probably
the volume) were different than your typical ISP.

I don't think there's any way to know for sure who screwed up here...
maybe the small provider wasn't as responsive as they say they are, but
are trying to shift the blame to (3). Or maybe (3) was trigger-happy. I
don't think anyone from the outside has the information to tell for sure
which happened.

But Level(3) is a pretty big provider, and if they were just shutting
off small competitors willy-nilly, I imagine we'd be hearing about it
more.

w

Not knowing anything about the case other than what I read in the article, my hang up is that a transit provider can make a phone call and destroy a customer's business with 30 minutes notice. On a DS3 that has actual real lead time to replace, that's a business killer. The argument of "should be multi homed" holds some water, but I've never considered multihoming as a typical remedy for a 30-90 day outage. And then it only works if lines are underutilized to the point of loosing one will constantly have zero affect on network performance, even during peak use, and if there's still some level of extra redundancy remaining. (Multiple contingency situations aside)

My opinion is probably somewhat influenced by the fact that I'm a small ISP with customers that want the internet to NOT be slow, facing that same DS3 lead time problem. I ordered a DS3 in early December, (who's local loop was to ride on a preexisting OC3, sounds easy, right?) and with dates slipping over and over again, and with no firm install date in site provided from the company last week, I had to finally cancel and order with a different company last week. For the last month, the last thing I wanted to do was "punt" and start the process over again, but at some point, one starts to feel "choiceless."

"Do you think I placed that order in December just for fun?"

I see talk over and over again on NANOG about "Maybe some provider will come in with [insert new technology here] and compete with the cable/DSL providers" but as a small provider doing fix broadband wireless, I just don't see how even an army of small providers can compete against the likes of TENS OF BILLIONS OF DOLLARS of cable/telco market capitalization.

After fighting Qwest for ten years, maybe I'm starting to feel a little hopeless.

Even from that one-sided account, I have serious problems with:

  Siwert said the Colorado-based Level 3 "cited several Internet
  abuses" by SaidCom customers as the reason for the shutdown, including
  spam problems. "Some customers abuse the system", but when that
  happens, SaidCom contacts the authorities, said Siwert.

When we have a customer spamming, we don't call the police. We either talk to, ACL, or shut off the customer. The above suggests to me that SaidCom had spam issues that they were either unable or unwilling to remedy themselves.

I also doubt that L3 shut them off without multiple prior warnings, though anything's possible. We had an issue many years ago where a leased line provider (coincidentally borged indirectly into L3) shut off all of our services with no warning at all on a Friday evening. Only after wasting some time with their NOC troubleshooting the circuits were we told we'd been shut off for non-payment. When we eventually got to the bottom of it (many hours later), it turned out they had another customer with a similar name that was way past due...and when billing told their NOC shut off "Atlantic" they shut off everything that looked like "Atlantic" even though we were two totally separate and unrelated customers.

ACL's are your friends with non-responsive customers. ;-), But maybe SaidCom did not know better.

How many Abuse tickets had they received from TelCove/(3) over what time frame?

I may be way off base here: Only knowing the "facts" presented in the above article, the "Abuse" complaints may have also included DMCA complaints, which, if not responded to in a timely manner, could also have resulted in liability for (3).* As per the quote above, the abuses included spam, he did not say they "were exclusively spam".

*I am not a lawyer, nor do I play one on TV.

Just reading between the lines here...

G

Probably no one besides SaidCom and Level3 knows the whole story. If you do some more research you will find allegations, besides spam, that
would involve law enforcement. I don't repeat them here because I don't know if they are true or not.

Skipping the faces (or lack of facts ) in this particular case.

People criticize ISPs for not shutting down customers. And people criticize ISPs when they do shut down customers. Whether it is a 10Gig link or a DSL/Cable link, do bad guys get more leeway just because they pay for more expensive circuits? Or should an ISP shutdown customers as soon as possible when they violate the AUP/TOS regardless who it is.
Three warnings, six warnings, six hundred warnings, how many is too
many?

Would you be more careful about your computers being compromised, copyright infringement, signing up downstream customers if you knew
violations of your ISP's AUP would result in disconnection? Everyone always wants the other guy's circuit terminated when something
bad happens, but never wants their own circuit terminated when they
screw up.

How many people thank the police officer for stopping them and giving
them a ticket for violating traffic rules?

I do, but perhaps I'm uncommon in this regard.

Your larger point, however, is completely valid: there
is a relatively normal desire to have rules enforced
on other people with more zeal than one would choose
for oneself.

Perhaps more transparency is a tonic for this? If ToS
and the AUP are more clearly written and enforced as
consistently as possible, I would expect customers to
be less horked off by AUP/ToS shutdowns.

It does surprise me that no enterprising person/group
has turned this into a salable feature: "we're the
network which shuts down spammers/infected/baddies."
I could imagine that there would be customers who
would rather give their business to providers who are
more active in this regard than less, and that would
be a way for a service provider to differentiate
themself from the rest of the pack.

-David

David Barak
Need Geek Rock? Try The Franchise:
http://www.listentothefranchise.com

People try that. They then get DDoS'ed. Then they stop.
(Thats the people that try to do it by providing internet-based services.
People who sell products probably fare slightly better.)

Adrian

IMHO being the good cop has never been a mass-marketable feature, whether
we're talking spam, botnets, phising, cracking attempts, whatever...

Almost ALL providers should be multihomed.

--Mike

Almost ALL?

Any company, or any person for that matter, that relies on their Internet connectivity for their lively hood should be multihomed.

-wil

Some locations are just too cost prohibitive to multihome, but that really
is a select few. Few places are out of the reach of a couple wireless hops
back to civilization.

--Mike

Surely all those except those who are competing with you for the same customers should multi-home. :slight_smile:

Joe

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Some locations are just too cost prohibitive to multihome, but that really
is a select few.

It isn't just cost but can be path diversity (or lack thereof). We used to be headquartered 210 miles from civilization. We had a choice of providers and could have "multihomed". However, the only realistic way for any of those providers to get to us would have been Bell frame relay. Since by far the most likely point of failure was the "last mile" (which was 210 miles), we made a decision that actually multihoming wasn't a good use of resources. We instead went with a good quality regional provider who was themselves multihomed. Now clearly there were cases where that wouldn't have any good but given the remoteness it just seems most likely that anything that took out one provider would have taken the other one as well.

Now this case we are discussing is probably the exception to our assumptions but we had a much better provider at the time than Level3 ;-]

From the sounds of the original post I wouldn't be too surprised if it was also fairly remote.

Chris

True :slight_smile: I'd also think (read: hope) if an organization was located in an area where multi-homing was not possible, then that organization and its customers would not be doing things that are mission critical, i.e. business stops if there is no Internet connectivity.

jms