What would be great though is a system where there is an automatic check to see if there is any return traffic for what a customer sends out. If someone keeps sending traffic to the same destination without anything coming back, 99% chance that this is a denial of service attack
This is fine until a customers sends out legitimate multicast traffic, so any such scheme has to ignore multicast traffic. Then the worms and virus writers will just switch to using multicast as a vector.
Yes, that would be cool. I'm surprised that Microsoft doesn't send out its updates over multicast yet. That would save them unbelievable amounts of bandwidth: all Windows boxes simply join the windows update multicast group so they automatically receive each and every update. But we can safely assume they won't use single source multicast so it's only a question of time before some industrious worm builder creates the ultimate worm: one that infects all windows systems world wide by sending a single packet to the windows update multicast group...
Ok, this could happen if:
1. more than five people world wide had interdomain multicast capability
2. anyone with multicast capability could send to any multicast group
And besides, this will happen if possible regardless of the utility of unicast for worm propagation.
Also this only works where routing is strictly symmetrical (e.g. edge connections, and to single homed edges at that).
It also has the problem that you have to retain some state (possibly little) for all outbound traffic until you can match it to inbound traffic. Given the paupacity of memory in most edge routers this is a problem. Even with a decent amount of memory, it would soon get overrun, even on a slowish circuit like a T1. A DSLAM with several hundred DSL lines would need lots of memory to implement this, and lots of CPU cycles to manage it.
Give implementers a little credit. There is no need to do this for every packet that flows through a box. You can simply sample the traffic at regular intervals and perform the return traffic check for only a small fraction of all traffic. Statistics is on your side here, as with Random Early Detect congestion/queue management, because you automatically see more packets from sources that send out a lot of traffic.
At the layer 3 level, all TCP traffic is revertive as it has to send ACKs back so this scheme can't simply work on '"I've seen another packet in the reverse direction, so it's OK".
That's exactly why this works: if the other end sends ACKs, then obviously at _some_ level they're willing to talk. So that would indeed be ok. With DOS and scanning this is very different: for many/most/all packets sent by the attacking system, nothing comes back, except maybe a port unreachable or RST.