VZ FiOS DNS issues:

Mailman List Mirror (Read Only)
1

Any Verizon techs around today? I don't know why you can't pass DNS traffic this morning, but it's the second time in as many weeks as it has been an issue, and it's rather annoying (Google is the example, but the exact same failure happens using any destination, on VZ's own or any other public DNS servers, phone support are of course, useless):

C:\Users\jamie>tracert -d 71.252.0.12

Tracing route to 71.252.0.12 over a maximum of 30 hops

  1 <1 ms <1 ms <1 ms 192.168.2.254
  2 <1 ms <1 ms <1 ms 192.168.1.1
  3 8 ms 9 ms 13 ms 96.231.199.1
  4 14 ms 9 ms 9 ms 130.81.183.118
  5 9 ms 9 ms 9 ms 130.81.151.232
  6 9 ms 9 ms * 130.81.20.19
  7 11 ms 9 ms 9 ms 71.252.0.12

Trace complete.

C:\Users\jamie>nslookup www.google.com 71.252.0.12
Server: nsrest01.verizon.net
Address: 71.252.0.12

DNS request timed out.
    timeout was 2 seconds.
DNS request timed out.
    timeout was 2 seconds.
DNS request timed out.
    timeout was 2 seconds.
DNS request timed out.
    timeout was 2 seconds.
*** Request to nsrest01.verizon.net timed-out

C:\Users\jamie>tracert -d 8.8.8.8

Tracing route to 8.8.8.8 over a maximum of 30 hops

  1 <1 ms <1 ms <1 ms 192.168.2.254
  2 <1 ms <1 ms <1 ms 192.168.1.1
  3 7 ms 8 ms 9 ms 96.231.199.1
  4 8 ms 9 ms 8 ms 130.81.183.118
  5 9 ms 28 ms 10 ms 130.81.22.56
  6 8 ms 9 ms 9 ms 152.63.36.237
  7 20 ms 19 ms 19 ms 152.63.0.153
  8 21 ms 18 ms 18 ms 152.63.21.73
  9 41 ms 47 ms 49 ms 152.179.72.66
10 17 ms 18 ms 19 ms 209.85.255.68
11 * * * Request timed out.
12 * * * Request timed out.
13 22 ms 19 ms 19 ms 72.14.236.200
14 20 ms 31 ms 18 ms 216.239.49.145
15 18 ms 19 ms 19 ms 8.8.8.8

Trace complete.

C:\Users\jamie>nslookup www.google.com 8.8.8.8
Server: google-public-dns-a.google.com
Address: 8.8.8.8

DNS request timed out.
    timeout was 2 seconds.
DNS request timed out.
    timeout was 2 seconds.
DNS request timed out.
    timeout was 2 seconds.
DNS request timed out.
    timeout was 2 seconds.
*** Request to google-public-dns-a.google.com timed-out

C:\Users\jamie>

2

I have FIOS and I have no issues. However I do know awhile back they had issues and I was affected by
the outage....

Maybe it hasn't made its way to me yet....

3

there have been instances over the time i've been a fios customer that
'upgrades' to devices in the field have caused this problem (last was
~2wks ago? in the washington, dc area).

Could be you are seeing this problem affecting you :frowning:

-chris

4

Any Verizon techs around today? I don't know why you can't pass DNS traffic this morning, but it's the second time in as many weeks as it has been an issue, and it's rather annoying (Google is the example, but the exact same failure happens using any destination, on VZ's own or any other public DNS servers, phone support are of course, useless):

Have a look at:

Are you by chance in So Cal? VZ has been having some serious pot holes on their information super highway of late.

Regards,

James Laszko
Mythos Technology Inc

5

Try a full rebind on your cpe or power cycle, whichever is easier. This seems to have worked for a few on the forums.

6

Christopher Morrow <morrowc.lists@gmail.com> writes:

I have FIOS and I have no issues. However I do know awhile back they had issues and I was affected by
the outage....

Maybe it hasn't made its way to me yet....

there have been instances over the time i've been a fios customer that
'upgrades' to devices in the field have caused this problem (last was
~2wks ago? in the washington, dc area).

Could be you are seeing this problem affecting you :frowning:

I'm a FIOS customer (LATA 246 not 236 like Chris), and haven't had any
issues with the network. On the other hand, between my location and
the fact that I'm on an old BPON build, perhaps the software upgrades
haven't affected me. To further complicate things, ever suspicious of
ISP nameservers that don't do DNSSEC validation and monetize rcode 3,
and not a fan of the Actiontec boxes that Verizon hands out I run my
own cacheing nameserver (hand-built openbsd+pf on embedded hardware
with latest bind or unbound and isc dhcpd).

Do things magically start working for you if you hard-code 8.8.8.8 or
4.2.2.1 or one of the other usual suspects? That would seem to be a
quick way of narrowing it down a bit.

-r

7

I don't care for the Actiontec boxes either, but the STB program guides and other features don't work without it, so I have mine forward all IP traffic unmolested to my own as the DMZ host (thus the dual layer of [P|N]AT you see). It's just UDP/TCP 53 traffic that's not flowing for whatever reason; it's every device in the house phones, tablets, computers, you name it, so I'm not inclined to attribute it to malware. My neighbor was also seeing it (and like last time, it seems to have magically resolved itself after ~1.5h). I'm just wondering what Verizon is DOING that they are screwing up their own DNS traffic. If they were capturing my queries and sending them to their own servers (I actually have Google's public facing servers at the top of the list handed out by DHCP) that would be one thing (irritating to be sure, but they aren't, so it's not), but when I'm explicitly hitting a name server down the street in Reston that VZ run and it's failing the same way? It makes me wonder.

Jamie

8

Jamie Bowden <jamie@photon.com> writes:

I don't care for the Actiontec boxes either, but the STB program
guides and other features don't work without it, so I have mine
forward all IP traffic unmolested to my own as the DMZ host

Actually this can be worked around. My config has SA, er, Cisco STBs
and a Netgear MCAB1001 MOCA to Ethernet bridge. This configuration is
very unsupported, which is why I keep a completely unmolested
Actiontec around to plug in if I have to have the guys at Verizon take
a look at it.

A little magic in dhcpd.conf:

  subnet 192.168.1.0 netmask 255.255.255.0 {
    option routers 192.168.1.1 ;
    option domain-name-servers 71.252.0.12 ;
    default-lease-time 86400 ;
    max-lease-time 172800 ;

    host stb100 { hardware ethernet 0:23:be:xx:xx:xx ; fixed-address 192.168.1.100 ; }
    host stb101 { hardware ethernet 0:21:be:xx:xx:xx ; fixed-address 192.168.1.101 ; }
    host stb102 { hardware ethernet 0:25:2e:xx:xx:xx ; fixed-address 192.168.1.102 ; }
    host stb103 { hardware ethernet 0:21:be:xx:xx:xx ; fixed-address 192.168.1.103 ; }
  }

and then some appropriate holes in the firewall (/etc/pf.conf):

# for STBs
pass in quick on $extif inet proto tcp from any to ($extif) port 35000 rdr-to 192.168.1.100 port 7547
pass in quick on $extif inet proto tcp from any to ($extif) port 35001 rdr-to 192.168.1.101 port 7547
pass in quick on $extif inet proto udp from any to ($extif) port 63145 rdr-to 192.168.1.100 port 63145

(I only have one DVR and one STB - the definitions for extra STBs came
out of the Actiontek. Not sure what I'll end up needing to do if I
get another DVR or STB in order to get them properly provisioned...)

Guide and VOD work fine. I don't feel like playing stuff from a PC on
the STBs badly enough to be willing to cram my whole life into a flat
192.168.1/24, so I give those up.

I've often wondered whether the boxes care about double-hopped NAT.
Perhaps one of these days I'll try putting the Actiontek and some new
pf.conf rules in place of the Netgear and give that a try.

(thus
the dual layer of [P|N]AT you see). It's just UDP/TCP 53 traffic
that's not flowing for whatever reason; it's every device in the
house phones, tablets, computers, you name it, so I'm not inclined
to attribute it to malware. My neighbor was also seeing it (and
like last time, it seems to have magically resolved itself after
~1.5h). I'm just wondering what Verizon is DOING that they are
screwing up their own DNS traffic. If they were capturing my
queries and sending them to their own servers (I actually have
Google's public facing servers at the top of the list handed out by
DHCP) that would be one thing (irritating to be sure, but they
aren't, so it's not), but when I'm explicitly hitting a name server
down the street in Reston that VZ run and it's failing the same way?
It makes me wonder.

No idea, just a datapoint that we're Not Seeing That Here... but if
it is failing on google's public dns servers that's troubling to say
the least.

-r