Is the snapshots.pfsense.org site blocking my ISP?

Luzemario

Hi guys,

I cannot reach snapshots.pfsense.org from my house from last three days. Below is a trace from my pfSense box:


traceroute to snapshots.pfsense.org (66.111.2.168), 64 hops max, 52 byte packets
 1  10.26.128.1 (10.26.128.1)  8.275 ms  8.011 ms  7.498 ms
 2  bd060001.virtua.com.br (189.6.0.1)  12.185 ms  10.058 ms  9.281 ms
 3  200.246.210.157 (200.246.210.157)  28.841 ms
    embratel-G0-5-3-7-tacc01.rjo.embratel.net.br (200.167.43.13)  33.843 ms
    embratel-T0-6-5-0-tacc01.rjoen.embratel.net.br (201.73.51.69)  26.023 ms
 4  ebt-T0-2-0-5-tcore01.spo.embratel.net.br (200.230.158.222)  146.557 ms
    ebt-T0-1-0-0-tcore01.rjo.embratel.net.br (200.230.252.250)  157.272 ms
    ebt-T0-4-0-2-tcore01.spoph.embratel.net.br (200.230.158.166)  146.225 ms
 5  ebt-Bundle-POS1111-intl01.nyk.embratel.net.br (200.230.220.46)  153.027 ms  148.751 ms
    ebt-Bundle-POS1211-intl01.nyk.embratel.net.br (200.230.220.42)  145.891 ms
 6  ae59.edge2.NewYork1.Level3.net (4.71.230.241)  215.554 ms  165.707 ms  160.200 ms
 7  vlan70.csw2.NewYork1.Level3.net (4.69.155.126)  155.671 ms
    vlan60.csw1.NewYork1.Level3.net (4.69.155.62)  153.429 ms
    vlan90.csw4.NewYork1.Level3.net (4.69.155.254)  160.497 ms
 8  ae-61-61.ebr1.NewYork1.Level3.net (4.69.134.65)  163.937 ms  151.827 ms
    ae-81-81.ebr1.NewYork1.Level3.net (4.69.134.73)  152.078 ms
 9  ae-2-2.ebr1.Newark1.Level3.net (4.69.132.98)  165.463 ms  155.549 ms  161.218 ms
10  ae-1-51.edge2.Newark1.Level3.net (4.69.156.9)  163.531 ms  150.001 ms  158.920 ms
11  THE-NEW-YOR.edge2.Newark1.Level3.net (4.30.130.234)  154.265 ms  159.074 ms  162.774 ms
12  cs20.cs59.v.jfk.nyinternet.net (64.147.125.126)  160.094 ms  166.031 ms  175.918 ms
13  * * *

But I can reach snapshots.pfsense.org from other networks:


traceroute to snapshots.pfsense.org (66.111.2.168), 30 hops max, 60 byte packets
 1  201-23-189-97.gprs.claro.net.br (201.23.189.97)  137.501 ms  167.515 ms  167.478 ms
 2  10.187.182.9 (10.187.182.9)  167.375 ms  177.014 ms  186.921 ms
 3  10.180.56.106 (10.180.56.106)  256.941 ms  256.924 ms  256.817 ms
 4  10.129.56.45 (10.129.56.45)  276.626 ms  276.605 ms  276.508 ms
 5  10.108.56.249 (10.108.56.249)  286.324 ms  286.260 ms  286.157 ms
 6  10.119.99.1 (10.119.99.1)  296.674 ms  188.883 ms  208.824 ms
 7  10.119.99.2 (10.119.99.2)  228.448 ms  229.592 ms  239.521 ms
 8  embratel-T0-0-0-0-tacc01.spo.embratel.net.br (189.86.58.5)  249.292 ms  209.531 ms  229.381 ms
 9  ebt-T0-9-0-10-tcore01.spo.embratel.net.br (200.230.252.94)  359.980 ms  330.791 ms  339.610 ms
10  ebt-Bundle-POS1111-intl01.nyk.embratel.net.br (200.230.220.46)  330.582 ms  319.770 ms  359.733 ms
11  ae59.edge2.NewYork1.Level3.net (4.71.230.241)  389.444 ms  349.905 ms  359.594 ms
12  vlan80.csw3.NewYork1.Level3.net (4.69.155.190)  349.734 ms vlan70.csw2.NewYork1.Level3.net (4.69.155.126)  330.430 ms vlan80.csw3.NewYork1.Level3.net (4.69.155.190)  359.719 ms
13  ae-81-81.ebr1.NewYork1.Level3.net (4.69.134.73)  359.723 ms ae-61-61.ebr1.NewYork1.Level3.net (4.69.134.65)  340.401 ms ae-91-91.ebr1.NewYork1.Level3.net (4.69.134.77)  359.601 ms
14  ae-2-2.ebr1.Newark1.Level3.net (4.69.132.98)  379.472 ms  330.401 ms  359.616 ms
15  ae-1-51.edge2.Newark1.Level3.net (4.69.156.9)  359.708 ms  330.266 ms  340.528 ms
16  THE-NEW-YOR.edge2.Newark1.Level3.net (4.30.130.234)  349.460 ms  349.643 ms  369.461 ms
17  cs20.cs59.v.jfk.nyinternet.net (64.147.125.126)  379.355 ms  339.776 ms  389.807 ms
18  66.111.2.168.static.nyinternet.net (66.111.2.168)  359.128 ms  359.675 ms  359.655 ms
19  66.111.2.168.static.nyinternet.net (66.111.2.168)  359.876 ms  348.137 ms  367.905 ms

So, Is the snapshots server blocking me or there are some routing error?

jimp

That server does block using a different bogons list that lists a bunch of unallocated networks. It's possible your subnet is still listed as unallocated.

What is the actual source IP of the request?

Luzemario

Very important explanation, jimp. This also can be the response to some troubles with my pfSense box too. The source IP is 179.214.109.248.

IPv4 blocks here are exhausting so it can be the point.

Here is the response from local registrar, registro.br (https://registro.br/cgi-bin/whois):


% Copyright (c) Nic.br
%  A utilização dos dados abaixo é permitida somente conforme
%  descrito no Termo de Uso (http://registro.br/termo), sendo
%  proibida a sua distribuição, comercialização ou reprodução,
%  em particular para fins publicitários ou propósitos
%  similares.
%  2013-08-12 12:42:39 (BRT -03:00)

inetnum:       179.212/14
asn:           AS28573
ID abusos:     GRSVI
entidade:      NET Serviços de Comunicação S.A.
documento:     000.108.786/0001-65
responsável:   Grupo de Segurança da Informação Vírtua
país:          BR
ID entidade:   GRSVI
ID técnico:    GRSVI
inetrev:       179.214.0/17
servidor DNS:  ns7.virtua.com.br 
status DNS:    11/08/2013 AA
último AA:     11/08/2013
servidor DNS:  ns8.virtua.com.br 
status DNS:    11/08/2013 AA
último AA:     11/08/2013
criado:        14/03/2013
alterado:      14/03/2013

ID:            GRSVI
nome:          Grupo de Segurança Vírtua
e-mail:        virtua@virtua.com.br
criado:        12/05/2008
alterado:      18/05/2009

% Problemas de segurança e spam também devem ser reportados ao
% cert.br, http://cert.br/, respectivamente para cert@cert.br
% e mail-abuse@cert.br
%
% whois.registro.br aceita somente consultas diretas. Tipos de
% consultas são: dominio (.br), ticket, provedor, ID, bloco
% CIDR, IP e ASN.

Can do I do something to avoid this?

jimp

I didn't see anything close to that IP address in the bogons list on there. So it may not be that after all. I didn't see anything in the firewall log there either but it's also a busy firewall and it may have scrolled out of the log already.

Luzemario

I'm pinging it right now with no response… can you look again?

I'll let it pinging for 24 hours. If you find (or not) please let me know so I can stop the pings.

jimp

That firewall blocks ping so that doesn't help.

Try making a few HTTP connections to the snapshots server, see what happens.

Luzemario

I am making some traceroutes using TCP, as in


traceroute -P tcp snapshots.pfsense.org

and triyng to connect via http too.

Luzemario

jimp,

I can access the site now. But I see a list of files instead of index.html.

jimp

Traceroute will not work properly no matter what, TCP or UDP. The firewall only lets tcp/80 through there.

The only proper test is http on port 80.

Depending on the way you accessed the site, a list of files may be normal.

Luzemario

Cool!!

I am outside house now, so I am using a SSH tunnel to reach snapshots.pfsense.org on port 80 from home. I will test better when at home, but since I can open port 80, all will be fine now.

Thank you very much, sorry for the extra work… ;)

st4fun

in General Setup

Do not use the DNS Forwarder as a DNS server for the firewall

Luzemario

@st4fun:

Do not use the DNS Forwarder as a DNS server for the firewall

Why not?

ggzengel

My pfsense shows:
Downloading new version information…done
Unable to check for updates.
Could not contact custom update server.

I can telnet port 80 and it gets open.

It's not good to block ping.

ggzengel

After package capture and finding nothing I understand.
The updater tries to update with ipv6, but ipv6 is disabled.
After making an entry in DNS Forwarder (snapshots.pfsense.org=66.111.2.168) the updater works.

Why will this work if not using DNS forwarder?
It's more a bug of the updater, because the other 6+ pfsenses work. It's only if there an ipv6 router on WAN and ipv6 is disabled in pfsense.

jimp

Your IPv6 must not be fully disabled. Usually it wouldn't attempt that unless you have an IPv6 default route/gateway or a GUA IPv6 address configured somewhere.

ggzengel

I never did something with ipv6 on this pfsense. From first minute on I disabled ipv6 and it worked perfectly.
The only thing which changed is that my ISP connected ipv6 to my vlan.

Know we know why it doesn't work. What to do that it will work again?
In my opinion the updater has to fall back to ipv4.

eri--

Probably report as an issue in redmine.pfsense.org to have it not forgotten.
Probably during 2.2. roadmap it will be solved

ggzengel

Issue #3152 created.