No updates or packages - DNS OK - detailed troubleshooting attached

[nwisrlit]

Sorry to bring up a popular topic, but I've read all the other "can't update" / "can't get packages" posts I could find & my error wasn't described.

SYSTEM
"System Information" - "Version" on "Status: Dashboard" reads:
= = = = = = = = = = = = = = = = = = = = = = = =
2.0.1-RELEASE (i386)
built on Mon Dec 12 17:53:52 EST 2011
FreeBSD 8.1-RELEASE-p6

Unable to check for updates.
= = = = = = = = = = = = = = = = = = = = = = = =
No NAT. WAN & LAN ports as out-of-box. Added DMZ interface as option. Running on a normal box with 3 NICs.

PROBLEMS
Can't get updates or packages.

Going to "System: Package Manager" & clicking "Available Packages" tab takes a long time & errors with "Unable to communicate with www.pfsense.com. Please verify DNS and interface configuration, and that pfSense has functional Internet connectivity."

Going to "System : Firmware" & clicking "Auto Update" takes a while & comes up with:
= = = = = = = = = = = = = = = = = = = = = = = =
Downloading new version information…done
Unable to check for updates.
Could not contact pfSense update server http://updates.pfsense.org/FreeBSD_RELENG_8_1/i386/pfSense_RELENG_2_0/.updaters/
= = = = = = = = = = = = = = = = = = = = = = = =

Going to "System : Firmware" & clicking "Updater Settings" takes a VERY long time. "Use a URL server for firmware upgrades other than www.pfsense.org" is NOT checked, so I'm using the default. Also, I'm not getting the drop-down box that many talk about - is that applicable to my version?

TROUBLESHOOTING
For now, for testing, first rule in WAN is allow all traffic to pfSense box. First rule in LAN is allow all traffic from pfSense box to WAN.  Both rules logged, but nothing's showing up in the logs (different issue?). Those rules shouldn't be necessary, though; there are no blocking rules on LAN & last LAN rule is | * | LAN net | * | * | * | * | none | | (allow anything from the LAN out and allow response traffic back in - works correctly on all LAN clients).

Manually pointing a browser inside the LAN to the updates URL listed above shows a 404 error, so it's getting to the server, which is talking back, just not with a list of updates, but http://redmine.pfsense.org/issues/2532 says that's as it should be.

At "Diagnostics: DNS Lookup":
www.pfsense.com = www.pfsense.org.
www.pfsense.org = 69.64.6.21
updates.pfsense.org = 91.227.27.66

"Diagnostics: Traceroute" shows hopping to my ISP, then all asterisks, in both "Use ICMP" mode and not. From a client on my network, traceroute shows the pfSense box, my ISP GW, one hop up from there, then all asterisks. tcptraceroute shows the same thing but only asterisks in lines 4-9; step 10 actually gets to 69.64.6.21.

BETTER TROUBLESHOOTING
Tried one more thing while composing this - ssh to box, nslookup for pfsense addresses works as expected, but "links http://www.pfsense.org" got nowhere. Tried several other websites, could not get to any of them. Like I said, for troubleshooting I already specifically opened up all traffic back & forth to pfSense box:

WAN RULE:
Action: Pass
Disabled: NO
Interface: WAN
Protocol: any
Source: any
Dest: LAN address
Log: YES

LAN RULE:
Action: Pass
Disabled: NO
Interface: LAN
Protocol: any
Source: LAN address
Dest: any
Log: YES

Am I missing something obvious somewhere? How do I allow my box to speak to the outside world (and listen when it speaks back?)

Thanks,
Dan

[Lee Sharp]

Yes, you are missing a drop down.  Try sticking this in the link window.  http://updates.pfsense.org/_updaters  That is for a 32 bit I am running.

[stephenw10]

I agree with Lee's suggestion.
The URL that your box is checking is no longer relevant. It was from an older 2.0 beta or snapshot. Have you updated from a beta version?

Steve

[nwisrlit]

Lee - Thanks for the updated link. Unfortunately, when I copied/pasted it in, it still doesn't see updates or packages (does that link affect packages?) FWIW, the dropdown doesn't show up in chrome, firefox, or IE.

stephenw10 - No upgrades - this was a fresh install using pfSense-2.0.1-RELEASE-i386.iso, md5sum 62dc3ed11a5161b9e4f19dec94da589c.

Am I wrong in thinking that I should be able to use "links" to reach the outside world? Should updates/packages work even if "links" doesn't?

Thanks,
Dan

[jimp]

If links doesn't work, the GUI probably won't work either.

If that firmware drop-down it missing it also means it can't reach the pfSense servers as it's populated from a manifest file that gets pulled from us.

It still looks/sounds like a general connectivity issue – not a problem with pfSense -- but perhaps your WAN IPs, gateway, subnet mask, or some filtering/proxy upstream.

[nwisrlit]

jimp -

I'm not being blocked upstream or misconfigured in basic networking because from any LAN client I can navigate to http://updates.pfsense.org/_updaters/ and get the 404, or get the manifest from http://updates.pfsense.org/manifest.

I'm not saying it's a bug/problem with pfSense - I'm assuming I made a misconfig somewhere & I'd like some help tracking it down.

More troubleshooting:
Used "links" to connect to webservers on my LAN & DMZ = works

Created alias "TESTpfSenseALL" containing WAN, LAN, & DMZ addresses of the pfSense box.

Changed first rule in LAN to | * | TESTpfSenseALL | * | * | * | * | none |  |
Changed first rule in WAN to | * | * | * | TESTpfSenseALL | * | * | none |  |

Still cannot update, see packages, or get to ANY outside websites via "links".

[jimp]

Check "netstat -rn"

Especially the default gateway. The default gateway should be your WAN gateway.

[nwisrlit]

Default gateway shows the number given to me by the provider as the next hop "up".

Traceroute from pfSense to any given outside address shows an immediate hop to the GW, but then all asterisks.

Traceroute from any client on the LAN to any outside address shows a hit to pfSense, then a hit to the GW, then one hop beyond that, then asterisks (usually). Client comms all work, so that gateway is valid.

I did "pfctl -d" as root, and links & updates (didn't check packages) didn't work, so I think my rules are OK.

Wireshark on a hub between defaultGW and pfSense showed no activity whatsoever for pfSense box's LAN address or WAN address while I tried to use "links http://<anything>.com", updates, and packages.

This install is about 6 months old, with 70 machines behind it, including websites accessible from the outside world. Everything works great except upgrades/packages & network ops from the command line to the outside world. Every endpoint BUT the pfSense box itself routes fine.

Thanks,
Dan</anything>

[jimp]

Do your internal users get NAT applied outbound to a different IP than your WAN IP?

[nwisrlit]

No NAT.

More troubleshooting:
Another tenant in our bldg lets us "borrow" some of their bandwidth for testing etc. - the box has one of our LAN addresses & NATs out to one of theirs (so multiple users in our net have access via them).

In "System: Gateways" I created a new GW pointing to our side of the "go-between" box. In "System: Static Routes", I set both 69.64.6.21/32 (www.pfsense.org) & 91.227.27.66/32 (updates.pfsense.org) to point to the new GW. Both updates & packages worked.

Just for kicks, I manually set those two IPs to route through WANGW. No go, of course.

Leaving the GW pointed to the co-tennant, a traceroute shows that my LAN clients also route through their internet connection.

So AFAICT, something makes my pfSense box different from all my other boxes. I don't know where that something is, but it's not in the firewall rules, since "pfctl -d" didn't help.

Thanks,
Dan

[jimp]

Actually you just managed to prove it was, in fact, your ISP on WAN (or your WAN settings).

(Unless you have some really broken outbound NAT rules…)

[nwisrlit]

ROOT PROBLEM FOUND

Duh.

Uplink is point-to-point with a pair of RFC1918 (private) addresses; wireshark confirms that attempts to reach the outside world source from there. Sorry to have wasted your time.

I don't suppose there's a reasonable way on a non-NAT box to force the traffic headed to the wider world to go "thru" the firewall & end up with a routeable outside IP, is there?

Thanks,
Dan

[nwisrlit]

FAIL on my part:

I had done a wireshark, checking for LAN & WAN addresses (see post #8). I used a capture filter and TYPO'd the IP for the WAN (got the LAN right, but that didn't matter). It was only by one digit…  :(

Would have been good to know 2 days ago.