Right, so I gave up on my install completely, I have gone for a complete reinstall, I'm still not seeing consistent internet access though.
What I have done: Tried two different versions of pfsense as clean installs on esxi host, no back up from config xml, set them both up from new.
Initial version that exhibited problems 2.3.1 i386 - problems described in this thread.
New versions tried:
2.2.6 x64
2.3.2 x64
Both new (clean) versions exhibit the same gateway issues.
Known differences from my initial setup, both the 2.2.6 and 2.3.2 x64 versions seem to have DNS resolver enabled by default, I was originally using dns forwarder. I don't recall if this was something I changed originally, it would have been years ago if I did. Not sure if/how this affects my problems now.
I don't know if I should go back to an even older version to check if that would work, but from by best memory 2.2.6 was working for me in my initial setup. So this is currently leading me to think this a setup issue of my making. No surprises there…! :o
So, in the new versions I have installed I am seeing WAN Interface gets an IP no problems. WAN_DHCP gateway fluctuates between online and offline quite frequently and regularly, and hence internet access is affected inline with this.
My system log seems to point to something weird going on with WAN, namely it repeats this error over and over again.
Aug 7 19:44:50 php-fpm 29545 /rc.newwanipv6: rc.newwanipv6: Info: starting on em1.
Aug 7 19:44:50 php-fpm 29545 /rc.newwanipv6: rc.newwanipv6: No IPv6 address found for interface WAN [wan].
Aug 7 19:44:50 php-fpm 29545 /rc.newwanipv6: rc.newwanipv6: Info: starting on em1.
Aug 7 19:44:50 php-fpm 29545 /rc.newwanipv6: rc.newwanipv6: No IPv6 address found for interface WAN [wan].
Aug 7 19:44:50 php-fpm 29545 /rc.newwanipv6: rc.newwanipv6: Info: starting on em1.
Aug 7 19:44:50 php-fpm 29545 /rc.newwanipv6: rc.newwanipv6: No IPv6 address found for interface WAN [wan].
Aug 7 19:44:50 php-fpm 29545 /rc.newwanipv6: rc.newwanipv6: Info: starting on em1.
Aug 7 19:44:50 php-fpm 29545 /rc.newwanipv6: rc.newwanipv6: No IPv6 address found for interface WAN [wan].
Aug 7 19:44:50 php-fpm 29545 /rc.newwanipv6: rc.newwanipv6: Info: starting on em1.
Aug 7 19:44:50 php-fpm 29545 /rc.newwanipv6: rc.newwanipv6: No IPv6 address found for interface WAN [wan].
Aug 7 19:44:50 php-fpm 29545 /rc.newwanipv6: rc.newwanipv6: Info: starting on em1.
Aug 7 19:44:50 php-fpm 29545 /rc.newwanipv6: rc.newwanipv6: No IPv6 address found for interface WAN [wan].
Aug 7 19:44:50 php-fpm 29545 /rc.newwanipv6: rc.newwanipv6: Info: starting on em1.
Aug 7 19:44:50 php-fpm 29545 /rc.newwanipv6: rc.newwanipv6: No IPv6 address found for interface WAN [wan].
Aug 7 19:44:50 php-fpm 29545 /rc.newwanipv6: rc.newwanipv6: Info: starting on em1.
Aug 7 19:44:50 php-fpm 29545 /rc.newwanipv6: rc.newwanipv6: No IPv6 address found for interface WAN [wan].
Can anyone explain what is going on here?? Is this a problem that I should fix??
My gateway logs are taken up with dpinger alerts that show the fluctuating internet access. See below.
Aug 7 18:57:38 dpinger WAN_DHCP 176.26.X.X: Alarm latency 8826us stddev 9725us loss 21%
Aug 7 18:58:08 dpinger WAN_DHCP 176.26.X.X: Alarm latency 1178329us stddev 6722886us loss 70%
Aug 7 18:58:39 dpinger WAN_DHCP 176.26.X.X: Alarm latency 7930us stddev 2873us loss 53%
Aug 7 18:59:08 dpinger WAN_DHCP 176.26.X.X: Clear latency 7733us stddev 2029us loss 5%
Aug 7 19:00:50 dpinger WAN_DHCP 176.26.X.X: Alarm latency 7837us stddev 2265us loss 21%
Aug 7 19:01:25 dpinger WAN_DHCP 176.26.X.X: Alarm latency 1800145us stddev 8209140us loss 81%
Aug 7 19:01:57 dpinger WAN_DHCP 176.26.X.X: Alarm latency 7688us stddev 640us loss 50%
Aug 7 19:02:27 dpinger WAN_DHCP 176.26.X.X: Clear latency 20475us stddev 44552us loss 7%
Aug 7 19:13:26 dpinger WAN_DHCP 176.26.X.X: Alarm latency 9490us stddev 16525us loss 21%
Aug 7 19:15:04 dpinger WAN_DHCP 176.26.X.X: Clear latency 83613us stddev 641992us loss 14%
Aug 7 19:15:16 dpinger WAN_DHCP 176.26.X.X: Alarm latency 79299us stddev 671765us loss 21%
Aug 7 19:15:55 dpinger WAN_DHCP 176.26.X.X: Alarm latency 1556632us stddev 8193464us loss 75%
Aug 7 19:16:20 dpinger WAN_DHCP 176.26.X.X: Alarm latency 8150us stddev 2703us loss 60%
Aug 7 19:16:54 dpinger WAN_DHCP 176.26.X.X: Clear latency 7864us stddev 1783us loss 5%
Aug 7 19:18:58 dpinger WAN_DHCP 176.26.X.X: Alarm latency 7711us stddev 403us loss 22%
Aug 7 19:19:30 dpinger WAN_DHCP 176.26.X.X: Alarm latency 2216698us stddev 8411963us loss 73%
Aug 7 19:20:06 dpinger WAN_DHCP 176.26.X.X: Alarm latency 0us stddev 0us loss 100%
Aug 7 19:20:09 dpinger WAN_DHCP 176.26.X.X: Alarm latency 35555248us stddev 1018363us loss 94%
Aug 7 19:20:46 dpinger WAN_DHCP 176.26.X.X: Alarm latency 7738us stddev 641us loss 68%
Aug 7 19:21:08 dpinger WAN_DHCP 176.26.X.X: Alarm latency 1045933us stddev 6229170us loss 68%
Aug 7 19:21:40 dpinger WAN_DHCP 176.26.X.X: Alarm latency 11149us stddev 7093us loss 92%
Aug 7 19:22:00 dpinger WAN_DHCP 176.26.X.X: Alarm latency 11225886us stddev 19425589us loss 89%
Aug 7 19:22:26 dpinger WAN_DHCP 176.26.X.X: Alarm latency 7700us stddev 460us loss 59%
Aug 7 19:22:59 dpinger WAN_DHCP 176.26.X.X: Clear latency 7673us stddev 507us loss 6%
Aug 7 19:25:38 dpinger WAN_DHCP 176.26.X.X: Alarm latency 8322us stddev 2945us loss 22%
Aug 7 19:26:09 dpinger WAN_DHCP 176.26.X.X: Alarm latency 1275590us stddev 6943279us loss 73%
Aug 7 19:26:39 dpinger WAN_DHCP 176.26.X.X: Alarm latency 8116us stddev 2273us loss 51%
Aug 7 19:27:08 dpinger WAN_DHCP 176.26.X.X: Clear latency 8017us stddev 2098us loss 5%
Aug 7 19:33:26 dpinger WAN_DHCP 176.26.X.X: Alarm latency 8084us stddev 2627us loss 22%
Aug 7 19:34:49 dpinger WAN_DHCP 176.26.X.X: Clear latency 7715us stddev 681us loss 5%
That's all for this update I think, I'm happy to provide anything that would be of use to help diagnose this.
Note the above log snippets are from the 2.3.2 x64 version. I figure I might as well use the latest version that I would prefer to continue with going forwards - even if it currently isn't working!
Any suggestions or discussion welcomed! :)
rancid