Not able to connect to some website

luckman212

@jeff_wuyo Could be a DNS/unbound problem. What are your outputs from the commands below when running directly on your pfSense console?

dig +short -t aaaa google.com
dig +short -t aaaa archlinux.org

Jeff_WuYo

@luckman212
Thanks for your reply.
Here is the output.

[2.6.0-RELEASE][root@pfSense.jeff.arpa]/root: dig +short -t aaaa google.com
2404:6800:4012:4::200e
[2.6.0-RELEASE][root@pfSense.jeff.arpa]/root: dig +short -t aaaa archlinux.com
[2.6.0-RELEASE][root@pfSense.jeff.arpa]/root:

So there does something wrong, but how to fix it tho? Any hint would be great.

luckman212

@jeff_wuyo Hmm it seems you're mixing up the hostnames -- you started by saying archlinux.org but then switched to archlinux.com

The .org seems to have AAAA records in DNS, while the .com does not. So it seems normal that you wouldn't be able to connect to archlinux.com via IPv6....

https://mxtoolbox.com/SuperTool.aspx?action=aaaa%3aarchlinux.com&run=toolpage

Jeff_WuYo

@luckman212
Oops, I mean "archlinux.org", not "archlinux.com". I used the wrong hostname on previous comment. Here are new output.

[2.6.0-RELEASE][root@pfSense.jeff.arpa]/root: dig +short -t aaaa google.com
2404:6800:4012:3::200e
[2.6.0-RELEASE][root@pfSense.jeff.arpa]/root: dig +short -t aaaa archlinux.org
2a01:4f9:c010:6b1f::1

For now, I have no clue.

luckman212

@jeff_wuyo ok, so DNS is working. And you have a 10/10 on test-ipv6.com? Might be time to do some digging with tcpdump -ni igb0 ip6 etc on pfSense or Wireshark and see where those packets are getting diverted.

Jeff_WuYo

@luckman212

I captured locally on MyPC, I'm not sure what to look for, here is a simple screenshot.
I filter out using [2a01:4f9:c010:6b1f::1], which is the IPv6 of archlinux.org I get from pinging.
On the left is connecting via pfSense as router, when MyPC try to connect to archlinux.org, it uses TLSv1, and there's no respond from server; on the right is connecting via my ISP router, when MyPC try to connect to archlinux.org, it uses TLSv1.3, and the server respond with same protocol.
Maybe wrong protocol is the reason, but why it behaves like this? During testing under pfSense, I still see TLSv1.3 being used.

luckman212

That is very odd. I just tested it on my end (I don't have IPv6 where I am currently, but via IPv4 it definitely connects via TLSv1.3 to archlinux.org). I am not aware of what pfSense being "in the middle" of this connection could possibly have to do with your browser's choice of TLS level, or the way the server "sees" your connection capabilities, unless there was some MITM going on. But I assume you are not running any kind of proxy (snort, haproxy etc) that could be tampering with these packets, right? Maybe someone else has some idea...

johnpoz

if you look at the ssl test for archlinux.org - they do not support 1.0 or 1.1 so no if your client isn't able to do atleast 1.2 then your not going to connect

https://www.ssllabs.com/ssltest/analyze.html?d=archlinux.org&s=2a01%3a4f9%3ac010%3a6b1f%3a0%3a0%3a0%3a1&latest

Unless you were running some sort of mitm setup pfsense would do nothing with your tls connection, what your client is capable of would come from the client.. You can look in the handshake to see what your client offers, etc.. In your sniff.

Here is an example handshake.. and what my client is saying it supports

You can read up on how the handshake works here

https://www.cloudflare.com/learning/ssl/what-happens-in-a-tls-handshake/

But again pfsense out of the box would have no way to mess with that handshake..

I would go here - what does it show for what your browser supports?

https://clienttest.ssllabs.com:8443/ssltest/viewMyClient.html

Jeff_WuYo

@johnpoz
Here is a screenshot about my browser. Seams all good.

Below is a screenshot about what TSL version MyPC support. (From the same sniffs above.) Both with and without pfSense, MyPC reports it support TSLv1.2 and 1.3 in both.

This results are consistent on my every device, windows, android, etc.

As for MITM, I can't not recall I did anyting like it, or I just don't realize what I did. My pfSense is quite vanilla, no other addon package.

Some weird thing I did is that I bridged LAN, OPT2, OPT3 together (as bridge0) but not assign bridge0, instead I assign each interface. I setup 3 DHCP server on LAN, OPT2, OPT3, tracking WAN on LAN, set RA assisted, no IPv6 configuration on OPT2 and OPT3. (MyPC is under LAN.) I can't get more interface to track WAN since my ISP gives me /64 prefix. Under this configuration, I can get IPv6 and IPv6 connectivity form every port which is kinda weird.
If I assign bridge0, and configure bridge0 IPv6 to track WAN interface, IPv6 just doesn't work, no ip, no connection. I suspect is due to some limitation or lack of proper switch chip.

johnpoz

@jeff_wuyo said in Not able to connect to some website:

I setup 3 DHCP server on LAN

huh? But you bridged them?

my ISP gives me /64 prefix

Yeah you got some weirdness setup there for sure.

If your wanting to use IPv6 behind pfsense, but your ISP does not do delegation of a prefix to use behind a router. Prob better to just get a IPv6 tunnel from hurricane electric. Here you can get a /48 to use how you wish. On your different segments behind pfsense.

I am not exactly clear what your doing with your bridging - but can tell you for sure if what you want is switch, then get a switch vs doing anything with a bridge.

nonick

@jeff_wuyo WAN interface MTU + MSS entered correctly?
e.g.

Bob.Dig

@nonick said in Not able to connect to some website:

@jeff_wuyo WAN interface MTU + MSS entered correctly?

I think you shouldn't reduce the value of MSS, it is done by pfSense, see the explanation under it.

nonick

@bob-dig said in Not able to connect to some website:

I think you shouldn't reduce the value of MSS, it is done by pfSense, see the explanation under it.

Unfortunately not, it's a bug in pfSense.

johnpoz

@nonick said in Not able to connect to some website:

Unfortunately not, it's a bug in pfSense.

And where is the redmine to that? I sure do not need to edit my mtu or mss values? Are you on some sort of say PPPoE connection or something where standard mtu does not work?

Unless your on some isp connection that requires something lower, there should be no need to edit those.

Bob.Dig

@nonick said in Not able to connect to some website:

Unfortunately not, it's a bug in pfSense.

Maybe it is fixed? That's why I think it is.

Jeff_WuYo

@johnpoz said in Not able to connect to some website:

but can tell you for sure if what you want is switch

Yes, what I want is indeed switch. I also want to have different subnet, or rather different network to manage other device. Perhaps I should get a managed switch.

huh? But you bridged them?

Yeah, it's the way I figure out how to make all port have IPv6 connectivity. The way I do it is against my knowledge about how router and switch work, it just looks wrong. The machine which pfSense runs on has 4 ports, thus I want to make it works as router/switch. But if I bridged them and assign bridge0, only IPv4 worked, no IPv6 connectivity. Assigning bridge to me is more correct way to do it, but I'll lose IPv6.

@nonick said in Not able to connect to some website:

WAN interface MTU + MSS entered correctly?

No. I leave it blank, which would be default. I'm not fully understand MTU and MSS, thus I ignore it for now.

johnpoz

@jeff_wuyo said in Not able to connect to some website:

Perhaps I should get a managed switch.

Doesn't have to be a full managed switch, you can get a 8 port gig smart switch (does vlans) and some other limited features of a fully managed switch. For very reasonable price, there are many on the market in the $40 price range.

But yes if your goal is to segment your network - a vlan capable switch is going to be be helpful ;) Next would be a access point that can also do vlans. If you want your wireless clients to be on different networks as well.

While a bridge does have use cases, trying to turn interfaces into switch ports is not really a good use for a bridge. You can fry up a hamburger patty and "call" it steak - but its not a very "good" steak ;)

Jeff_WuYo

@johnpoz said in Not able to connect to some website:

trying to turn interfaces into switch ports is not really a good use for a bridge

That's true. I'm asking pfSense for too much lol.

The topic is altered.
Is the bridge-thingy I did causing my TLS connection to change? Hard to tell I think. I might reset pfSense and test again eventually. For now, I think I can still tinkering around.

johnpoz

@jeff_wuyo said in Not able to connect to some website:

I'm asking pfSense for too much lol

Not so much that - but bridging is not switching. While it may mimic a switch in some aspects. Its not really switching..

There are some valid use cases where sure bridging is the solution - but wanting switch ports is not one of them ;)

Jeff_WuYo

@bob-dig
I set my MTU 6000 on both WAN and LAN. The reason I chose big MTU is I always see some lost fragment package, and according to how IPv6 handle package exceed MTU, I assume that the package is large. I could be totally wrong, if so, please do correct me.
Beside the adjustment above, I didn't bridge any interface together this time.
For the sake of convenience, I just prepare the data I sniffed. Please use wireshark for more detail.

File1
I was able to connect to archlinux, the handshake was successful, but not able load the site properly.
In this case, IPv6 for archlinux.org is [2a01:4f9:c010:6b1f::1]
File2 is too large to upload, I put it at google drive.
In this example, I connect to multiple site, youtube.com [2404:6800:4012:2::200e], archlinux.org [2a01:4f9:c010:6b1f::1], and ipv6-test.com [multiple IPv6]. Only archlinux.org could not establish connection. Occasionally seeing pfSense complain to MyPC that the package is too large.

It seems like I always losing package from archlinux.org, as least that's what I saw on wireshark. Maybe wireshark is interpret the information wrong. Due to the lack of knowledge, I can't tell anymore form the data. I hope someone can point out a thing or two.