Can't HTTP, HTTPS, SSH or Map drive across networks

bingo600

Sounds a bit like a DNS issue to me.

When you say you can ping the device on VL11 , is that by dns-name or by ip ?

/Bingo

rmoroni3

@bingo600 :
Everything is by IP address. I am trying to http or https from my PC on 192.168.10.2 to a VMware Host server on 192.168.11.20. You have to use http://192.168.11.20 to get to the tools like the VMware Client which in turn uses https://192.168.11.20 to actually manage it. None of this works unless my PC is on the same network as the server. I'll try to post a Wire Shark packet capture of what the traffic looks like when it won't connect.

Griffo

@rmoroni3 Wireshark will help, you can also do a packet trace on teh firewall itself on the two different interfaces, so you can see what's entering the interface and what's exiting.

You say that your devices has 5 NIC's and then mention vlans. Can you clarify which interfaces are physical and which are vlans?

I'd be double checking all your vlan tagging configs on your physical and virtual switches too.

rmoroni3

@griffo
I have two dual-port Intel PRO/1000 NICs plus the RealteK gigabit NIC built-in on the mobo. The Intel NICs are used for the WAN1, VLAN10, VLAN11 and VLAN12 networks. The Realtek NIC is used for WAN2 which is the WiFi Internet failover in case my cable Internet goes down. I can tether that NIC to my cellphone which is set up as a hotspot so I can keep working and all of my Smart devices still work.

When I first embarked on this adventure I started to use VLANs but decided not to. I never changed the network names. SO, there aren't any VLANs, just the 5 separate networks. I really should change that unless VLANs is what will solve the problem!

You mention virtual switches. I don't have any. I do have two unmanaged switches, one plugged into the 192.168.10.X network, and one plugged into the 192.168.11.X network.

I work on the packet capture tomorrow.

johnpoz

Well sniff on the destination network interface on pfsense. Then send traffic from the source network.. Do you see pfsense send on the traffic to the destination IP?

If so then its not pfsense.

You say you can ping.. Lets see your rules on the source network interface.

rmoroni3

@johnpoz
Here are the requested screenshots/packet captures. When I tried to capture the traffic on VLAN11 from my PC to the server on pfSense, it was blank so I captured the traffic from VLAN10.

From VLAN10 to Server on pfSense.png
From my PC to Server.png
VLAN11 Firewall Rules.png
VLAN10 Firewall Rules.png
Ping and Tracert from my PC to Server.png

I tried to upload the actual packet capture files but it didn't look like they uploaded properly. I got an error on both when I tried. Sorry!

rmoroni3

@johnpoz
Here are two traceroutes on pfSense:
Tracert to Server on pfSense1.png
Tracert to Server on pfSense2.png

The only difference is I turned on ICMP on the one and it completed. I find the interesting and I am sure it is probably telling me something but what??

johnpoz

When I tried to capture the traffic on VLAN11 from my PC to the server on pfSense, it was blank

When you ping? Then your not pinging what you think your pinging.

Lets see the sniff on dest vlan interface while you ping from your source network.

Do you have any rules in floating? For example shoving traffic out a specific wan gateway. As I see from your posted rules your doing something or trying to with multiple wans?

How exactly are you doing the captures? Just do packet capture on pfsense.. There is no point in sniffing on the source side to be honest.. Since your not getting any response.. But to rule out pfsense you need to see it sending traffic on your dest interface to whatever your trying to talk to in the destination network.

If your saying its blank, but you can ping - then your not pinging what your think your pinging. Or you would of seen that in the sniff (packet capture)

edit: One thing I could see for sure presenting itself exactly like what your seeing is a floating rule shoving traffic out your wan for tcp.. Or tcp/udp while your normal lan rule with no policy routing allows for the ping.

rmoroni3

@johnpoz
Here are some screenshots. Here is the sniff on the dest vlan while pinging from my PC on vlan 10:

Here is the sniff on the same setup but trying to https fo the destination:

As you can see it IS blank.

Here is the firewall Floating rule:

Now, just for chuckles, I disabled the floating rule and now everything works. Here is the sniff while connecting from my PC the VMware host server:

I am not sure I fully understand what is going on. Looking at the rule it sort of makes sense but some explanation would be a great help.

I didn't think to look at the floating rule. Now my Internet failover doesn't work. It was a pain to get it working in the first place and now it is broken. I suppose I need to open a new post for that issue, correct?

Thanks for all of your help on this. This was really frustrating me and I need this to work the way I want it to. Now I can go back and change the rules the way I want them to be.

johnpoz

See that rule there in your floating - its forcing traffic that is TCP, ie http or https out your gateway. So no your not going to get to your other vlans.

See the bypassing policy routing section here.

https://docs.netgate.com/pfsense/en/latest/multiwan/policy-route.html#bypassing-policy-routing

You need a rule that allows the traffic to your other vlans - before you shove traffic out the wan.. Which isn't going get your vlans.