Multi-WAN vs Whatever this is...
-
@johnpoz First and foremost, thank you for taking your time to not only explain but show me with examples from your own setup. While I'm still unclear , you've helped already in my understanding of the broader picture regarding routing & ISP innerworkings.
Here's what I am still not understanding.
This scan was run on pfSense. pfSense WAN is connected via 192.168.0.254. I don't understand how it is reaching these hosts, that's why I said it was like multi-wan. All other traffic flows through 192.168.0.254. How could there be another route? Using what?
-
Sorry to go off topic, but the last time I went to install zenmap, I found it was tied to a deprecated version of python. Did someone finally decide to fix this, or did you have to load python libraries from many years ago?
-
@dotdash I'm using Windows here. I did have to go through that process on my linux hosts. Don't think it's fixed yet.
-
@boethius via your rings those are multiple hops away..
Why don't you look at the traceroute for those specific IPs
So your ran the command on pfsense - how exactly are you getting the gui picture when you ran it on pfsense?
What are the interfaces on pfsense? If your using IP pass through as you mention then pfsense should have a public IP on its wan..
Per your routing table it does
default 23-118-48-1.lights UGS igb0So your topology there with showing 192.168.0.254 makes less sense.. In what scenario would pfsense send traffic to that IP, its not in your routing table. How is it doing the IP passthru exactly.. What would make sense is its routing the traffic and its answering you via its loopback address.
What I would suggest vis trying to do discovery for the internet and trying to understand how nmaps topology map works, and can be confusing when things respond with different IPs (we went over this already).. Why not do a simple just traceroute.. to single IP on the internet if your trying to understand the path of traffic.
From your routing table you posted.. Pfsense will send ALL traffic that is not directly attached or has another router for to that IP.. Be it that IP answer back via a traceroute with that same IP we went over. Be it that IP some VIP on a HA setup in your isp makes no matter.
I am having a hard time understanding what your looking to understand. Pfsense is going to send ALL traffic to that IP - period, there are no other routes listed in your routing table. All that topology stuff is doing is confusing you because of how traceroute works, and icmp info sent back when a ttl expires.. That IP that icmp info is sent back from doesn't have to be the IP you show as your gateway..
edit
I'm using Windows here.
Then how are you running it on pfsense like you said??
This scan was run on pfSense
-
@johnpoz said in Multi-WAN vs Whatever this is...:
So your ran the command on pfsense - how exactly are you getting the gui picture when you ran it on pfsense?
I used
nmap -oA <filename>which will output, among others, an .xml which can then be opened in zenmap and viewed.What are the interfaces on pfsense? If your using IP pass through as you mention then pfsense should have a public IP on its wan..
I have interfaces in reply to Patch, 3rd message of thread.
So your topology there with showing 192.168.0.254 makes less sense.. In what scenario would pfsense send traffic to that IP, its not in your routing table. How is it doing the IP passthru exactly.. What would make sense is its routing the traffic and its answering you via its loopback address.
I don't know how IP passthrough works exactly. It's weird. Here's a pcap of arps on WAN.
I am having a hard time understanding what your looking to understand. Pfsense is going to send ALL traffic to that IP - period, there are no other routes listed in your routing table. All that topology stuff is doing is confusing you because of how traceroute works, and icmp info sent back when a ttl expires.. That IP that icmp info is sent back from doesn't have to be the IP you show as your gateway..
When I switched from comcast to AT&T, it took 3 days before it worked. It was a simple config, I didn't change anything from the first attempt, just poured over stuff trying to figure out what in the heck was happening, as I wasn't able to reach the internet. Just this 23.118.48.0/23 block. Overnight, it magically connected to the internet. I made no changes, nothing explains it.
-
@boethius said in Multi-WAN vs Whatever this is...:
t, it magically connected to the internet. I made no changes, nothing explains it.
Your isp finally turned it on, you finally got a dhcp address.. Nothing magic about it - why did you not call your isp about it on day 1? I would of been calling them first hour, etc.
So yeah your connected to that network on L2.. Still not getting what your trying to glean from running a discovery on the internet or your wan network, etc.
All traffic will be sent to your gateway IP - that is how it works.. what is confusing you is how traceroute works, and how the answers can be interpreted by not too bright software trying to draw a map based on icmp responses.
-
In comparing nmaps's traceroute with TCP on port 80, to regular traceroute utility on pfSense, well, they give different results.
-
@johnpoz said in Multi-WAN vs Whatever this is...:
All traffic will be sent to your gateway IP - that is how it works.. what is confusing you is how traceroute works, and how the answers can be interpreted by not too bright software trying to draw a map based on icmp responses.
I think nmap is very bright software.. the map makes sense..
I think nmap does traceroute in reverse essentially, so it will find how many hops away the target is, then it will decrement TTL. If our path to some hosts has 192.168.0.254 as the first hop, and it responds to ICMP time exceeded, and we get no response for the TTL 1 packet, then it was a different path.
And this is it as far as other hosts on the internet that I could connect to for those few days. Weird, right?
-
@boethius not that software doesn't do what it designed to do.. My point is interpretation of what it draws when the info is not what you would expect.
In a perfect world when I traceroute through a router at IP address 1.2.3.4 that is where the response would come from... But when it comes from 1.2.4.5 how do you graph that were it actually makes sense.. Since your gateway is 1.2.3.4
As to getting back 192.168.0.254, again that is how traceroute works.. That your isp gateway responds back with that IP is odd and exactly what I am talking about for where something like nmap has a hard time drawing that on a chart.
-
I just discovered this tool -- record route on ping. Look at these results... (This is a good example of how it works) https://flylib.com/books/en/3.223.1.87/1/)
So my traffic is actually going OUT via 192.168.254.254.
Coming back in on 192.168.0.254.And I have no idea what this 192.168.20 stuff is.
-
@boethius How many times do we have to go over how this works... That works the same freaking way - it only knows what devices respond from..
Why are you pinging 23.118.48.241 ???
What part are you not understanding about loopback answers and other interfaces on routers?
So in your RECORD_ROUTE test to 23.118.48.241 its a hop in its own path? And 192.168.20.11 is used twice? ;)
so is 107.208.8.1 ;)
Your confusing yourself with info that doesn't matter - again these tools don't always work how you think they do ;)
-
How many times do we have to go over how this works... That works the same freaking way - it only knows what devices respond from..
It doesn't work the exact same way, obviously, as it produces different results. It shows what interfaces are used going outbound toward gateway. I only saw the outbound interfaces/what the devices respond from TOWARD me. Now I see that 192.168.254.254 is used for traffic going toward the gateway.
Now I understand why I wasn't seeing the gateway in the hops to anywhere on the internet. Traffic was being routed to the gateway, but the traceroute responses were from another interface. So don't think I am confusing myself pointlessly, I am feeling much more clear on routing / interfaces / etc. And I appreciate your help. I can be slow sometimes but I do try to have a full picture of what's going on.
and I am wondering what to make of that result to 23.118.48.241.
-
@boethius sorry but it doesn't work that way either.
Sniff on your traffic, what is the destination of the traffic mac address. This is obtained by arping for the mac. So if your gateway is 23.118.48.1 then you have to be able to arp for it.
If there was a router at 192.168.254.254 in your path - you wouldn't be able to arp for 23.118.48.1
A device when the IP is not local, arps for its gateway, or the gateway in its routing table for that network. Then sends the traffic to that mac address with the destination IP address.
Maybe your isp device is doing proxy arp for for every IP?
I would suggest you sniff while you do you traceroute - what is the mac address of where your sending and what is the destination. What is the mac address when you ping other IPs on your same wan network..
Also notice the ttl on your pings their with -R when you ping your actual gateway IP the 48.1 you have a ttl of 63, so this is only 1 hop away. But your showing 3 IPs their in your route path.. when you ping 48.241 your getting back ttl of 61..
I am afraid you are going to draw some odd conclusion on how routing works.. When your isp could be doing odd things with bridges and proxy arp, etc. etc..
Routing is pretty simple. If IP is not on my network, send it to gateway be that default or one in my routing table. How do I do that, arp for that IP. Send to that mac..
Now what your isp might be doing in its network to support its customers spread over a wide area and wanting to put on the same L3 network with bridges and proxy arp and extended L2 for an area.. And what you might get back with a trace from loopback addresses and vips, etc. in such a network.
But routing is basic.. What is the IP of my gateway, ok arp for that - send to that mac with my destination IP. Just not understanding what your trying to figure out other than the isp network is prob doing all kinds of odd stuff. They don't have just your device via a cable to a switch to their single router with 1 IP on it.
-
@boethius said in Multi-WAN vs Whatever this is...:
So your topology there with showing 192.168.0.254 makes less sense.. In what scenario would pfsense send traffic to that IP, its not in your routing table. How is it doing the IP passthru exactly.. What would make sense is its routing the traffic and its answering you via its loopback address.
I don't know how IP passthrough works exactly. It's weird. Here's a pcap of arps on WAN.
@johnpozPCEngine is my pfsense box. HUMAX is the AT&T BGW.
If I use nmap with ARP (default) for discovery and traceroute to my /23 public subnet, it fills with every address in that space showing online, 1 hop away. Because the BGW is answering for them all. In fact, I believe I could set my gateway to any address there, say, 23.118.48.69, or 23.118.49.255 even, I bet it would work all the same.
In my first post in creating this thread, I said
First off, the "upstream gateway" that AT&T specifies is never the hop after 192.168.0.254. I am wondering if that is because with IP Passthrough, the bgw320-500 essentially adopts that address routing
It didn't seem like you were quite oriented when you started to help me out, maybe overlooking the oddities of this IP Passthrough thing, but you've also overlooked insight I've had. You asked twice about how it was pfSense shown in the GUI screenshot. I thought maybe you were doubting me entirely, that I was so clueless as to think I was running those commands from pfSense when I was on a windows host. While I do make mistakes, I'm more able than that.
Indeed routing is fairly simple. This isn't that, as I've been saying. I am trying to understand my network / ISP's network and routing of my traffic. Beyond a simple curiosity, other reasons I am interested I've noted already. My network has behaved (very) strangely in the past. A previous discussion on here with another knowledgeable member of the administration had me finding more questions and few answers in the course of investigating. https://forums.netgate.com/topic/168975/permission-errors-running-commands-as-admin/14 I was also seeing these errors while troubleshooting this, as shown here
It seems that the command worked... so I don't know what's causing this to happen. Is it possible for a permission denied error to be seen in pfSense due to some form of rejection that occurs with the BGW? I wouldn't think so. -
I tried to edit the message to have your
@johnpozreply outside of the quote text region and was seeing "Post content flagged as spam by askimet.com"
