Odd behavior on WAN interface



  • I am experiencing odd behavior with my pfSense box and I am not quite sure how to diagnose it.

    I have the following setup…
    Cable modem connected to a switch (unmanaged layer 2).
    psSense box connected to same switch.
    Laptop connected to same switch.

    The cable modem is configured in bridge mode and I have a block of public addresses.

    I have the WAN interface of pfSense configured with x.x.x.2 and I also have x.x.x.3, x.x.x.4, x.x.x.5, x.x.x.6 and x.x.x.7 configured as aliases.

    I have a rule on the WAN to allow ICMP.

    I can ping the .2 address from the Internet (using various websites and a computer in another state).

    However, I can't ping the .3, .4, .5 or .6 address.

    I can ping the .7 address. (although yesterday, it was .4 that was working and .7 was not)

    The Laptop has .13 assigned to it and can ping all the addresses on the pf box. It can also be pinged from the Internet.

    So it seems that traffic from the Internet destined for the 3, .4, .5 or .6 address is unable to get there.

    ICMP traffic is not the only traffic with the issue. I have been using port forwarding to host websites from these public addresses and that traffic is not passing either.

    This was all setup and working for several months and just recently stopped working.

    I have tried rebooting the modem and the pfSense box.


  • Netgate Administrator

    So you have at least 3 public IPs? How many do you have?
    What Alias type are you using?

    Steve



  • I have a /28 block.
    So 13 assignable addresses.

    the .2 address is the WAN interface address and I have .3 through .7 assigned as virtual IP/IP Alias.

    I can ping the .2 & .7 address from the outside world, but the rest are dead. (However, from the laptop they are alive.)


  • Netgate Administrator

    Well the laptop can ping them perhaps because it's in the same subnet. It certainly doesn't have to deal with any routing issues your ISP might be having. Have you tested this setup with anything else ?

    Run a packet capture on WAN to see what's actually arriving from your ISP.

    Steve



  • @coreybrett:

    Cable modem connected to a switch (unmanaged layer 2).

    psSense box connected to same switch.
    Laptop connected to same switch.
    The cable modem is configured in bridge mode and I have a block of public addresses.

    Not that I can help you, but might I ask why you put the modem in the switch, and not in the pfSense NIC? (I'm asking because I just have received my cable modem yesterday, I want to do dual WAN with my VDSL, but the cable modem seems to be configured as WAN2/DHCP, which gives it a 192.168.1.2 adress as WAN. I find this strange since I would be expecting a public IP, not a private one). Is your putting it in the switch a solution for that, or does it serve a different goal for you?

    Thank you  :D



  • @stephenw10:

    Well the laptop can ping them perhaps because it's in the same subnet. It certainly doesn't have to deal with any routing issues your ISP might be having. Have you tested this setup with anything else ?

    Run a packet capture on WAN to see what's actually arriving from your ISP.

    Steve

    Ok…

    Started the capture, pinged one of the aliases from the outside world. Stopped the capture. Nothing to show for it.

    Started the capture, pinged one of the other aliases from the outside world. Stopped the capture. Saw the ping in the capture.

    Both alias IPs are in the same block and assigned the same way in pfSense.



  • @Hollander:

    Is your putting it in the switch a solution for that, or does it serve a different goal for you?

    My cable modem only has one physical port. The switch allows me to connect more than one device.


  • Netgate Administrator

    Ok, so ping packets aren't reaching your WAN interface. Your ISP is not routing them to you. Possibly they have a congiuration issue. Possibly they see your virtual IPs as all having the same MAC address and they are not registering enough distinct devices in their arp records. I'm not really terribly familiar with this sort of issue but I seems to recall reading something similar in another thread.
    I would call them to find out why they're not routing packets to your connection with one of your registered IPs.

    @Hollander having a switch between the modem and pfSense box is a valid config here because he has multiple public IPs on the connection.

    Steve



  • I feel like the issue is on the ISPs end as well. This setup was working fine for several months and just quit one day for no apparent reason. The aliases that work change every time I reboot the pfSense box.

    The ISPs tech support has not been very helpful and keep telling me that everything is working properly on their end. They also couldn't understand why I was assigning more than one public address to my router and told me that was not something they supported. I really didn't know how to respond to that.


  • Netgate Administrator

    Hmm, well as I said this isn't really something I have a lot of experience with but if it was all working fine and then stopped it sure sounds like they changed something.
    It maybe that using a single real interface and multiple virtual IPs NATed to internal addresses (I assume) is an unusual configuration from their point of view but there are plenty of others using that here.  The only other thing you could do would be to disable NAT and just route the traffic. In that case they may be requiring some routing protocol to advertise the public IPs downstream of pfSense. If that's the case they should be able to tell you.

    Either their routers are not sending you the traffic because they're misconfigured or because they require something from your end to tell them where to send it, they should know what their own requirements are though. I would have thought.  ;)

    Steve



  • Is there a proper term for referring to the practice of assigning multiple IP addresses to the WAN interface of a router? I'm looking for the right lingo to explain my setup to the ISP.


  • Netgate Administrator

    Well I guess it depends what type of virtual IP you're using but in Unix world what you're doing here is IP aliasing. It wouldn't surprise me at all to find that other router manufacturers have a different name for it. This is not an area I have much experience in though, perhaps someone else could answer this better?  :-\

    Steve



  • @coreybrett:

    Is there a proper term for referring to the practice of assigning multiple IP addresses to the WAN interface of a router? I'm looking for the right lingo to explain my setup to the ISP.

    You mentioned that you have an allow rule on WAN for ICMP.  What is the destination address/ network you have listed in the rule?

    Your ISP does seem to be routing/ forwarding your subnet in an unusual manner.  Most will deliver in a 1 + 8 or 1 + 16 manner.
    i.e.  There is a separate /30 for WAN and all of the allocated static IPs in the block will be forwarded through that.  How you want to use them (Virtual IP/ routed) is up to you.


Log in to reply