Any Idea



  • Sorry if this has been discussed before.  If you can link me to an article on this problem that would help a lot.

    I'm having some really odd things going on.  Last night I made my M0n0wall router into a PFSense router because I want to do Load Balancing with my Static DSL and my DS3 (T1's).  But I have not set that up yet, I just wanted to get PFSense setup like M0n0wall was, with a little extras.  The only difference between the M0n0wall setup that I had and the PFsense is I moved another network onto the PFSense…..

    In my office we have 3 networks....

    192.168.2.0---->  Goes to our workstations and is set up for DHCP
    192.168.1.0--->  All servers on this are set with static IP's and typically will not have a gateway setup on the NIC for this IP.  No DHCP.
    204.180.228.0-->  This is what connects the PFsense to the INTERNET, out the DS3.  This network comes from a CISCO router that is plugged into the DS3.  All the 192.168.2.0 network uses this interface for INTERNET.

    We also have a Domain server that is on the 192.168.1.0 network.

    My Problem is that the 192.168.2.0 and the 192.168.1.0 networks can not ping each other.  I have it set on both interfaces firewall to allow access to each others subnets.

    Now when I had M0n0wall setup the 192.168.2.0 network could ping anything on the 192.168.1.0 network.  But anything on the 192.168.1.0 network could not ping anything on the 2.0 network.  But with the M0n0wall I did not have 3 NICs in it.  The 192.168.1.0 network was being managed by a linksys router that has dual wans that totally sucks.  I hate that linksys router.... anyway...

    Now with the PFsense I have 4 NICs in it, 192.168.2.3(office Network), 204.180.228.250(To CISCO that has the DS3), 192.168.1.1(1 Server Network), and DSL 75...217.  I'm currently not using the DSL at all, that interface is disabled.

    Now the really odd thing is..... the 2.0 network can ping about 5 out of the about 20 servers on the 1.0 network.  When I had M0n0wall setup, the 2.0 network could ping everything on the 1.0 network, but the 1.0 network could not ping the 2.3 network.  Remember at this time the 1.0 network was managed by a Linksys router.

    I figured since the NIc's for both networks are in the same router now with the PFsense setup, the router would just know hey that packet needs to go to this NIC here.  But its not doing that.  And why its only pinging like a quarter of the servers baffles me as well.

    If you can point me in the right direction I would appreciate soooo much!

    Thank you all for spending time reading this!  I hope you can follow all that gibberish about my network.......



  • @neemers:

    192.168.1.0–->  All servers on this are set with static IP's and typically will not have a gateway setup on the NIC for this IP.  No DHCP.

    ...

    My Problem is that the 192.168.2.0 and the 192.168.1.0 networks can not ping each other.  I have it set on both interfaces firewall to allow access to each others subnets.

    The Servers at the 192.168.1.0 subnet need a gateway to fond the way back to the 192.168.2.0 network. Either configure the pfSense as default gateway for these machines or add a static route for the 192.168.0.2 subnet to the servers. I would set a default gateway and restrict the access on the firewall level if needed. This is simply a routing issue due to a missing gateway on the servers.



  • Ok well I used ethereal on one of the servers that has a 204.180.228.* and 192.168.1.* address and set it to capture.  Then from the 192.168.2.3 network I pinged the server and ethereal showed the ping request come to the server but showed no reply.  So they are getting to the server.  Just not getting back.  So then I tried on the server I did a trace route with my workstations IP, 192.168.2.7 and its first hit was going to the 204.180.228.1 address, which is the CISCO with the DS3.  This would make sense since on the server the 204. address is the IP that has a gateway in the settings, while the 1.0 does not.  BUT…. when I had M0n0wall some how the servers knew to send that request back out the 1.0 NIC to the M0n0wall and not to the CISCO, or where ever it is going.  Ethereal did not show where it tried sending the request which was a bummer and I dont know why it did not.....

    Any ideas why these packets do not know where to go?  I could try to setup a route on the CISCO to make any 192.168.0.0 packet go to the PFSense but I can not try that right now because if something happens while people are here at work heads will roll.  :)



  • This is really a routing issue and unless m0n0 was configured to do advanced outbound NAT between the .1.0  and .2.0 networks I don't see any reason why this should have worked with m0n0 in the current configuration.



  • I also had another question off topic.  Its about the Firewall settings.  I'm trying to block web sites like lets say www.myspace.com.  I ping that web site from my workstation and use that IP on the LAN firewall to block packets going to that IP.  Then I go and try to go to myspace.com and I still get there.  It used to work then stopped.

    Could someone give me a tutorial to blocking web sites with PFSense.

    For the Routing problem I'm going to try setting up a rout from the CISCO to tell it to go rout all 192.168.0.0 traffic to the PFSense IP.  That way my servers can keep the gateway on their 204. external address.  It looks like the servers are trying to send their packets out the 204. address which is the CISCO.  I will let you know how that goes. :)

    Thak you for the fast help Hoba!



  • I think myspace is more than one IP. You have to block a subnet I think….oh, it already has been answered here: http://forum.pfsense.org/index.php/topic,1735.0.html



  • Thank you that was very helpful!

    I have another question for you all….

    Since I have gotten PFSense up and running we have been having issues.....

    Randomly our internet will fail......  Our network structure will still be up....  I can log into the PFSense.... just the internet will not work.  I think I remember M0n0wall having this problem when I first set it up.  Seems like it is a DNS problem.  My DNS IP is the IP of my Domain server.....  When PFSense drops our internet on the network, if you go in and put a static IP on your computer with the DNS IP it will work fine.....  It actual just did it again while I was typing this......  Seems to be happening more and more......

    To get PFsense working again I have to reboot it..... im glad it boots fast :)

    Any ideas on what might be wrong?



  • When this happens, log into to pfSense and check to see how many states are in use.



  • @sullrich:

    When this happens, log into to pfSense and check to see how many states are in use.

    Well it just did a little glitchy thing.  We lost internet for about 1 minute but it came back on its own, i did not have to reboot it.  I saw that the state dropped to around 1900.  It is usually at around 2800.  When the internet came back it jumped up to around 3400 states.  its currecntly at around 3000 states right now.  What the heck is this router doing!   ???  ;D



  • @neemers:

    @sullrich:

    When this happens, log into to pfSense and check to see how many states are in use.

    Well it just did a little glitchy thing.  We lost internet for about 1 minute but it came back on its own, i did not have to reboot it.  I saw that the state dropped to around 1900.  It is usually at around 2800.  When the internet came back it jumped up to around 3400 states.  its currecntly at around 3000 states right now.  What the heck is this router doing!   ???  ;D

    Never mind this looks like another IT reset the router so I'm not sure what the state was at.  I will update you when it happens again.



  • @sullrich:

    When this happens, log into to pfSense and check to see how many states are in use.

    Well it has not crashed yet.  what should I be looking for?  High states Low states #?



  • How have you sertup DNS at the pfSense (system>general, dns settings)? Are you using the DNS-forwarder? Does pfSense do DHCP for the clients too (any special settings for DNS at services>dhcp server)? It sounds like only DNS dies for some reason. Also is your WAN DHCP or static or PPPoE and you get the DNS assigned by the ISP?



  • @hoba:

    How have you sertup DNS at the pfSense (system>general, dns settings)? Are you using the DNS-forwarder? Does pfSense do DHCP for the clients too (any special settings for DNS at services>dhcp server)? It sounds like only DNS dies for some reason. Also is your WAN DHCP or static or PPPoE and you get the DNS assigned by the ISP?

    Thank you for the reply!

    In "General Settings", for the DNS I have the IP of our domain server which does our DNS.  DNS forwarder was on by default and I just left it on.  Yes PFsense does DHCP for one of my Lans, the one with all the employee workstations.  The second LAN is just servers with static IPs.  My WAN is static IP plugged into a CISCO router that is plugged into a DS3 for internet.  I want to setup a second WAN for my 5 Static IP DSL as soon as I get PFSense stable.

    Thank you!



  • Make sure your internal DNS server doesn't use the pfSense to resolve too. This might cause a DNS loop. What DNS do you assign to your clients?



  • @hoba:

    Make sure your internal DNS server doesn't use the pfSense to resolve too. This might cause a DNS loop. What DNS do you assign to your clients?

    Nope the gateway on the DNS/Domain server is the second LAN on the PFSense…. and its DNS is its own ip, 192.168.1.195 and another..

    Domain/DNS Server ipconfig /all:

    IP Address. . . . . . . . . . . . : 192.168.1.195
    Subnet Mask . . . . . . . . . . . : 255.255.255.0
    Default Gateway . . . . . . . . . : 192.168.1.1  <----- Second LAN on the PFSense router.
    DNS Servers . . . . . . . . . . . : 192.168.1.195
                                        208.29.225.20

    So everything there looks just fine to me.

    The clients/workstations get a DNS of....

    IP Address. . . . . . . . . . . . : 192.168.2.62
    Subnet Mask . . . . . . . . . . . : 255.255.255.0
    Default Gateway . . . . . . . . . : 192.168.2.3 <------ First LAN with DHCP on PFSense router.
    DHCP Server . . . . . . . . . . . : 192.168.2.3
    DNS Servers . . . . . . . . . . . : 192.168.1.195

    Okay well while I was typing this it went down again.....  The 192.168.2.3 Network, the main network lost Internet.  I could not even ping the PFSense Lan IP, 192.168.2.3

    Now I could not log in to check the state # because I could not even ping 192.168.2.3...... But....

    since I have a second network setup I went to one of the servers and was able to log into the PFSense web console on the 192.168.1.1 LAN.

    The state # was around 500+ and dropping slowly.  Now before this happened the State last I saw was 3000+.

    Here is part of the system log....

    Sep 15 16:23:50 msntp[85624]: msntp: unable to locate IP address/number
    Sep 15 16:23:50 msntp[85624]: msntp: Unknown error: 0
    Sep 15 16:24:50 msntp[85930]: msntp options: a=2 p=0 v=1 e=0.100 E=5.000 P=2147483647.000
    Sep 15 16:24:50 msntp[85930]: d=18000 c=5 x=18000 op=1 l=/var/run/msntp.pid f=/var/db/msntp.state pool.ntp.org
    Sep 15 16:24:50 msntp[85930]: msntp: bad daemon restart information
    Sep 15 16:25:05 msntp[85930]: msntp: unable to locate IP address/number
    Sep 15 16:25:05 msntp[85930]: msntp: Unknown error: 0
    Sep 15 16:25:43 dnsmasq[2625]: reading /var/dhcpd/var/db/dhcpd.leases
    Sep 15 16:26:05 msntp[86175]: msntp options: a=2 p=0 v=1 e=0.100 E=5.000 P=2147483647.000
    Sep 15 16:26:05 msntp[86175]: d=18000 c=5 x=18000 op=1 l=/var/run/msntp.pid f=/var/db/msntp.state pool.ntp.org
    Sep 15 16:26:05 msntp[86175]: msntp: bad daemon restart information
    Sep 15 16:26:20 msntp[86175]: msntp: unable to locate IP address/number
    Sep 15 16:26:20 msntp[86175]: msntp: Unknown error: 0
    Sep 15 16:27:20 msntp[86423]: msntp options: a=2 p=0 v=1 e=0.100 E=5.000 P=2147483647.000
    Sep 15 16:27:20 msntp[86423]: d=18000 c=5 x=18000 op=1 l=/var/run/msntp.pid f=/var/db/msntp.state pool.ntp.org
    Sep 15 16:27:20 msntp[86423]: msntp: bad daemon restart information
    Sep 15 16:27:35 msntp[86423]: msntp: unable to locate IP address/number
    Sep 15 16:27:35 msntp[86423]: msntp: Unknown error: 0
    Sep 15 16:27:40 pftpx[2713]: #520 client reset connection
    Sep 15 16:27:40 pftpx[2713]: #520 client reset connection
    Sep 15 16:27:40 dnsmasq[2625]: reading /var/dhcpd/var/db/dhcpd.leases
    Sep 15 16:27:45 pftpx[2713]: #521 server timeout
    Sep 15 16:27:45 pftpx[2713]: #521 server timeout

    I dont really understand any of this…...

    Now I think I will try tonight to swap network cards because I bought 4 brand new network cards for this project and I did not test them first and the LAN card may be bad.  I'm not using the card setup for DSL yet so I will try and change this and hope I do not have to resetup the whole router again.  :)



  • Please try the following:

    Let the pfSense itself use the external IPS DNS Servers at system>general. Assign your Domain DNS to the clients (you already do that). Also make sure your WAN connection is up and usable during that state. Try pinging an IP like 64.233.187.99 (google.com) while your DNS is down. The systemlogs only show that the DNS was lost and that different processes were not able to resolve IPs due to that for example to run an ntp time sync. Finally check your local DNS for interoperability. Eventprotocols might help you here.



  • @hoba:

    Please try the following:

    Let the pfSense itself use the external IPS DNS Servers at system>general. Assign your Domain DNS to the clients (you already do that). Also make sure your WAN connection is up and usable during that state. Try pinging an IP like 64.233.187.99 (google.com) while your DNS is down. The systemlogs only show that the DNS was lost and that different processes were not able to resolve IPs due to that for example to run an ntp time sync. Finally check your local DNS for interoperability. Eventprotocols might help you here.

    Well now I don't think it has to do with the DNS….. just not sure because so much weired stuff is happening.  I swapped NIC's and that did not help.

    About 15 minutes ago the internet dropped again.  So I logged in to the router and reset it.  Then not even 10 minutes we stopped getting internet again.  So I went to log into the router and I could not even ping it.  I had to go to one of my servers that are on the second LAN and log in that way and reset it.  It's like its getting overloaded........  Every time we drop odd things happen... Sometimes I can ping the router others I can not ping it at all and I'm talking about on the main LAN interface.......

    Under the General I put in a second DNS IP that of the external internet.... it has not helped.....



  • Just for kicks, increase the maximum statelimit at system>advanced to a higher value. This is only limited by your RAM. Search the forum on how to calculate how much states you can push with the amount of ram that your machine has. Bill has described this somewhere. Also set the firewalloptimizations to aggressive. This can also be found at system>advanced.



  • @hoba:

    Just for kicks, increase the maximum statelimit at system>advanced to a higher value. This is only limited by your RAM. Search the forum on how to calculate how much states you can push with the amount of ram that your machine has. Bill has described this somewhere. Also set the firewalloptimizations to aggressive. This can also be found at system>advanced.

    Ok I will try that.  It is really bad today.  I have had everything drop about 3 times within 5 minutes.  I have rebooted it about 10 times already today.  On the last reboot it went down instatly.

    As a reminder I have 3 NICs, 2 LAN and one WAN.  Main LAN is our office 192.168.2.0, second LAN is 192.168.1.0.

    So I have up 3 command windows pinging 192.168.2.3 (Router IP on Main LAN), another pinging 192.168.1.195 ( our domain/DNS server), and another pinging www.yahoo.com.  As of right now I have everyone on the Office using the router as the DNS now.  When everything is about to crash on us I will look at my pings and what I see is….

    Most the time I the ping that is pinging the 192.168.1.195 is not getting a request.  But I will still have internet becasue I changed it that my computer uses the router as the DNS.  Then sometimes I will lose ping to the 192.168.1.195 and I will be ping the router just fine but I can not log into the web console.

    I go in to one of the servers on the 192.168.1.1 and I am able to get on to the web console just fine and reboot it.

    Right now it is crashing about every minute.  I get it booted up and then it crashes.

    On the DOS console it says........ sk2 watchdog timeout



  • @neemers:

    On the DOS console it says…..... sk2 watchdog timeout

    Try a different NIC/NICS.  Intel NICS work great in FreeBSD.



  • @sullrich:

    @neemers:

    On the DOS console it says…..... sk2 watchdog timeout

    Try a different NIC/NICS.   Intel NICS work great in FreeBSD.

    LOL sucks becasue its live right now so all my employees are freaking out. lol i love it.  Any way the NICs I'm using are D-Link DGE-530T High performance Networking 10/100/1000



  • @sullrich:

    @neemers:

    On the DOS console it says…..... sk2 watchdog timeout

    Try a different NIC/NICS.   Intel NICS work great in FreeBSD.

    Is there an Intel Gig NIC you recommend?



  • Depending on your needs grab a 10/100 or 10/100/1000. Intel cards are supported pretty good by freebsd. If you don't have need for a multiportcard you should be able to get them for small money.


Locked