Traffic Static routing problem

nnicanor · Sep 11, 2011, 4:04 AM

I have installed pfsense 1.2.1 with static routes to connect different offices with deferents subnet and works fine, i'm and testing with new 2.0rc3 and i have problem with traffic through static routes (SMTP,FTP,HTTP, any) all transfers begin and stops about 60Kb or 70kb, when i use 1.2.1 version all work fine.

This is my network

Principal Office LAN Network 192.168.0.0/24 WAN 19X.9X.16X.2XX

Static Routes 192.168.5.0/24 Gateway 192.168.0.1
192.168.9.0/24 Gateway 192.168.0.1

Office1 Network LAN 192.168.5.0/24 WAN 19X.10X.24X.1X

Static Routes 192.168.0.0/24 Gateway 192.168.5.1
192.168.9.0/24 Gateway 192.168.5.1

Office2 Network LAN 192.168.9.0/24 WAN 19X.X.13X.13X

Static Routes 192.168.0.0/24 Gateway 192.168.9.3
192.168.5.0/24 Gateway 192.168.5.1

I'am testing since version 2.0Beta and problem still reaming, used with different MTU, different network cards and i 'cant solve problem, Traffic through internet is ok

no rules defined to block traffic, system logs and firewall logs don't show any error, or report any block

Network cards used: Dlink 528T, 520TX, Intel , and virtualized environment and same problem occurs, this is dmesg output.

Rebooting…
Copyright (c) 1992-2010 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 8.1-RELEASE-p4 #1: Sat Sep 10 17:04:53 EDT 2011
root@FreeBSD_8.0_pfSense_2.0-snaps.pfsense.org:/usr/obj./usr/pfSensesrc/src/sys/pfSense.8 i386
Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: Intel(R) Pentium(R) 4 CPU 2.80GHz (2799.52-MHz 686-class CPU)
Origin = "GenuineIntel" Id = 0xf29 Family = f Model = 2 Stepping = 9
Features=0xbfebfbff <fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,clflush,dts,acpi,mmx,fxsr,sse,sse2,ss,htt,tm,pbe>Features2=0x4400 <cnxt-id,xtpr>real memory = 1207959552 (1152 MB)
avail memory = 1158365184 (1104 MB)
ACPI APIC Table: <compaq intel845="">pnpbios: Bad PnP BIOS data checksum
ioapic0 <version 2.0="">irqs 0-23 on motherboard
netisr_init: forcing maxthreads to 1 and bindthreads to 0 for device polling
wpi: You need to read the LICENSE file in /usr/share/doc/legal/intel_wpi/.
wpi: If you agree with the license, set legal.intel_wpi.license_ack=1 in /boot/loader.conf.
module_register_init: MOD_LOAD (wpi_fw, 0xc0988330, 0) error 1
ipw_bss: You need to read the LICENSE file in /usr/share/doc/legal/intel_ipw/.
ipw_bss: If you agree with the license, set legal.intel_ipw.license_ack=1 in /boot/loader.conf.
module_register_init: MOD_LOAD (ipw_bss_fw, 0xc0789370, 0) error 1
ipw_ibss: You need to read the LICENSE file in /usr/share/doc/legal/intel_ipw/.
ipw_ibss: If you agree with the license, set legal.intel_ipw.license_ack=1 in /boot/loader.conf.
module_register_init: MOD_LOAD (ipw_ibss_fw, 0xc0789410, 0) error 1
ipw_monitor: You need to read the LICENSE file in /usr/share/doc/legal/intel_ipw/.
ipw_monitor: If you agree with the license, set legal.intel_ipw.license_ack=1 in /boot/loader.conf.
module_register_init: MOD_LOAD (ipw_monitor_fw, 0xc07894b0, 0) error 1
wlan: mac acl policy registered
kbd1 at kbdmux0
cryptosoft0: <software crypto="">on motherboard
padlock0: No ACE support.
acpi0: <compaq intel845="">on motherboard
acpi0: Overriding SCI Interrupt from IRQ 9 to IRQ 20
acpi0: [ITHREAD]
acpi0: Power Button (fixed)
Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x808-0x80b on acpi0
cpu0: <acpi cpu="">on acpi0
acpi_button0: <power button="">on acpi0
pcib0: <acpi host-pci="" bridge="">port 0xcf8-0xcff on acpi0
pci0: <acpi pci="" bus="">on pcib0
vgapci0: <vga-compatible display="">mem 0xd0000000-0xd7ffffff,0xdff80000-0xdfffffff irq 16 at device 2.0 on pci0
agp0: <intel 82845m="" (845m="" gmch)="" svga="" controller="">on vgapci0
agp0: detected 892k stolen memory
agp0: aperture size is 128M
pcib1: <acpi pci-pci="" bridge="">at device 30.0 on pci0
pci3: <acpi pci="" bus="">on pcib1
bfe0: <broadcom bcm4401="" fast="" ethernet="">mem 0xdfdfe000-0xdfdfffff irq 19 at device 10.0 on pci3
miibus0: <mii bus="">on bfe0
bmtphy0: <bcm4401 10="" 100basetx="" phy="">PHY 1 on miibus0
bmtphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
bfe0: [ITHREAD]
re0: <d-link dge-528(t)="" gigabit="" ethernet="" adapter="">port 0xcc00-0xccff mem 0xdfdfdf00-0xdfdfdfff irq 21 at device 11.0 on pci3
re0: Chip rev. 0x10000000
re0: MAC rev. 0x00000000
miibus1: <mii bus="">on re0
rgephy0: <rtl8169s 8110s="" 8211b="" media="" interface="">PHY 1 on miibus1
rgephy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto
re0: [FILTER]
re1: <d-link dge-528(t)="" gigabit="" ethernet="" adapter="">port 0xc800-0xc8ff mem 0xdfdfde00-0xdfdfdeff irq 22 at device 13.0 on pci3
re1: Chip rev. 0x10000000
re1: MAC rev. 0x00000000
miibus2: <mii bus="">on re1
rgephy1: <rtl8169s 8110s="" 8211b="" media="" interface="">PHY 1 on miibus2
rgephy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto
re1: [FILTER]
re2: <d-link dge-528(t)="" gigabit="" ethernet="" adapter="">port 0xc400-0xc4ff mem 0xdfdfdd00-0xdfdfddff irq 18 at device 15.0 on pci3
re2: Chip rev. 0x10000000
re2: MAC rev. 0x00000000
miibus3: <mii bus="">on re2
rgephy2: <rtl8169s 8110s="" 8211b="" media="" interface="">PHY 1 on miibus3
rgephy2: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto
re2: [FILTER]
isab0: <pci-isa bridge="">at device 31.0 on pci0
isa0: <isa bus="">on isab0
atapci0: <intel ich4="" udma100="" controller="">port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xfc00-0xfc0f at device 31.1 on pci0
ata0: <ata 0="" channel="">on atapci0
ata0: [ITHREAD]
ata1: <ata 1="" channel="">on atapci0
ata1: [ITHREAD]
pci0: <serial bus,="" smbus="">at device 31.3 (no driver attached)
pci0: <multimedia, audio="">at device 31.5 (no driver attached)
fdc0: <floppy drive="" controller="">port 0x3f2-0x3f3,0x3f4-0x3f5,0x3f7 irq 6 drq 2 on acpi0
fdc0: [FILTER]
uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
uart0: [FILTER]
atrtc0: <at realtime="" clock="">port 0x70-0x71 irq 8 on acpi0
atkbdc0: <keyboard controller="" (i8042)="">port 0x60,0x64 irq 1 on acpi0
atkbd0: <at keyboard="">irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
atkbd0: [ITHREAD]
pmtimer0 on isa0
orm0: <isa option="" rom="">at iomem 0xe0000-0xe0fff pnpid ORM0000 on isa0
sc0: <system console="">at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
vga0: <generic isa="" vga="">at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
ppc0: parallel port not found.
p4tcc0: <cpu frequency="" thermal="" control="">on cpu0
Timecounter "TSC" frequency 2799523520 Hz quality 800
Timecounters tick every 1.000 msec
IPsec: Initialized Security Association Processing.
acd0: CDRW <lite-on ltr-48327s="" pqs3="">at ata0-master UDMA33
ad2: 38166MB <seagate st340015a="" 3.01="">at ata1-master UDMA100
GEOM: ad2: partition 1 does not start on a track boundary.
GEOM: ad2: partition 1 does not end on a track boundary.
GEOM: ad2s1: geometry does not match label (16h,63s != 16h,255s).
Trying to mount root from ufs:/dev/ad2s1a
ovpns1: link state changed to UP
pflog0: promiscuous mode enabled
bfe0: link state changed to UP
re2: link state changed to UP
re2_vlan2: link state changed to UP
re2_vlan3: link state changed to UP
re1: link state changed to UP
re0: link state changed to UP
re1: promiscuous mode enabled
re1: promiscuous mode disabled
re1: promiscuous mode enabled

Regards,

Nicanor Martinez Martinez</seagate></lite-on></cpu></generic></system></isa></at></keyboard></at></floppy></multimedia,></serial></ata></ata></intel></isa></pci-isa></rtl8169s></mii></d-link></rtl8169s></mii></d-link></rtl8169s></mii></d-link></bcm4401></mii></broadcom></acpi></acpi></intel></vga-compatible></acpi></acpi></power></acpi></compaq></software></version></compaq></cnxt-id,xtpr></fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,clflush,dts,acpi,mmx,fxsr,sse,sse2,ss,htt,tm,pbe>

stephenw10 · Sep 11, 2011, 4:30 PM

This is confusing.
Are these three separate networks with three pfSense boxes or three subnets behind one box?

If this is simply 3 subnets behind one box you don't need to add static routes and gateways to access subnets on another adapter. You just need to set firewall rules to allow it.

Also it appears you have VLANs, which interfaces are doing what?

Do you mean 1.2.1? Why aren't you using 1.2.3?

Steve

nnicanor · Sep 11, 2011, 11:48 PM

@stephenw10:

This is confusing.
Are these three separate networks with three pfSense boxes or three subnets behind one box?

If this is simply 3 subnets behind one box you don't need to add static routes and gateways to access subnets on another adapter. You just need to set firewall rules to allow it.

Also it appears you have VLANs, which interfaces are doing what?

Do you mean 1.2.1? Why aren't you using 1.2.3?

Steve

I have a pfsense box in each office with 3 Network cards for WAN1 , WAN2 and LAN, all are phisical interfaces no using VLAN.

Regards
Nicanor Martinez

wallabybob · Sep 12, 2011, 5:51 AM

@nnicanor:

I have a pfsense box in each office with 3 Network cards for WAN1 , WAN2 and LAN, all are phisical interfaces no using VLAN.

Then why does the dmesg output report:
@nnicanor:

re2: link state changed to UP
re2_vlan2: link state changed to UP
re2_vlan3: link state changed to UP

It would be helpful to have a diagram of the network showing the interfaces used, their IP addresses and network masks, a clear statement of what transfers work and what transfers don't work and a copy of the failure report from the application doing the transfer.

@nnicanor:

Office2 Network LAN 192.168.9.0/24 WAN 19X.X.13X.13X

Static Routes 192.168.0.0/24 Gateway 192.168.9.3
192.168.5.0/24 Gateway 192.168.5.1

If the second static route is correct, how does this system get to 192.168.5.1?

cmb · Sep 13, 2011, 12:50 AM

Newer versions of PF are more strict on TCP correctness, you have asymmetric routing and must check "Bypass firewall rules for traffic on the same interface" under System>Advanced, Firewall.

nnicanor · Sep 28, 2011, 3:34 PM

Hello,

This option is already enabled since beta version, all office have internet connections and direct data channel with one router that is configured in static route configuration of pfsense 2.0, i dont have problem with this configuration with 1.2.1.

Regards

Nicanor Martinez

cmb · Sep 29, 2011, 10:01 PM

@nnicanor:

This option is already enabled since beta version, all office have internet connections and direct data channel with one router that is configured in static route configuration of pfsense 2.0, i dont have problem with this configuration with 1.2.1.

See my last post. You don't have that set somewhere where you have asymmetric routing.