Am I in over my head?

  • So... I scavenged this industrial motherboard from the recycling bin at work. Its a first gen intel i7 with two realtek gigabit NICs. I fabricated a neat little box, soldered a DB9 serial port. installed pfSence through the console. No problem at all. supper easy and was happily in my element with all of that (fun!). The WAN is up, the LAN is up and I can browse into the web GUI. Piece of cake!.... Well that's where my comfort level abruptly ends. I can setup a dumb $50 router from BestBuy. You plug it in and set a password done! that s my level of networking knowledge pretty much.

    So I want to lean and set up a decent firewall and VPN and strictly for home use only I also want to learn a little network engineering along the way. However I am so overwhelmed by pfSence that I'm just lost. Even though I was able to easily install pfSence on some junk I found lying around, it dose not work at all... well I consider 0.03mbps to be a failure of throughput. That's all I can get it to do. I have been reading guides and searching google for a couple of days but I'm a bit cross eyed now.

    Would anyone here be willing to help? I'm usually really good at learning things on my own but this subject has got overwhelmed.

  • Oh I will be willing to donate to anyone who teaches me my complete hardware setup. I scavenged a few of these. I have the last one up on ebay right now. No one seems to want it and i will give it away now. link text

    alt text

  • pfsense in its default form is very usable and safe. Change the default password if you haven't..

    What kind of internet connection are you using? Model of modem? Draw us a mental picture of your whole setup.

    What does this test tell you?

  • Netgate Administrator

    @badfrogg said in Am I in over my head?:

    well I consider 0.03mbps to be a failure of throughput.

    Ha, yeah I'd agree with that! Realtek NICs are not great but.... not that bad.

    With that sort of throttling I would first check the Status > Interfaces page for errors/collisions on the interfaces. Make sure they both show 1Gbps full duplex with autonegotiation. Assuming that that both are connected to Gigabit devices that autonegotiating.


  • @chpalmer said in Am I in over my head?:

    What kind of internet connection are you using? Model of modem? Draw us a mental picture of your whole setup.

    Ok, it couldnt be much simpler. At the moment I have two different ISPs

    The one I'm currently testing is Frontier FiOS (old verison ONT - Motorola ONT1000GJ2) No router, I get a WAN address directly from the ONT.
    So the topology is:
    FiOS ONT > pfSense WAN (re0) - LAN (re1) > computer. That's it.

  • This is what the Status > Interfaces page looks like.


  • Did you have to call frontier to ask them to make any changes to take their router out of the equation?

    Such as convert from the cable connection to the ethernet port? (if the Motorola ONT has that..)

    Have you ever used any other device on the FIOS? Do you get full speed from a direct connection?

    Just trying to rule items out to locate the source.

  • @chpalmer said in Am I in over my head?:

    Did you have to call frontier to ask them to make any changes to take their router out of the equation?

    Such as convert from the cable connection to the ethernet port? (if the Motorola ONT has that..)

    Have you ever used any other device on the FIOS? Do you get full speed from a direct connection?

    Just trying to rule items out to locate the source.

    You may be on to something. I originally asked them to set me up with the direct WAN connection and never used their router. Instead I used a DD-WRT router and that has been set that way for many years.

    Maybe they see a new MAC and shut me down...?

    Oh, and yes. I just plugged the ONT directly to the computer and it did not work,

  • Yeah that is highly possible. We had to pull the battery at one of our sites when we installed.

  • So. while I wait on the phone with Frontier to reset my ONT. I wanted to ask; Do you guys think that this hack job router pieced together out of 10 year old parts is better then a Walmart router for my home?

    Or maybe I was thinking of just getting a Ubiquiti EdgeRouter X for $60

  • Well. Frontier would not provide any support as soon as they figured out that I was using a third party router. It must be some WAN setting specific to this modem but I just dont know.

  • Netgate Administrator

    Try spoofing the MAC address on WAN to whatever the DD-WRT device was.


  • @stephenw10 said in Am I in over my head?:

    Try spoofing the MAC address on WAN to whatever the DD-WRT device was.


    So following these directions to spoof the MAC it says to add a shell command into the config XML. "may require running the NIC in promiscuous mode" I put the MAC of the DD-WRT router in and that did nothing

    Can you tell me what that syntax would be? The example given is:
    "/sbin/ifconfig em0 promisc"
    But for my NIC wouldnt it be, and is the parameter "promisc" even valid for a realtek NIC?
    "/sbin/ifconfig re0 promisc"

  • Netgate Administrator

    That's only in some special cases. Almost all NICs can be spoofed in the WAN interface settings directly, there's a field for it in Interfaces > WAN.


  • I was just doing due diligence since putting in the MAC did not fix the problem. I found an example of a NIC command for promiscuous mode.
    I added the line into the config XML.

    So I was trying to post in the code but I cannot because it flags the post as spam???

  • So here is some more information. I switched over from my Frontier FiOS modem to my Spectrum cable modem and now my speed is 100 time faster. That's 0.03mbps to 3mbps.

    While that is clearly still broken, why would the two ISPs act differently. The FiOS is tested at >50mbps and the Spectum was tested to >400mbps. (with a direct connection to the PC)

    Should I chuck this hardware into the recycle bin where I found it?

  • So this whole time I have been running "speed tests" from my browser. All the top ones when you find with a search for "speed test". They all fail or report less then 3mbps. So then I though to go download real world stuff and this is what happened;

    (I let each test run for at least 15 minutes)

    these were the max speeds reached
    Nvidia Geforce driver - 3mbps
    Download A game from GOG - 3mbps
    Download Windows 10 ISO with the Microsoft media creation tool 1mbps

    Here is where it gets weird.

    So windows 10 was abysmal and was still at only 1% after 15 minutes so I decide to try to download something big at a torrent site. So I go and turn on my VPN and the Microsoft speed instantly goes to 15mbps. So that's interesting. So I turn off VPN, now I'm getting 30mbps. Ok, so I turn it back on and that kills the download...

    So I want to get back to testing stuff after geting derailed by the whole VPN thing. so i just reboot everything and do all the above test over just to make sure its still the same. It is.

    Ok I wanted to download a torrent so I turn on VPN and find something big from the top 100 list so there are plenty of seeds.
    one torrent = about 1 mbps. I loaded up 3 and got about 3mbps

    Now here goes the craziness...
    The VPN is off and i open Steam and download a big game "Whitcher 3" and I get 298mbps !!! peak and no less then 250mbps sustained.

    I turn on the VPN just for fun and nothing changed. (That dip is when I turned the VPN on)

    I tried the other test with VPN on and other then that weird spike from the Microsoft too it seemed to have no impact.

  • @badfrogg said in Am I in over my head?:

    Should I chuck this hardware into the recycle bin where I found it?

    Really knowing nothing about the hardware at all Im curious what is slowing your system down so much. None of my boxes do not do wire speed. But even the one box I have that still is running 32bit (soon to be replaced) will easily do the 100mbps connection that the customer pays for.

    Remember that 8 bits equals 1 byte. If you are seeing download sizes of 10 MBPS when you are expecting 80mbps that is a difference. 10 megabytes per second is 80 megabits per second. Not something to throw into the mix is it?

    pfsense will do 10gbps on the correct hardware. Easily.

  • I've been using megabit per second as a unit of measure this whole time. It's that like using the metric system in America? LOL. because I do that too.

    Well now I'm off to bed. the hardware can clearly do 300 Mbps (or 37 MB/s). But only downloading the Witcher 3 with Steam.

    What that means, I have no idea. I did say I was completely inept with network engineering.

  • Sleep well! I know the frustration.

  • Netgate Administrator

    Hmm, that is odd!

    I would setup a local test between clients on WAN and LAN directly and make sure it can hit line rate in a simple iperf test for example. It should be able to do that no problem except the Realtek NICs will probably limit it. Should be far higher than 400Mbps though.

    Maybe something defaulting to some low speed. CPU stuck at 100MHz or similar? Does it have a standard BIOS?


  • I am using this NOVA-HM551 motherboard with 8GB DDR3 and1066, Core i7-820QM. It has a UEFI BIOS.

  • @stephenw10 said in Am I in over my head?:

    iperf test

    Could you give a high level overview of how to connect one machine to the WAN side. I'm guessing that I would need iperf to run a server on the WAN (or LAN) and then a client on the other side. But how would a computer get on IP from the WAN port?

  • Netgate Administrator

    Exactly, one computer WAN side running iperf server and another LAN side connecting to it using iperf client.

    You could just set both the pfSense WAN and the server machine using static IP addresses since it's only two thigs to configure and only as a test.
    Or you could connect both the pfSense WAN and the server to some other router and use whatever IPs that is handing out via DHCP as long as it's subnet does not conflict with the pfSense LAN. That's what I do here but as you might imagine I am not short of pfSense boxes! 😉


  • Sorry, I tried. I'm certain that I dont know what I'm doing though. This is what I did:

    WAN PC has IP address
    psSence WAN is set to staticIPv4
    psSence LAN is set to staticIPv4
    LAN PC has IP address

    WAN PC
    C:\iperf3 -s

    LAN PC
    C:\iperf3 -c

    .... Nothing happens.

  • Ok. So I changed the subnet mask from /16 to /24 on the computers, I rebooted pfSence and I tuned off the WiFi NIC on the client PC.

    One or all of those things may have solved the iperf problem. iperf says I'm getting 900 Mbps. That's fine with me.


  • Netgate Administrator

    Yeah should have been /24 on everything for that. Otherwise and are inside the same /16 so they try to talk directly.... and fail.

    Yeah 900Mbps is what I'd expect there. You'd probably get close to 941Mbps with Intel NICs.
    There's clearly nothing fundamentally wrong with your hardware.
    Something odd with the connection then somewhere....


  • Well. I just ordered the Ubiquiti EdgeRouter X. Man I hope it is easier than pfsence.

    I will leave the pfsence box on my bench to taunt me. I hear it now.... "you suck....."

  • LAYER 8 Global Moderator

    @badfrogg said in Am I in over my head?:

    Well. I just ordered the Ubiquiti EdgeRouter X. Man I hope it is easier than pfsence.

    No - sorry not even close ;)

  • Netgate Administrator

    Mmm, hard to recommend that on this forum. 😉

    I would expect to be able to get that board working with your connection.

    The fact you get much better speed over a VPN which is probably UDP rather than straight TCP traffic shouts hardware off loading to me. Try going to Sys > Adv > Networking. Make sure all the hardware off-loading options there are checked (which disables them). Reboot to apply that change.


  • LAYER 8 Global Moderator

    Did you take a look see at the manual and how to do things on the device before purchase? Your going to be very disappointed if you think its easier than pfsense that is for damn sure. While it does have a gui - It is multiple steps to just get a single firewall rule in place. You have to create policy, then you have to attach the rule to the policy in the direction you want, etc..

    While I can not say anything bad about it at the price point, and it does route and firewall packets at decent speeds.. Its just not simple to configure..

    If your a cli guy, you can do things via that - which is a pfsense is limited in. Is that something your looking for? ;)

    Good luck with it - let us know when your ready to come back to pfsense ;)

  • Oops, Sorry. Did not intend to troll about a competitor product. I figured that this forum was for the DIY'ers that didn't buy the support. I definitely want to get the pfsense box working, and if for nothing else, to learn.

    It is still running here on my bench. I will start reading the manual but everyone here has said that the pfsense should just work pretty much with default setting. Since it doesn't, I'm afraid of wasting time chasing my tail with the possibility that the hardware that I have will never work.

    I just wanted something better then the average Best Buy or Walmart router. I would not be able to spend $500 on a Netgate router plus the $400 support just to fool around at home.

    I will read the manual. Also. Like I said at the beginning, Anyone who wants to try an make this setup I have work and show me how you did it, I will send you the hardware (you can keep it) I have two of them. Just let me know.

  • @stephenw10 said in Am I in over my head?:

    better speed over a VPN

    No. Let me clarify; the VPN being switched on and then off again created a weird anomaly with just the Microsoft Windows 10 ISO downloader. It was not reproducible and did not occur with any other test.

  • LAYER 8 Global Moderator

    Why would you have to buy support? Paid support, while sure anyone can get it - is more geared towards the enterprise where something down cost money.. Do you think your going to get a phone number to call with your ER-X at $60 ;)

    To play around, why not get the sg1100.. Or the 3100? All in with the price hike its only $400.. Just ordered our 4th one (sitting on desk waiting for setup). We don't have any support on them - being that I have been using pfsense for 10 some years doesn't really have anything to do with it ;) They are not currently used in a mission critical scenario... If went down for a day or so while it would suck.. Not going to hurt the business in any significant way. Kind of like your home setup.. If the pfsense does blow up, just throw any old $20 wifi router on til you can get it up and running again.

    Not sure where are you having difficulties... Looks like you tested the hardware and were getting 900mbps... how can you complain at that with that old hardware?

    If your seeing 900mbps through pfsense locally - and slowness on internet - I would look to isp what/how your connecting to your internet.. Duplex mismatch, isp throttling new mac? Did you try the mac clone idea?

  • Netgate Administrator

    I would still make sure all hardware off-loading is disabled. That's exactly the sort of thing that some Realtek NICs get very wrong!


  • If I plug my modem directly into my computer, I get my full service speed of 400 Mbps. (actually I get a bit more). If the pfsence box is in between the speed is between 0.03 and 3 Mbps. I can hardly even post to this forum at that rate.

    The one anomaly that I found was downloading a game from Steam. which gave me almost 300 Mbps (I think Steam is throttling this because I get that same speed when I tried with a direct connection)

    Everything else that I could think to test showed a severely crippled connection data rate. Could not download files or stream video with any acceptable speed (Netflix, Sling, YouTube etc... ) i.e. Yes the file would download (6 hours for an Nvidia driver) and video would play for 5 seconds and buffer for a 1 minute.

  • LAYER 8 Global Moderator

    So your seeing wan connect on pfsense at gig? Your not seeing errors on the interface?

  • @stephenw10 said in Am I in over my head?:

    make sure all hardware off-loading is disabled

    Oh yes. I tried this many time and it made no difference. I have loaded system defaults about 6 or 7 times trying different suggestions that i have found thorough the manual and searching google for answers.

    After trying stuff that i dont even understand (like command line tweaking in system tunables) and not getting results I just load the defaults and try again.

  • @johnpoz said in Am I in over my head?:

    So your seeing wan connect on pfsense at gig? Your not seeing errors on the interface?

    Not sure I understand but. I got 900 Mbps through the box on the bench with iperf. The dashboard interface status shows 0 errors and 0 collisions.

    Does that answer? When the box is connected to my modem is when it goes bad.

    Oh, and I just tried this. I put the pfsence box behind my old DD-WRT router and it works fine. (at the limit of that old router which is like 80 Mbps)

  • LAYER 8 Global Moderator

    This should work out of the box.... If out of the box you see 900 mbps natting from wan to lan test with iperf... Why would you think it would be any different sending a few packets back and forth to your isp? That screams bad connection to whatever you wan is..

    What connection speed is coming on the wan interface when you plug it into your isp? Your getting a public IP? Or do you have a overlap issue with wan and lan networks?

    edit: works fine behind the other router would hint at you have an overlap on networks maybe... Like you first had when you tried to test where you thought you could use /16 and then put a /24 on your lan that is inside the /16 network.

    Maybe your dd-wrt is using 192.168.2 vs 192.168.1 like your isp device... You can not have overlapping networks on your wan and lan and expect anything to actually work.

Log in to reply