ATT Uverse RG Bypass (0.2 BTC)
-
@t41k2m3 said in ATT Uverse RG Bypass (0.2 BTC):
@aus et al - just a quick update, tried trimming down the graph (to just ngeh0, vlan0, ONT_IF, RG_IF) and am seeing same issues:
Some speed degradation (though much more manageable relative to other examples here - 100-200 Mbps loss on Gig WAN);
Connection drops 1-2 times per 24-hour period, which did not use to happen with quite the same frequency when connected via ATT RG in passthrough mode. When it drops, WAN/ngeth0 keeps public IP, however gateway becomes unreachable which triggers disconnect/reconnect bringing whole network down). Sometimes (not always) when disconnects happen, they are preceded by an error like "arpresolve: can't allocate llinfo for <GW IP> on ngeth0". Unable so far to pinpoint a root cause or possible fix (tried suggestions from @aus ). May try disabling gateway monitoring or action, though not sure that's ideal long term in case of more serious connectivity disruptions.Anyone else experience this?
That's too bad the netgraph alterations didn't work. At least, we have ruled out that most of the netgraph does not cause speed degradation for you. The "stripped" netgraph is pretty simple, leaving pretty much only ng_vlan (for VLAN0 tagging) and ng_eiface (for creating a NIC). Neither would be very CPU intensive, in theory.
Regarding your connection drops, the best way to debug this is to catch the problem with tcpdumps running on
$RG_IF
and$ONT_IF
. Maybe you could filter to just EAP and DHCP traffic to reduce load if you can't reproduce. It kind of sounds like something is confusing your DHCP lease on your WAN.@pyrodex said in ATT Uverse RG Bypass (0.2 BTC):
I just got a new Xeon E3-1230v6 thanks to work and replaced my C2758 firewall last night. Right now I am using IP-Passthrough but will swap back to the method in this thread and see if the performance is still an issue.
If I can't get good performance with a semi modern xeon then I am afraid it is just never going to happen.Interested to see how the Xeon shakes out for you.
-
@aus looks like I was able to pull nearly full line speed with the bridge on a E3-1230V6, I saw ~112+ MB/s when pulling a public linux distro torrent.
/0 /1 /2 /3 /4 /5 /6 /7 /8 /9 /10 Load Average || Interface Traffic Peak Total ngeth0 in 30.936 KB/s 114.354 MB/s 27.332 GB out 467.484 KB/s 1.479 MB/s 979.165 MB igb3 in 0.000 KB/s 0.082 KB/s 14.170 KB out 0.000 KB/s 0.000 KB/s 4.050 KB igb0 in 33.320 KB/s 114.846 MB/s 27.479 GB out 472.396 KB/s 1.619 MB/s 999.016 MB
-
Thanks for the update! Glad you were able sort the speed issues out.
-
I wanted to come here and post a giant "Thank You" for all the work aus and others helping him put forth.
I was finally able to get my SuperMicro C2758 pfsense box working directly with the fiber ONT.
I have 4 Gig ports configured as follows:
igb0 - AT&T Fiber ONT
igb1 - L3 Switch
igb2 - UVERSE DVR - VIP2250
igb3 - RG - BGW210One thing I've noticed, is the RG only ever has a single GREEN LED lit. I've powered cycled the RG and the fiber ONT and everything still worked, so.... //shrug//
My throughput is fantastic (I have Gig service) and my latency dropped ever so slightly as well. Now, my task is getting the DVR to work. Coincidentally, my old unit died, so I just received my replacement.
One thing I noticed, is the UVERSE LAN needs the AT&T DNS Servers in order to work. I can't filter them through Cloudfare, or other.
I had IGMP Proxy setup just fine with the old box, but now it doesn't seem to be working. Any thoughts? -
@misterbaz said in ATT Uverse RG Bypass (0.2 BTC):
I had IGMP Proxy setup just fine with the old box, but now it doesn't seem to be working. Any thoughts?
You should probably start another thread.
-
@misterbaz said in ATT Uverse RG Bypass (0.2 BTC):
I wanted to come here and post a giant "Thank You" for all the work aus and others helping him put forth.
Glad to help out!One thing I've noticed, is the RG only ever has a single GREEN LED lit. I've powered cycled the RG and the fiber ONT and everything still worked, so.... //shrug//
This is normal and expected. The RG never reaches full green status because it is expecting to negotiate a DHCP lease. However, netgraph drops that traffic because pfSense is handling the DHCP. You can actually keep the RG disconnected after the 802.1X EAP-TLS authentication completes. However, if your igb0 looses its link (due to power outage, unplug, reboot, or whatever), you will loose connectivity until the RG is reconnected and can authenticate you.
One thing I noticed, is the UVERSE LAN needs the AT&T DNS Servers in order to work. I can't filter them through Cloudfare, or other.
This might be true for set top boxes or DVRs, but your entire LAN does not need to use AT&T DNS servers.
I had IGMP Proxy setup just fine with the old box, but now it doesn't seem to be working. Any thoughts?
There's been a few threads about configuring the IGMP proxy for AT&T. Basically, it involved adding some of AT&Ts IP ranges. I had it working a while ago, but no longer have TV service to test. You might continue the conversation here:
https://github.com/aus/pfatt/issues/3
Definitely accepting PRs if you figure it out.
-
@aus said in ATT Uverse RG Bypass (0.2 BTC):
@misterbaz said in ATT Uverse RG Bypass (0.2 BTC):
I wanted to come here and post a giant "Thank You" for all the work aus and others helping him put forth.
Glad to help out!One thing I've noticed, is the RG only ever has a single GREEN LED lit. I've powered cycled the RG and the fiber ONT and everything still worked, so.... //shrug//
This is normal and expected. The RG never reaches full green status because it is expecting to negotiate a DHCP lease. However, netgraph drops that traffic because pfSense is handling the DHCP. You can actually keep the RG disconnected after the 802.1X EAP-TLS authentication completes. However, if your igb0 looses its link (due to power outage, unplug, reboot, or whatever), you will loose connectivity until the RG is reconnected and can authenticate you.
One thing I noticed, is the UVERSE LAN needs the AT&T DNS Servers in order to work. I can't filter them through Cloudfare, or other.
This might be true for set top boxes or DVRs, but your entire LAN does not need to use AT&T DNS servers.
Correct. The rest of my LAN has Cloudfare DNS servers assigned. The UVERSE DVR is the only thing that needed to see the AT&T DNS Servers.
I had IGMP Proxy setup just fine with the old box, but now it doesn't seem to be working. Any thoughts?
There's been a few threads about configuring the IGMP proxy for AT&T. Basically, it involved adding some of AT&Ts IP ranges. I had it working a while ago, but no longer have TV service to test. You might continue the conversation here:
https://github.com/aus/pfatt/issues/3
Definitely accepting PRs if you figure it out.
I figured it out, sort of. I had been trying to nail down every multicast server I could see through pfTop, but it was still hanging up on a lot of channels. So, I instead made a blanket 0.0.0.0/1 (pfSense won't let you use /0) statement in the Upstream setting and every channel came through. This might seem terrible, but remember my UVERSE DVR is on its own separate LAN independent from my normal LAN. Also, this still won't work until you setup an allow rule through your firewall. I believe I only had to setup an alow rule to my UVERSE LAN for 224.0.0.0/8. There might have also been an allow rule for 239.0.0.0/8. I'll have to check it out when I get back home.
-
Just wanted to chime in and say that this worked great for me. I also have some static IPs from ATT (10$/mo for 5 extra) and I was able to utilize them no issues without the gateway.. Not sure if it was mentioned how to before.
I did so by creating an another lan interface tagged to vlan 99 dubbed Public Vlan. I assigned the "Gateway IP" given to me by ATT to this interface. after some proper firewall rules to allow traffic from outside to this network and disabling NAT on those addresses. I can confirm the public ip subnet is able to get out and traffic does return as desired. If you need more information just PM me
-
@aus Thanks for your work on this. https://btc.com/cae38d113459909c8b23dc20553628cdc79a2150ed87c9724701fcb95f67814c
-
Thank you! That’s very generous of you and much appreciated. I’m glad this solution worked for you. Cheers!
-
-
@Dade I figured out the static IP, I just setup 1:1 nat, it was pretty straightforward.
-
I set this up a month ago and worked great for 2 week so. Then for the past 2 weeks my connection will drop randomly every day or few days. My logs are linked below. But another question I have is the link to the gateway suppose to work also? When I connect to it it says there is no link and to contact ATT but my PFsense internet works.
-
@Makaveli6103 said in ATT Uverse RG Bypass (0.2 BTC):
I set this up a month ago and worked great for 2 week so. Then for the past 2 weeks my connection will drop randomly every day or few days. My logs are linked below. But another question I have is the link to the gateway suppose to work also? When I connect to it it says there is no link and to contact ATT but my PFsense internet works.
I believe @aus stated that (the gateway not having link) is expected since all traffic from the gateway other than authentication is blocked.
-
Has anyone here using this bypass method noticed the WAN lease time obtained from ATT DHCP is only 1 hour long? Apparently people using other bypass methods have noticed it as well (see here) It only seems to happen when bypassing the gateway. The lease obtained when using ip-passthrough mode is much longer. I can see that in /var/db/dhclient.leases.ngeth0 the leases are 3600 seconds and going a grep "renewal in" /var/log/dhcpd.log shows the lease renewing every 1800 seconds which is the 1/2way point of a 3600 second lease. I'm curious if anyone can come up with a way to increase that lease time. I tried using a "send" dhcp-lease-time option to increase it, but it didn't help; the lease obtained was still 3600 seconds.
-
@gfeiner ok thanks. But any idea why my connection drops?
-
@Makaveli6103 said in ATT Uverse RG Bypass (0.2 BTC):
@gfeiner ok thanks. But any idea why my connection drops?
What is connected to igb2? The ONT or the gateway? Your logs are showing link going up and down on that port. Bad cable or faulty device connected to that port.
-
@gfeiner igb2 is the gateway. I will change the cable. I did also turn of gateway monitoring to see if that does anything.
-
@gfeiner I think I know what was wrong. I have the Pace 5268AC gateway and I did add the script to help with the EAP-Logoff issue. But when i added the script to the /rc.d folder I forgot to add .sh. I added it and now see it running in the logs. Hopefully this was the issue.
-
@Makaveli6103 said in ATT Uverse RG Bypass (0.2 BTC):
@gfeiner I think I know what was wrong. I have the Pace 5268AC gateway and I did add the script to help with the EAP-Logoff issue. But when i added the script to the /rc.d folder I forgot to add .sh. I added it and now see it running in the logs. Hopefully this was the issue.
Good to know.
-
@gfeiner
I just got the ATT 1g service, internet only, no TV or VOIP. The RG is BGW210. I changed it's IP addr to 192.168.100.1 because as delivered it was the same as my pfSense box. Passthrough gives me a 5 minute lease although the setup screen is has a different lease time.I have an SG-2440 and behind that an unmanaged 1g switch. My speedtests run around 550 Mbps, the RG Diagnostic menu has a speed test built in which shows that it is doing ~950Mbps.
I realize my setup is double NAT, I am reading here to find out if I can get rid of the double NAT and if that will increase my throughput.
I'm still seeking more info on the pfatt patch.
-
@JonH did you read the instructions on the GitHub? There is a little learninf curve but isn't too hard.
-
Just a quick note that the etf kernel module is now available as a command-line-installable package from the Netgate repos.
[2.4.4-RELEASE][root@pfSense]/root: pkg search etf ng_etf-kmod-0.1 ng_etf kernel module [2.4.4-RELEASE][root@pfSense]/root: pkg install ng_etf-kmod Updating pfSense-core repository catalogue... pfSense-core repository is up to date. Updating pfSense repository catalogue... pfSense repository is up to date. All repositories are up to date. The following 1 package(s) will be affected (of 0 checked): New packages to be INSTALLED: ng_etf-kmod: 0.1 [pfSense] Number of packages to be installed: 1 3 KiB to be downloaded. Proceed with this action? [y/N]:
No need to scp it from another FreeBSD node and it should track updates by FreeBSD.
-
@Derelict said in ATT Uverse RG Bypass (0.2 BTC):
etf kernel module is now available
Nice. Thanks for this info
-
@JonH I've installed pfatt 2 days ago, running w/o problems except my speed tests are still ~550 (~950 if wired directly as per AT&T). I'm not running Snort or Suricata. My cpu generally runs < 15%.
pfatt.sh contains (in addition to RG MAC addr:
ONT_IF='igb0'
RG_IF='igb3'/usr/sbin/ngctl list
There are 13 total nodes:
Name: igb0 Type: ether ID: 00000001 Num hooks: 1
Name: <unnamed> Type: socket ID: 00000007 Num hooks: 0
Name: <unnamed> Type: socket ID: 0000006a Num hooks: 0
Name: <unnamed> Type: socket ID: 0000006b Num hooks: 0
Name: <unnamed> Type: socket ID: 0000006c Num hooks: 0
Name: <unnamed> Type: socket ID: 0000006d Num hooks: 0
Name: o2m Type: one2many ID: 0000000d Num hooks: 3
Name: vlan0 Type: vlan ID: 00000010 Num hooks: 2
Name: ngctl25207 Type: socket ID: 000000d3 Num hooks: 0
Name: ngeth0 Type: eiface ID: 00000013 Num hooks: 1
Name: waneapfilter Type: etf ID: 00000017 Num hooks: 2
Name: laneapfilter Type: etf ID: 0000001b Num hooks: 1
Name: igb3 Type: ether ID: 0000005d Num hooks: 0One question is my interface assignments in the pfSense web configurator: The pfatt readme says "pfSense will detect new interfaces on bootup. Follow the prompts on the console to configure
ngeth0
as your pfSense WAN."
In my case I didn't get any prompts so I read this to mean I should have ngeth0 as my WAN interface. Thus, I changed the WAN from igb0 to ngeth0 (and spoofing RG MAC). This leaves igb0 as "available".Is this correct or am I misreading the readme? Should WAN remain igb0?
There was one comment earlier in this thread to make sure pfatt was being executed at <earlyshellcmd>. How would I determine that? And the etf filters have less hooks than an example posted earlier in this thread. Is that important?
-
I would not edit the configuration to add the shell command. I would use the Shell Command package. There is an option there to select early.
-
@Derelict said in ATT Uverse RG Bypass (0.2 BTC):
I would use the Shell Command package.
Thank you. I was not aware of that package.
I'll give it a shot. -
re: which interface, your WAN should be ‘ngeth0’. If pfSense doesn’t prompt you to configure, you should manually set it.
re: performance, early shell cmd won’t improve that. Unfortunately, Netgraph configured as such does add a bit of CPU overhead at high network utilization. If your total CPU does not exceed ~15% under high network utilization, I would double check your single core performance. It may be maxed on a single core.
I’ve tested pfatt on a couple different boxes. Some performed better than others. My current CPU can mostly saturate (900+) my 1000/1000 plan:
AMD GX-420CA SOC
Current: 800 MHz, Max: 2000 MHz
4 CPUs: 1 package(s) x 4 core(s)
AES-NI CPU Crypto: Yes (active)Supplicant mode has a little less overhead since the Netgraph is simpler. You might get more out of your hardware with that.
-
@aus: Thanks for feedback.
ngeth0 is on WAN. In the Interface Assignments menu that leaves igb0 down.
My CPU at ~15% is just average network usage. I don't run web servers. I have minimal streaming.
According to top, running in the shell, my largest cpu load is ntopng, I have disabled that and there is no noticeable improvement.pfSense is running on a SG-2440 appliance (pre-Netgate appliance). It has 2 Atom C2358 1.7 GHz cpu's. I don't know how to check the individual cpu performance.
For crypto I think my setting is default, I don't recall setting it. It is set to BSD cryptodev but I will try no crypto to see if there is a noticeable difference.
I'm using a dumb switch.
I'm have a BGW210-700 & not using the AT&T wifi.
Is Supplicant Mode a function of compiling the etf.ko? If not, how do I remove it? I'm using Derelict's Build.For kicks, I unplugged my LAN cable (igb1) and plugged a linux box directly into it (leaving a single NAS on igb2 & the RG on igb3). Same ~500 speedtest.net results. That linux box plugged into AT&T default setup is ~800-900.
You are at 4 cores, I'm at 2 cores. Maybe my throughput is the best I can expect with my SG-2440?
-
FYI. I'm doing this bypass on my netgate SG5100 and I can get in the 900-940Mb range with ATT UVERSE gigabit plan. So maybe it is your CPU.
-
@gfeiner The pfSense CPU? I'm starting to think that.
-
Be sure powerd is enabled and set to Hiadaptive or Maximum in System > Advanced, Miscellaneous
-
@JonH said in ATT Uverse RG Bypass (0.2 BTC):
I'm using Derelict's Build.
To be clear, it is not my build I'm just the messenger. The main developers at Netgate built it.
-
@Derelict Thank You, it (powerd)was previously set that way.
I'm going to disable pfBlockerNG to see if that is making a substantial hit on throughput. -
@gfeiner said in ATT Uverse RG Bypass (0.2 BTC):
So maybe it is your CPU
Link below show results of shell command 'systat load' while doing speed test. If I understand the output correctly it looks like my CPU are doing ok.
![ScreenShot](<a href="https://imgur.com/oW4yqgC"><img src="https://i.imgur.com/oW4yqgC.png" title="source: imgur.com" /></a>)I also did a speed test with pfBlockerNG disabled and there was negligible improvement.
-
Has anyone tried using this netgraph method along with the certificate extraction from gateway method? I have the wpa_supplicant method working, but still have to use the 5port netgear switch in the middle of my ONT and PFsense WAN because of VLAN0. Wondering how i could use netgraph to deal with VLAN 0 issue.
-
So I got things working by not using any netgraph scripts on my ESXi 6.7u2 virtualized pfSense instance. If you follow the instructions below, you should get things working.
- Set up a new VSWITCH, port group with VLAN(0) and uplink on a dedicated network uplink (Allow mac address spoofing and the other two just incase)
- Connect the ONT to this uplink
- Create a new e1000e interface that resides in the port group from 1) in pFsense (em0 for me). I tried vmxnet3 and didn't seem to work
- I just took the portion of the script below to start wpa_supplicant. Find all em0 below and change with your adapter.
/usr/bin/logger -st "pfatt" "starting wpa_supplicant..." WPA_PARAMS="\ set eapol_version 2,\ set fast_reauth 1,\ ap_scan 0,\ add_network,\ set_network 0 ca_cert \\\"/conf/pfatt/wpa/ca.pem\\\",\ set_network 0 client_cert \\\"/conf/pfatt/wpa/client.pem\\\",\ set_network 0 eap TLS,\ set_network 0 eapol_flags 0,\ set_network 0 identity \\\"$EAP_SUPPLICANT_IDENTITY\\\",\ set_network 0 key_mgmt IEEE8021X,\ set_network 0 phase1 \\\"allow_canned_success=1\\\",\ set_network 0 private_key \\\"/conf/pfatt/wpa/private.pem\\\",\ enable_network 0\ " WPA_DAEMON_CMD="/usr/sbin/wpa_supplicant -Dwired -iem0 -B -C /var/run/wpa_supplicant" # kill any existing wpa_supplicant process PID=$(pgrep -f "wpa_supplicant.*em0") if [ ${PID} > 0 ]; then /usr/bin/logger -st "pfatt" "terminating existing wpa_supplicant on PID ${PID}..." RES=$(kill ${PID}) fi # start wpa_supplicant daemon RES=$(${WPA_DAEMON_CMD}) PID=$(pgrep -f "wpa_supplicant.*em0") /usr/bin/logger -st "pfatt" "wpa_supplicant running on PID ${PID}..." # Set WPA configuration parameters. /usr/bin/logger -st "pfatt" "setting wpa_supplicant network configuration..." IFS="," for STR in ${WPA_PARAMS}; do STR="$(echo -e "${STR}" | sed -e 's/^[[:space:]]*//')" RES=$(eval wpa_cli ${STR}) done # wait until wpa_cli has authenticated. WPA_STATUS_CMD="wpa_cli status | grep 'suppPortStatus' | cut -d= -f2" /usr/bin/logger -st "pfatt" "waiting EAP for authorization..." # TODO: blocking for bootup while true; do WPA_STATUS=$(eval ${WPA_STATUS_CMD}) if [ X${WPA_STATUS} = X"Authorized" ]; then /usr/bin/logger -st "pfatt" "EAP authorization completed..." break else sleep 1 fi done /usr/bin/logger -st "pfatt" "em0 should now be available to configure as your WAN..." /usr/bin/logger -st "pfatt" "done!" else /usr/bin/logger -st "pfatt" "error: unknown EAP_MODE. '$EAP_MODE' is not valid. exiting..." exit 1 fi
- Set em0 as your wan, DHCP, mac spoof (RG of cert MAC address)
- Voila!
I think this works because ESXI will strip and add VLAN0 tags on the port group so no need netgraph business. I don't think this would work by plugging into my Cisco SG500x because I can't define VLAN0 and so the switch would just drop everything. Too bad! Let me know if anyone has any ideas to improve on things.
-
Would @GoldServe (or others) know how to work a similar scenario (without netgraph) with a physical switch (e.g. cisco), instead of an ESXi virtual switch (ONT --> Switch --> pfSense WAN)? Switch should do VLAN0 tagging via dot1p. Is that possible and what (affordable) switches could do that?
-
@bulldog5 sounds like you are looking for the supplicant branch:
https://github.com/aus/pfatt/tree/supplicant
Edit pfatt.sh to use EAP_MODE="supplicant" - that should create a simpler netgraph and call wpa_supplicant.
What netgear switch are you using and does it do outgoing VLAN0 tagging? -
@t41k2m3
I'm using a GS105Ev2 switch currently. This switch handles the VLAN0 fine, which is why the esxi method also works. But i'm running my pfsense on baremetal, so that option doesn't really apply to me. It would be "nice" to eliminate the GS105E in the middle of my ONT and pfsense WAN. I'll give the link you sent a shot and see how it goes. Thanks