ATT Uverse RG Bypass (0.2 BTC)

t41k2m3

@aus et al - just a quick update, tried trimming down the graph (to just ngeh0, vlan0, ONT_IF, RG_IF) and am seeing same issues:

Some speed degradation (though much more manageable relative to other examples here - 100-200 Mbps loss on Gig WAN);
Connection drops 1-2 times per 24-hour period, which did not use to happen with quite the same frequency when connected via ATT RG in passthrough mode. When it drops, WAN/ngeth0 keeps public IP, however gateway becomes unreachable which triggers disconnect/reconnect bringing whole network down). Sometimes (not always) when disconnects happen, they are preceded by an error like "arpresolve: can't allocate llinfo for <GW IP> on ngeth0". Unable so far to pinpoint a root cause or possible fix (tried suggestions from @aus ). May try disabling gateway monitoring or action, though not sure that's ideal long term in case of more serious connectivity disruptions.

Anyone else experience this?

pyrodex

I just got a new Xeon E3-1230v6 thanks to work and replaced my C2758 firewall last night. Right now I am using IP-Passthrough but will swap back to the method in this thread and see if the performance is still an issue.

If I can't get good performance with a semi modern xeon then I am afraid it is just never going to happen.

aus

@t41k2m3 said in ATT Uverse RG Bypass (0.2 BTC):

@aus et al - just a quick update, tried trimming down the graph (to just ngeh0, vlan0, ONT_IF, RG_IF) and am seeing same issues:

Some speed degradation (though much more manageable relative to other examples here - 100-200 Mbps loss on Gig WAN);
Connection drops 1-2 times per 24-hour period, which did not use to happen with quite the same frequency when connected via ATT RG in passthrough mode. When it drops, WAN/ngeth0 keeps public IP, however gateway becomes unreachable which triggers disconnect/reconnect bringing whole network down). Sometimes (not always) when disconnects happen, they are preceded by an error like "arpresolve: can't allocate llinfo for <GW IP> on ngeth0". Unable so far to pinpoint a root cause or possible fix (tried suggestions from @aus ). May try disabling gateway monitoring or action, though not sure that's ideal long term in case of more serious connectivity disruptions.

Anyone else experience this?

That's too bad the netgraph alterations didn't work. At least, we have ruled out that most of the netgraph does not cause speed degradation for you. The "stripped" netgraph is pretty simple, leaving pretty much only ng_vlan (for VLAN0 tagging) and ng_eiface (for creating a NIC). Neither would be very CPU intensive, in theory.

Regarding your connection drops, the best way to debug this is to catch the problem with tcpdumps running on $RG_IF and $ONT_IF. Maybe you could filter to just EAP and DHCP traffic to reduce load if you can't reproduce. It kind of sounds like something is confusing your DHCP lease on your WAN.

@pyrodex said in ATT Uverse RG Bypass (0.2 BTC):

I just got a new Xeon E3-1230v6 thanks to work and replaced my C2758 firewall last night. Right now I am using IP-Passthrough but will swap back to the method in this thread and see if the performance is still an issue.
If I can't get good performance with a semi modern xeon then I am afraid it is just never going to happen.

Interested to see how the Xeon shakes out for you.

pyrodex

@aus looks like I was able to pull nearly full line speed with the bridge on a E3-1230V6, I saw ~112+ MB/s when pulling a public linux distro torrent.

               /0   /1   /2   /3   /4   /5   /6   /7   /8   /9   /10
     Load Average   ||

      Interface           Traffic               Peak                Total
         ngeth0  in     30.936 KB/s        114.354 MB/s           27.332 GB
                 out   467.484 KB/s          1.479 MB/s          979.165 MB

           igb3  in      0.000 KB/s          0.082 KB/s           14.170 KB
                 out     0.000 KB/s          0.000 KB/s            4.050 KB

           igb0  in     33.320 KB/s        114.846 MB/s           27.479 GB
                 out   472.396 KB/s          1.619 MB/s          999.016 MB

aus

Thanks for the update! Glad you were able sort the speed issues out.

MisterBaz

I wanted to come here and post a giant "Thank You" for all the work aus and others helping him put forth.

I was finally able to get my SuperMicro C2758 pfsense box working directly with the fiber ONT.

I have 4 Gig ports configured as follows:
igb0 - AT&T Fiber ONT
igb1 - L3 Switch
igb2 - UVERSE DVR - VIP2250
igb3 - RG - BGW210

One thing I've noticed, is the RG only ever has a single GREEN LED lit. I've powered cycled the RG and the fiber ONT and everything still worked, so.... //shrug//

My throughput is fantastic (I have Gig service) and my latency dropped ever so slightly as well. Now, my task is getting the DVR to work. Coincidentally, my old unit died, so I just received my replacement.
One thing I noticed, is the UVERSE LAN needs the AT&T DNS Servers in order to work. I can't filter them through Cloudfare, or other.
I had IGMP Proxy setup just fine with the old box, but now it doesn't seem to be working. Any thoughts?

Derelict

@misterbaz said in ATT Uverse RG Bypass (0.2 BTC):

I had IGMP Proxy setup just fine with the old box, but now it doesn't seem to be working. Any thoughts?

You should probably start another thread.

aus

@misterbaz said in ATT Uverse RG Bypass (0.2 BTC):

I wanted to come here and post a giant "Thank You" for all the work aus and others helping him put forth.
Glad to help out!

One thing I've noticed, is the RG only ever has a single GREEN LED lit. I've powered cycled the RG and the fiber ONT and everything still worked, so.... //shrug//

This is normal and expected. The RG never reaches full green status because it is expecting to negotiate a DHCP lease. However, netgraph drops that traffic because pfSense is handling the DHCP. You can actually keep the RG disconnected after the 802.1X EAP-TLS authentication completes. However, if your igb0 looses its link (due to power outage, unplug, reboot, or whatever), you will loose connectivity until the RG is reconnected and can authenticate you.

One thing I noticed, is the UVERSE LAN needs the AT&T DNS Servers in order to work. I can't filter them through Cloudfare, or other.

This might be true for set top boxes or DVRs, but your entire LAN does not need to use AT&T DNS servers.

I had IGMP Proxy setup just fine with the old box, but now it doesn't seem to be working. Any thoughts?

There's been a few threads about configuring the IGMP proxy for AT&T. Basically, it involved adding some of AT&Ts IP ranges. I had it working a while ago, but no longer have TV service to test. You might continue the conversation here:

https://github.com/aus/pfatt/issues/3

Definitely accepting PRs if you figure it out.

MisterBaz

@aus said in ATT Uverse RG Bypass (0.2 BTC):

@misterbaz said in ATT Uverse RG Bypass (0.2 BTC):

I wanted to come here and post a giant "Thank You" for all the work aus and others helping him put forth.
Glad to help out!

One thing I've noticed, is the RG only ever has a single GREEN LED lit. I've powered cycled the RG and the fiber ONT and everything still worked, so.... //shrug//

This is normal and expected. The RG never reaches full green status because it is expecting to negotiate a DHCP lease. However, netgraph drops that traffic because pfSense is handling the DHCP. You can actually keep the RG disconnected after the 802.1X EAP-TLS authentication completes. However, if your igb0 looses its link (due to power outage, unplug, reboot, or whatever), you will loose connectivity until the RG is reconnected and can authenticate you.

One thing I noticed, is the UVERSE LAN needs the AT&T DNS Servers in order to work. I can't filter them through Cloudfare, or other.

This might be true for set top boxes or DVRs, but your entire LAN does not need to use AT&T DNS servers.

Correct. The rest of my LAN has Cloudfare DNS servers assigned. The UVERSE DVR is the only thing that needed to see the AT&T DNS Servers.

I had IGMP Proxy setup just fine with the old box, but now it doesn't seem to be working. Any thoughts?

There's been a few threads about configuring the IGMP proxy for AT&T. Basically, it involved adding some of AT&Ts IP ranges. I had it working a while ago, but no longer have TV service to test. You might continue the conversation here:

https://github.com/aus/pfatt/issues/3

Definitely accepting PRs if you figure it out.

I figured it out, sort of. I had been trying to nail down every multicast server I could see through pfTop, but it was still hanging up on a lot of channels. So, I instead made a blanket 0.0.0.0/1 (pfSense won't let you use /0) statement in the Upstream setting and every channel came through. This might seem terrible, but remember my UVERSE DVR is on its own separate LAN independent from my normal LAN. Also, this still won't work until you setup an allow rule through your firewall. I believe I only had to setup an alow rule to my UVERSE LAN for 224.0.0.0/8. There might have also been an allow rule for 239.0.0.0/8. I'll have to check it out when I get back home.

Dade

Just wanted to chime in and say that this worked great for me. I also have some static IPs from ATT (10$/mo for 5 extra) and I was able to utilize them no issues without the gateway.. Not sure if it was mentioned how to before.

I did so by creating an another lan interface tagged to vlan 99 dubbed Public Vlan. I assigned the "Gateway IP" given to me by ATT to this interface. after some proper firewall rules to allow traffic from outside to this network and disabling NAT on those addresses. I can confirm the public ip subnet is able to get out and traffic does return as desired. If you need more information just PM me

random003

@aus Thanks for your work on this. https://btc.com/cae38d113459909c8b23dc20553628cdc79a2150ed87c9724701fcb95f67814c

aus

Thank you! That’s very generous of you and much appreciated. I’m glad this solution worked for you. Cheers!

untamedgorilla

Thank you @aus. I'm actually using the supplicant version and it works like a charm!!! Great work. Now I have to figure your static ips, @Dade, I looking at you!

untamedgorilla

@Dade I figured out the static IP, I just setup 1:1 nat, it was pretty straightforward.

Makaveli6103

I set this up a month ago and worked great for 2 week so. Then for the past 2 weeks my connection will drop randomly every day or few days. My logs are linked below. But another question I have is the link to the gateway suppose to work also? When I connect to it it says there is no link and to contact ATT but my PFsense internet works.

Logs

gfeiner

@Makaveli6103 said in ATT Uverse RG Bypass (0.2 BTC):

I set this up a month ago and worked great for 2 week so. Then for the past 2 weeks my connection will drop randomly every day or few days. My logs are linked below. But another question I have is the link to the gateway suppose to work also? When I connect to it it says there is no link and to contact ATT but my PFsense internet works.

Logs

I believe @aus stated that (the gateway not having link) is expected since all traffic from the gateway other than authentication is blocked.

gfeiner

Has anyone here using this bypass method noticed the WAN lease time obtained from ATT DHCP is only 1 hour long? Apparently people using other bypass methods have noticed it as well (see here) It only seems to happen when bypassing the gateway. The lease obtained when using ip-passthrough mode is much longer. I can see that in /var/db/dhclient.leases.ngeth0 the leases are 3600 seconds and going a grep "renewal in" /var/log/dhcpd.log shows the lease renewing every 1800 seconds which is the 1/2way point of a 3600 second lease. I'm curious if anyone can come up with a way to increase that lease time. I tried using a "send" dhcp-lease-time option to increase it, but it didn't help; the lease obtained was still 3600 seconds.

Makaveli6103

@gfeiner ok thanks. But any idea why my connection drops?

gfeiner

@Makaveli6103 said in ATT Uverse RG Bypass (0.2 BTC):

@gfeiner ok thanks. But any idea why my connection drops?

What is connected to igb2? The ONT or the gateway? Your logs are showing link going up and down on that port. Bad cable or faulty device connected to that port.

Makaveli6103

@gfeiner igb2 is the gateway. I will change the cable. I did also turn of gateway monitoring to see if that does anything.