ATT Uverse RG Bypass (0.2 BTC)

dc81

Goal is for my traffic to bypass the RG. I have a pfsense box with multiple gigabit ports. I want to connect the ONT and RG to the pfsense box and have LAN traffic on the remaining ports.

Currently:

ONT > RG > pfSsense > LAN

Want:


ONT > pfSense
      |    |
      RG   LAN

So I'm not really concerned with any specific option, but I would want to be able to keep my gigabit speed. My box has an Atom C2558 if that matters.

zevlag

If I provide a kernel patch, are you familiar with compiling kernels in FreeBSD?

dc81

I have not, but it seems fairly straightforward. I have a backup pfsense in a vm so the downtime is not an issue.

dc81

any progress?

dc81

bump… anyone?

ermax

I would love to see this too.

pyrodex

Looks like it maybe more viable now with a python script:

https://github.com/jaysoffian/eap_proxy

See https://github.com/jaysoffian/eap_proxy/issues/2 also since it seems PFRING is required but BSD doesn't have PFRING.

I've heard someone got this working on a debian box running shorewall so maybe just adapting it for BSD could be the solution.

variance

@pyrodex:

Looks like it maybe more viable now with a python script:

https://github.com/jaysoffian/eap_proxy

See https://github.com/jaysoffian/eap_proxy/issues/2 also since it seems PFRING is required but BSD doesn't have PFRING.

I've heard someone got this working on a debian box running shorewall so maybe just adapting it for BSD could be the solution.

I've seen a report of this solution (https://github.com/kuwerty/eapolproxy)
eap proxy sucessfully compiled and running on pfsense after installing some dependencies (libstdc++).
and it passing packets.

I checked out the eapolproxy, and successfully compiled it on my freebsd dev box. After getting some dependencies (libstdc++) onto pfsense, it does start and appears to be passing the EAP traffic from the RG on OPT1 up to the WAN interface where the ONT is, but nothing ever comes back - it just keeps spamming the EAPOL start and logoffs. I will have to keep playing with it, but I feel like the solution is close. It would be good to have some others try this…

Source: (http://www.dslreports.com/forum/r31632582-)

as for pfring dependency.. i don't know if thats kernel or something that needs to be compiled specifically for freebsd…
my gigapower install doesn't happen for another day or 2. I'm going to go ahead and try compiling the this eapolproxy myself to see if I can get it working.

rajl

If this feature bounty still open? I may have an elegant FreeBSD solution to the problem.

random003

I'll give you $100 in bitcoin for a set-and-forget solution that works on pfsense.

PnoT

Any progress on this?

random003

$200 for a set-and-forget solution.

I may consider other payments methods besides bitcoin if preferred.

rajl

There's been a surprising amount of progress, although I've encountered some issues with PFSense specifically (vanilla FreeBSD and OPNSense work more smoothly).

Here's a summary so far of what I've done, and the issues I am encountering.

For background, here are links for Linux specific solutions:

http://blog.0xpebbles.org/Bypassing-At-t-U-verse-hardware-NAT-table-limits
https://strscrm.io/bypassing-gigapowers-provided-modem.html

To summarize, there are two Linux approaches:
(1a) Create a Bridge and enable the Bridge to Forward EAPOL traffic - Use ebtables (Linux layer 2 firewall) to filter what is forwarded across the bridge.
(1b) Run a Python script called EAP-Proxy that proxies EAPOL traffic in order to avoid creating the bridge
(2) Create a Virtual Interface for your WAN assigned VLAN 0 and assign your services to it.

However, FreeBSD is not Linux and there are some issues due to FreeBSD's undying commitment to standard's compliance:
(1) The FreeBSD bridge code does not allow for EAPOL traffic to be forwarded across the bridge because this violates the IEEE 802.1D standard.
(2) The EAP-Proxy Script that is circulating is pretty specific to Linux and requires a lot of massaging to work with FreeBSD. Also, it requires installing python on your firewall, which you may not want to do for security reasons.
(3) FreeBSD's vlan code does not allow you to create a virtual interface assigned to VLAN 0 because IEEE 802.1Q specifies that vlan 0 is reserved.

So there are two technical issues that need to be resolved:
(1) Make FreeBSD forward EAPOL traffic between the RG and the ONT.
(2) Allow the FreeBSD WAN interface to communicate on vlan 0 so that it can communicate with the ONT.

I have solved the first problem and can demonstrate two working solutions on FreeBSD and OPNSense. However, the preferred solution does not work on PFSense yet. To briefly summarize my work:
(1) The universally working (but not preferred) solution is to patch the kernel's bridge code to forward EAPOL traffic. I've created a kernel patch for the bridge code that allows you to enable or disable forwarding of EAPOL traffic on a bridge using a sysctl variable. However, it's definitely not a set and forget solution as you have to manually patch your kernel and then potentially repatch the kernel with every update. Long-term, it can be a set and forget solution once I submit the patch to -CURRENT and it migrates to -STABLE and it then makes it to -RELEASE. But that's a long migration path.
(2) I've created a set and forget solution for forwarding EAPOL traffic using FreeBSD's builtin Netgraph framework. Basically, I use Netgraph to create a layer 2 firewall that forwards all EAPOL frames between interfaces and forwards all other Ethernet frames on up the network stack per normal. I've tested it successfully on vanilla FreeBSD and OPNSense. However, it's not currently working on PFSense for two reasons.

The first reason is a show stopper - there's a bug in the PFSense netgraph module ng_eth. When ng_eth is loaded, it is supposed to automatically create a Netgraph node for each ethernet interface on your system. Unfortunately, PFSense's ng_eth module only creates a netgraph node for some, but not all of the ethernet interfaces on my test system. It also deletes nodes it has created when I try to switch the LAN and WAN interfaces to the existing ng_eth nodes. Curiously, the ng_eth module works on FreeBSD 11.1 and OPNSense as expected.
The second reason is rather trivial to solve, but still annoying. PFSense doesn't load all of the necessary netgraph modules by default and doesn't allow you to load kernel modules at run time. While this is a good security behavior, it means you have to manually edit your boot configuration to load the necessary netgraph modules

I am still attempting to solve the second problem (communicating on VLAN 0). Looking at TCPDump, I can see the DHCP request and response between my WAN and the ONT, but I am not getting an IP address assigned to my WAN interface. I suspect this is because the traffic is tagged as VLAN 0 in a non-standard compliant manner, which is causing FreeBSD to disregard the DHCP response as invalid. I may be able to solve this problem as well using Netgraph, but I need to investigate further when I have some more time and I can test my solution (which is when my family is not home streaming Netflix).

If anyone else is a networking ninja, I would be glad to partner with them to finish this project more quickly.

random003

Do you have a bitcoin address? I'd like to give you $100 worth right now to keep the progress going. I don't mind a kernel patch.

Ryu945

I see that ATT uses IP-DSL to do the log in. I wonder if there is a way to get Pfsense to do the log in directly.

rajl

@random003:

Do you have a bitcoin address? I'd like to give you $100 worth right now to keep the progress going. I don't mind a kernel patch.

I'm doing this for the love and the technical challenge, but tips are always welcome! ;)

My bitcoin wallet address for any donations towards this project (from you or any other generous souls) is:

1H8CaLNXembfzYGDNq1NykWU3gaKAjm8K5

If anyone is going to tip me for my efforts, I figure I should at least give them something in return.

Here’s the Netgraph based solution I have so far (which I have tested more thoroughly than the kernel patch):

Step 1: Copy the ng_etf.ko module from a FreeBSD 11.1 system to /boot/kernel. If you are security conscious, you will copy this module yourself from a FreeBSD 11.1 system you already own/control. For convenience, I have uploaded a copy of the module here which trusting souls may download at their convenience.

Step 2: Add the following line to your /boot/loader.conf file – ng_etf_load=”YES” (include quotation marks)
Step 3: Reboot so that the kernel modules are loaded
Step 4: Clone the MAC address of the RG to your “WAN” port.
Step 5: Use the following Netgraph commands to create the EAP Netgraph Bridge. As of right now you need to enter these commands at the console or include them in a startup/boot script. Because you are disconnecting and reconnecting your Ethernet interfaces to create the necessary graph, you will lock yourself out of your own box if you are doing this over SSH. Hence, you need physical access to your machine when entering these commands or they need to be executed automatically on boot.


## Replace “em0” and “em1” with your WAN and ONT Ethernet interfaces
    ## as appropriate for your machine

    ngctl mkpeer em0: etf lower downstream
    ngctl name em0:lower waneapfilter
    ngctl connect waneapfilter: em0: nomatch upper

    ngctl mkpeer em1: etf lower downstream
    ngctl name em1:lower laneapfilter
    ngctl connect laneapfilter: em1: nomatch upper

    ngctl connect waneapfilter: laneapfilter: eapout eapout

    ngctl msg waneapfilter: 'setfilter { matchhook="eapout" ethertype=0x888e }'
    ngctl msg laneapfilter: 'setfilter { matchhook="eapout" ethertype=0x888e }'

If these commands throw a cryptic error message, one of three things happened.

Most likely, you omitted or added a colon (“:”) somewhere it didn’t belong (I speak from experience).
Less likely, but possible, is that I accidentally omitted or added a colon while copying and pasting these commands into this writeup. :)
Possibly, PFSense’s ng_eth module is not recognizing one or more of your interfaces. This happens on some, but not all computers. I’m not sure if this is a software or hardware bug. For example, on my ZOTAC CI323, all of my interfaces are recognized. However, on my QOTOM machine, only 2 of the 4 are recognized (OPNSense and vanilla FreeBSD recognize all four, which leads me to believe there is a subtle PFSense bug rather than an hardware issue).

Step 6: Connect the ONT to your PFSense Box and the RG to your PFSense Box (connecting from PFSense to the ONT port on the RG)

Step 7: Power cycle the RG in order to force authentication with ATT

Step 8: Confirm authentication. After 1-2 minutes, you will see the “Broadband” light on your RG flash green and then go to solid green for a short period of time. This means that the 802.1X port authentication has completed successfully. However, your Broadband light will then start flashing read and then go blank. This is because the RG is not receiving an IP address from the ATT network via DHCP (your PFSense Box is attempting request and receive the IP address).

At this point, I can see the DCHP requests and responses between the PFSense box and the ATT network using tcpdump. However, the PFSense box is currently unable to use the IP address provided by ATT. I assume this is because ATT is tagging all responses as being on VLAN 0 (you can see this in TCPDump). With Linux based solutions, you can solve the problem by assigning vlan 0 to your WAN and then moving all your services over to the virtual interface created for vlan0. However, FreeBSD doesn't handle frames explicitly tagged as vlan 0 very well.

I submitted a bug report to FreeBSD for to see if I could get this resolved. The short answer is "no" but they suggested that the Netgraph VLAN code might be able to convert frames tagged as vlan 0 into untagged frames prior to forwarding them up the network stack. I have not had time to investigate this, but it seems promising. If it works, I'll update my netgraph script above to incorporate the appropriate ng_vlan nodes.

EDIT - UPDATED Link for Kernel Module

rajl

@Ryu945:

I see that ATT uses IP-DSL to do the log in. I wonder if there is a way to get Pfsense to do the log in directly.

Unfortunately, no. ATT uses cryptographic certificates installed in the ROM of the RG to authenticate the RG with the ATT network. This allows them to prevent "unauthorized" equipment from being attached to the network. Unless you feel like dumping the contents of the ROM, identifying the cryptographic certificates, uploading them to your PFSense box, and creating a custom authorization script, it's easier to just perform a man-in-the-middle attack on the 802.1X authentication.

random003

Thanks for your work on this.

https://tradeblock.com/bitcoin/tx/5be26573726e21c9f70d18af1223fb8e307cae6194a656ca294cc4afa99ae767

rajl

@random003:

Thanks for your work on this.

https://tradeblock.com/bitcoin/tx/5be26573726e21c9f70d18af1223fb8e307cae6194a656ca294cc4afa99ae767

Received. Thanks!

Digging into this a little more, the VLAN netgraph node may provide the "missing link." I'm hoping to be able to test it in the reasonably near future and see if it actually works.

aus

Hi rajl.

For the past few weeks, I've independently gone down similar avenues to explore a solution to this problem. I considered porting the EAP Proxy script, but that seemed painful. I opted to patch the kernal (if_bridge.c to be specific) to remove the drop on the 802.1X MAC 01:80:c2:00:00:03. That worked, but only for the EAP problem.

After discovering this post (a great start btw!) and reading more about netgraph, I agree that is probably the best approach. I hoping we can get this working using a combination of: ng_ether, ng_etf, ng_vlan, ng_tee and ng_eiface

For the VLAN0 problem, I expect the netgraph to look like something like this:


# em0 - ATT RG
# em1 - ONT
# em2 - LAN
# ngeth0 - "WAN" netgraph creates interface, removes VLAN0 tag from ONT traffic

# make eth devices addressable in netgraph
# (kernel module may already be loaded for you)
kldload ng_ether     

# from em1, create a vlan peer
# connect em1's lower hook to vlan's downstream hook
ngctl mkpeer em1: vlan lower downstream

# name peer vlan
ngctl name em1:lower vlan

# connect em1's upper hook to vlan's nomatch hook
ngctl connect em1: vlan: upper nomatch

# from vlan, create eiface peer (ngeth0)
# connect vlan's untagged hook to eiface's ether hook
ngctl mkpeer vlan: eiface untagged ether

# instruct vlan: to send vlan0 traffic to untagged hook
# which gets sent to the eiface ether hook (ngeth0)
ngctl msg vlan: addfilter '{ vlan=0 hook="untagged" }'

I've tested in locally in a VM and I think this part is working. However, the problem I'm struggling with now is combining the EAP netgraph solution with the VLAN netgraph solution. I think this is where ng_tee comes into play, but I'm still trying to wrap my head around it.

I think we need to use ng_tee to split out the ng_ether-em1 interface. Then hook up the EAP graph to one side and the VLAN graph to the other. But my head spins trying to keep left, right, right2left and left2right straight. :) The lacking documentation about netgraph doesnt help either. It seems no one talks about netgraph much.

Have you had any progress or success?