ATT Uverse RG Bypass (0.2 BTC)

Makaveli6103

@gfeiner I think I know what was wrong. I have the Pace 5268AC gateway and I did add the script to help with the EAP-Logoff issue. But when i added the script to the /rc.d folder I forgot to add .sh. I added it and now see it running in the logs. Hopefully this was the issue.

gfeiner

@Makaveli6103 said in ATT Uverse RG Bypass (0.2 BTC):

@gfeiner I think I know what was wrong. I have the Pace 5268AC gateway and I did add the script to help with the EAP-Logoff issue. But when i added the script to the /rc.d folder I forgot to add .sh. I added it and now see it running in the logs. Hopefully this was the issue.

Good to know.

JonH

@gfeiner
I just got the ATT 1g service, internet only, no TV or VOIP. The RG is BGW210. I changed it's IP addr to 192.168.100.1 because as delivered it was the same as my pfSense box. Passthrough gives me a 5 minute lease although the setup screen is has a different lease time.

I have an SG-2440 and behind that an unmanaged 1g switch. My speedtests run around 550 Mbps, the RG Diagnostic menu has a speed test built in which shows that it is doing ~950Mbps.

I realize my setup is double NAT, I am reading here to find out if I can get rid of the double NAT and if that will increase my throughput.

I'm still seeking more info on the pfatt patch.

Makaveli6103

@JonH did you read the instructions on the GitHub? There is a little learninf curve but isn't too hard.

Derelict

Just a quick note that the etf kernel module is now available as a command-line-installable package from the Netgate repos.

[2.4.4-RELEASE][root@pfSense]/root: pkg search etf
ng_etf-kmod-0.1                ng_etf kernel module
[2.4.4-RELEASE][root@pfSense]/root: pkg install ng_etf-kmod
Updating pfSense-core repository catalogue...
pfSense-core repository is up to date.
Updating pfSense repository catalogue...
pfSense repository is up to date.
All repositories are up to date.
The following 1 package(s) will be affected (of 0 checked):

New packages to be INSTALLED:
	ng_etf-kmod: 0.1 [pfSense]

Number of packages to be installed: 1

3 KiB to be downloaded.

Proceed with this action? [y/N]:

No need to scp it from another FreeBSD node and it should track updates by FreeBSD.

JonH

@Derelict said in ATT Uverse RG Bypass (0.2 BTC):

etf kernel module is now available

Nice. Thanks for this info

JonH

@JonH I've installed pfatt 2 days ago, running w/o problems except my speed tests are still ~550 (~950 if wired directly as per AT&T). I'm not running Snort or Suricata. My cpu generally runs < 15%.

pfatt.sh contains (in addition to RG MAC addr:
ONT_IF='igb0'
RG_IF='igb3'

/usr/sbin/ngctl list
There are 13 total nodes:
Name: igb0 Type: ether ID: 00000001 Num hooks: 1
Name: <unnamed> Type: socket ID: 00000007 Num hooks: 0
Name: <unnamed> Type: socket ID: 0000006a Num hooks: 0
Name: <unnamed> Type: socket ID: 0000006b Num hooks: 0
Name: <unnamed> Type: socket ID: 0000006c Num hooks: 0
Name: <unnamed> Type: socket ID: 0000006d Num hooks: 0
Name: o2m Type: one2many ID: 0000000d Num hooks: 3
Name: vlan0 Type: vlan ID: 00000010 Num hooks: 2
Name: ngctl25207 Type: socket ID: 000000d3 Num hooks: 0
Name: ngeth0 Type: eiface ID: 00000013 Num hooks: 1
Name: waneapfilter Type: etf ID: 00000017 Num hooks: 2
Name: laneapfilter Type: etf ID: 0000001b Num hooks: 1
Name: igb3 Type: ether ID: 0000005d Num hooks: 0

One question is my interface assignments in the pfSense web configurator: The pfatt readme says "pfSense will detect new interfaces on bootup. Follow the prompts on the console to configure ngeth0 as your pfSense WAN."
In my case I didn't get any prompts so I read this to mean I should have ngeth0 as my WAN interface. Thus, I changed the WAN from igb0 to ngeth0 (and spoofing RG MAC). This leaves igb0 as "available".

Is this correct or am I misreading the readme? Should WAN remain igb0?

There was one comment earlier in this thread to make sure pfatt was being executed at <earlyshellcmd>. How would I determine that? And the etf filters have less hooks than an example posted earlier in this thread. Is that important?

Derelict

I would not edit the configuration to add the shell command. I would use the Shell Command package. There is an option there to select early.

JonH

@Derelict said in ATT Uverse RG Bypass (0.2 BTC):

I would use the Shell Command package.

Thank you. I was not aware of that package.
I'll give it a shot.

aus

re: which interface, your WAN should be ‘ngeth0’. If pfSense doesn’t prompt you to configure, you should manually set it.

re: performance, early shell cmd won’t improve that. Unfortunately, Netgraph configured as such does add a bit of CPU overhead at high network utilization. If your total CPU does not exceed ~15% under high network utilization, I would double check your single core performance. It may be maxed on a single core.

I’ve tested pfatt on a couple different boxes. Some performed better than others. My current CPU can mostly saturate (900+) my 1000/1000 plan:

AMD GX-420CA SOC
Current: 800 MHz, Max: 2000 MHz
4 CPUs: 1 package(s) x 4 core(s)
AES-NI CPU Crypto: Yes (active)

Supplicant mode has a little less overhead since the Netgraph is simpler. You might get more out of your hardware with that.

JonH

@aus: Thanks for feedback.

ngeth0 is on WAN. In the Interface Assignments menu that leaves igb0 down.
My CPU at ~15% is just average network usage. I don't run web servers. I have minimal streaming.
According to top, running in the shell, my largest cpu load is ntopng, I have disabled that and there is no noticeable improvement.

pfSense is running on a SG-2440 appliance (pre-Netgate appliance). It has 2 Atom C2358 1.7 GHz cpu's. I don't know how to check the individual cpu performance.
For crypto I think my setting is default, I don't recall setting it. It is set to BSD cryptodev but I will try no crypto to see if there is a noticeable difference.
I'm using a dumb switch.
I'm have a BGW210-700 & not using the AT&T wifi.
Is Supplicant Mode a function of compiling the etf.ko? If not, how do I remove it? I'm using Derelict's Build.

For kicks, I unplugged my LAN cable (igb1) and plugged a linux box directly into it (leaving a single NAS on igb2 & the RG on igb3). Same ~500 speedtest.net results. That linux box plugged into AT&T default setup is ~800-900.

You are at 4 cores, I'm at 2 cores. Maybe my throughput is the best I can expect with my SG-2440?

gfeiner

FYI. I'm doing this bypass on my netgate SG5100 and I can get in the 900-940Mb range with ATT UVERSE gigabit plan. So maybe it is your CPU.

JonH

@gfeiner The pfSense CPU? I'm starting to think that.

Derelict

Be sure powerd is enabled and set to Hiadaptive or Maximum in System > Advanced, Miscellaneous

Derelict

@JonH said in ATT Uverse RG Bypass (0.2 BTC):

I'm using Derelict's Build.

To be clear, it is not my build I'm just the messenger. The main developers at Netgate built it.

JonH

@Derelict Thank You, it (powerd)was previously set that way.
I'm going to disable pfBlockerNG to see if that is making a substantial hit on throughput.

JonH

@gfeiner said in ATT Uverse RG Bypass (0.2 BTC):

So maybe it is your CPU

Link below show results of shell command 'systat load' while doing speed test. If I understand the output correctly it looks like my CPU are doing ok.
![ScreenShot](<a href="https://imgur.com/oW4yqgC"><img src="https://i.imgur.com/oW4yqgC.png" title="source: imgur.com" /></a>)

I also did a speed test with pfBlockerNG disabled and there was negligible improvement.

bulldog5

Has anyone tried using this netgraph method along with the certificate extraction from gateway method? I have the wpa_supplicant method working, but still have to use the 5port netgear switch in the middle of my ONT and PFsense WAN because of VLAN0. Wondering how i could use netgraph to deal with VLAN 0 issue.

GoldServe

So I got things working by not using any netgraph scripts on my ESXi 6.7u2 virtualized pfSense instance. If you follow the instructions below, you should get things working.

Set up a new VSWITCH, port group with VLAN(0) and uplink on a dedicated network uplink (Allow mac address spoofing and the other two just incase)
Connect the ONT to this uplink
Create a new e1000e interface that resides in the port group from 1) in pFsense (em0 for me). I tried vmxnet3 and didn't seem to work
I just took the portion of the script below to start wpa_supplicant. Find all em0 below and change with your adapter.

  /usr/bin/logger -st "pfatt" "starting wpa_supplicant..."

  WPA_PARAMS="\
    set eapol_version 2,\
    set fast_reauth 1,\
    ap_scan 0,\
    add_network,\
    set_network 0 ca_cert \\\"/conf/pfatt/wpa/ca.pem\\\",\
    set_network 0 client_cert \\\"/conf/pfatt/wpa/client.pem\\\",\
    set_network 0 eap TLS,\
    set_network 0 eapol_flags 0,\
    set_network 0 identity \\\"$EAP_SUPPLICANT_IDENTITY\\\",\
    set_network 0 key_mgmt IEEE8021X,\
    set_network 0 phase1 \\\"allow_canned_success=1\\\",\
    set_network 0 private_key \\\"/conf/pfatt/wpa/private.pem\\\",\
    enable_network 0\
  "

  WPA_DAEMON_CMD="/usr/sbin/wpa_supplicant -Dwired -iem0 -B -C /var/run/wpa_supplicant"

  # kill any existing wpa_supplicant process
  PID=$(pgrep -f "wpa_supplicant.*em0")
  if [ ${PID} > 0 ];
  then
    /usr/bin/logger -st "pfatt" "terminating existing wpa_supplicant on PID ${PID}..."
    RES=$(kill ${PID})
  fi

  # start wpa_supplicant daemon
  RES=$(${WPA_DAEMON_CMD})
  PID=$(pgrep -f "wpa_supplicant.*em0")
  /usr/bin/logger -st "pfatt" "wpa_supplicant running on PID ${PID}..."

  # Set WPA configuration parameters.
  /usr/bin/logger -st "pfatt" "setting wpa_supplicant network configuration..."
  IFS=","
  for STR in ${WPA_PARAMS};
  do
    STR="$(echo -e "${STR}" | sed -e 's/^[[:space:]]*//')"
    RES=$(eval wpa_cli ${STR})
  done

  # wait until wpa_cli has authenticated.
  WPA_STATUS_CMD="wpa_cli status | grep 'suppPortStatus' | cut -d= -f2"

  /usr/bin/logger -st "pfatt" "waiting EAP for authorization..."

  # TODO: blocking for bootup
  while true;
  do
    WPA_STATUS=$(eval ${WPA_STATUS_CMD})
    if [ X${WPA_STATUS} = X"Authorized" ];
    then
      /usr/bin/logger -st "pfatt" "EAP authorization completed..."
      break
    else
      sleep 1
    fi
  done
  /usr/bin/logger -st "pfatt" "em0 should now be available to configure as your WAN..."
  /usr/bin/logger -st "pfatt" "done!"
else
  /usr/bin/logger -st "pfatt" "error: unknown EAP_MODE. '$EAP_MODE' is not valid. exiting..."
  exit 1
fi

Set em0 as your wan, DHCP, mac spoof (RG of cert MAC address)
Voila!

I think this works because ESXI will strip and add VLAN0 tags on the port group so no need netgraph business. I don't think this would work by plugging into my Cisco SG500x because I can't define VLAN0 and so the switch would just drop everything. Too bad! Let me know if anyone has any ideas to improve on things.

t41k2m3

Would @GoldServe (or others) know how to work a similar scenario (without netgraph) with a physical switch (e.g. cisco), instead of an ESXi virtual switch (ONT --> Switch --> pfSense WAN)? Switch should do VLAN0 tagging via dot1p. Is that possible and what (affordable) switches could do that?