ATT Uverse RG Bypass (0.2 BTC)

JonH

@JonH I've installed pfatt 2 days ago, running w/o problems except my speed tests are still ~550 (~950 if wired directly as per AT&T). I'm not running Snort or Suricata. My cpu generally runs < 15%.

pfatt.sh contains (in addition to RG MAC addr:
ONT_IF='igb0'
RG_IF='igb3'

/usr/sbin/ngctl list
There are 13 total nodes:
Name: igb0 Type: ether ID: 00000001 Num hooks: 1
Name: <unnamed> Type: socket ID: 00000007 Num hooks: 0
Name: <unnamed> Type: socket ID: 0000006a Num hooks: 0
Name: <unnamed> Type: socket ID: 0000006b Num hooks: 0
Name: <unnamed> Type: socket ID: 0000006c Num hooks: 0
Name: <unnamed> Type: socket ID: 0000006d Num hooks: 0
Name: o2m Type: one2many ID: 0000000d Num hooks: 3
Name: vlan0 Type: vlan ID: 00000010 Num hooks: 2
Name: ngctl25207 Type: socket ID: 000000d3 Num hooks: 0
Name: ngeth0 Type: eiface ID: 00000013 Num hooks: 1
Name: waneapfilter Type: etf ID: 00000017 Num hooks: 2
Name: laneapfilter Type: etf ID: 0000001b Num hooks: 1
Name: igb3 Type: ether ID: 0000005d Num hooks: 0

One question is my interface assignments in the pfSense web configurator: The pfatt readme says "pfSense will detect new interfaces on bootup. Follow the prompts on the console to configure ngeth0 as your pfSense WAN."
In my case I didn't get any prompts so I read this to mean I should have ngeth0 as my WAN interface. Thus, I changed the WAN from igb0 to ngeth0 (and spoofing RG MAC). This leaves igb0 as "available".

Is this correct or am I misreading the readme? Should WAN remain igb0?

There was one comment earlier in this thread to make sure pfatt was being executed at <earlyshellcmd>. How would I determine that? And the etf filters have less hooks than an example posted earlier in this thread. Is that important?

Derelict

I would not edit the configuration to add the shell command. I would use the Shell Command package. There is an option there to select early.

JonH

@Derelict said in ATT Uverse RG Bypass (0.2 BTC):

I would use the Shell Command package.

Thank you. I was not aware of that package.
I'll give it a shot.

aus

re: which interface, your WAN should be ‘ngeth0’. If pfSense doesn’t prompt you to configure, you should manually set it.

re: performance, early shell cmd won’t improve that. Unfortunately, Netgraph configured as such does add a bit of CPU overhead at high network utilization. If your total CPU does not exceed ~15% under high network utilization, I would double check your single core performance. It may be maxed on a single core.

I’ve tested pfatt on a couple different boxes. Some performed better than others. My current CPU can mostly saturate (900+) my 1000/1000 plan:

AMD GX-420CA SOC
Current: 800 MHz, Max: 2000 MHz
4 CPUs: 1 package(s) x 4 core(s)
AES-NI CPU Crypto: Yes (active)

Supplicant mode has a little less overhead since the Netgraph is simpler. You might get more out of your hardware with that.

JonH

@aus: Thanks for feedback.

ngeth0 is on WAN. In the Interface Assignments menu that leaves igb0 down.
My CPU at ~15% is just average network usage. I don't run web servers. I have minimal streaming.
According to top, running in the shell, my largest cpu load is ntopng, I have disabled that and there is no noticeable improvement.

pfSense is running on a SG-2440 appliance (pre-Netgate appliance). It has 2 Atom C2358 1.7 GHz cpu's. I don't know how to check the individual cpu performance.
For crypto I think my setting is default, I don't recall setting it. It is set to BSD cryptodev but I will try no crypto to see if there is a noticeable difference.
I'm using a dumb switch.
I'm have a BGW210-700 & not using the AT&T wifi.
Is Supplicant Mode a function of compiling the etf.ko? If not, how do I remove it? I'm using Derelict's Build.

For kicks, I unplugged my LAN cable (igb1) and plugged a linux box directly into it (leaving a single NAS on igb2 & the RG on igb3). Same ~500 speedtest.net results. That linux box plugged into AT&T default setup is ~800-900.

You are at 4 cores, I'm at 2 cores. Maybe my throughput is the best I can expect with my SG-2440?

gfeiner

FYI. I'm doing this bypass on my netgate SG5100 and I can get in the 900-940Mb range with ATT UVERSE gigabit plan. So maybe it is your CPU.

JonH

@gfeiner The pfSense CPU? I'm starting to think that.

Derelict

Be sure powerd is enabled and set to Hiadaptive or Maximum in System > Advanced, Miscellaneous

Derelict

@JonH said in ATT Uverse RG Bypass (0.2 BTC):

I'm using Derelict's Build.

To be clear, it is not my build I'm just the messenger. The main developers at Netgate built it.

JonH

@Derelict Thank You, it (powerd)was previously set that way.
I'm going to disable pfBlockerNG to see if that is making a substantial hit on throughput.

JonH

@gfeiner said in ATT Uverse RG Bypass (0.2 BTC):

So maybe it is your CPU

Link below show results of shell command 'systat load' while doing speed test. If I understand the output correctly it looks like my CPU are doing ok.
![ScreenShot](<a href="https://imgur.com/oW4yqgC"><img src="https://i.imgur.com/oW4yqgC.png" title="source: imgur.com" /></a>)

I also did a speed test with pfBlockerNG disabled and there was negligible improvement.

bulldog5

Has anyone tried using this netgraph method along with the certificate extraction from gateway method? I have the wpa_supplicant method working, but still have to use the 5port netgear switch in the middle of my ONT and PFsense WAN because of VLAN0. Wondering how i could use netgraph to deal with VLAN 0 issue.

GoldServe

So I got things working by not using any netgraph scripts on my ESXi 6.7u2 virtualized pfSense instance. If you follow the instructions below, you should get things working.

Set up a new VSWITCH, port group with VLAN(0) and uplink on a dedicated network uplink (Allow mac address spoofing and the other two just incase)
Connect the ONT to this uplink
Create a new e1000e interface that resides in the port group from 1) in pFsense (em0 for me). I tried vmxnet3 and didn't seem to work
I just took the portion of the script below to start wpa_supplicant. Find all em0 below and change with your adapter.

  /usr/bin/logger -st "pfatt" "starting wpa_supplicant..."

  WPA_PARAMS="\
    set eapol_version 2,\
    set fast_reauth 1,\
    ap_scan 0,\
    add_network,\
    set_network 0 ca_cert \\\"/conf/pfatt/wpa/ca.pem\\\",\
    set_network 0 client_cert \\\"/conf/pfatt/wpa/client.pem\\\",\
    set_network 0 eap TLS,\
    set_network 0 eapol_flags 0,\
    set_network 0 identity \\\"$EAP_SUPPLICANT_IDENTITY\\\",\
    set_network 0 key_mgmt IEEE8021X,\
    set_network 0 phase1 \\\"allow_canned_success=1\\\",\
    set_network 0 private_key \\\"/conf/pfatt/wpa/private.pem\\\",\
    enable_network 0\
  "

  WPA_DAEMON_CMD="/usr/sbin/wpa_supplicant -Dwired -iem0 -B -C /var/run/wpa_supplicant"

  # kill any existing wpa_supplicant process
  PID=$(pgrep -f "wpa_supplicant.*em0")
  if [ ${PID} > 0 ];
  then
    /usr/bin/logger -st "pfatt" "terminating existing wpa_supplicant on PID ${PID}..."
    RES=$(kill ${PID})
  fi

  # start wpa_supplicant daemon
  RES=$(${WPA_DAEMON_CMD})
  PID=$(pgrep -f "wpa_supplicant.*em0")
  /usr/bin/logger -st "pfatt" "wpa_supplicant running on PID ${PID}..."

  # Set WPA configuration parameters.
  /usr/bin/logger -st "pfatt" "setting wpa_supplicant network configuration..."
  IFS=","
  for STR in ${WPA_PARAMS};
  do
    STR="$(echo -e "${STR}" | sed -e 's/^[[:space:]]*//')"
    RES=$(eval wpa_cli ${STR})
  done

  # wait until wpa_cli has authenticated.
  WPA_STATUS_CMD="wpa_cli status | grep 'suppPortStatus' | cut -d= -f2"

  /usr/bin/logger -st "pfatt" "waiting EAP for authorization..."

  # TODO: blocking for bootup
  while true;
  do
    WPA_STATUS=$(eval ${WPA_STATUS_CMD})
    if [ X${WPA_STATUS} = X"Authorized" ];
    then
      /usr/bin/logger -st "pfatt" "EAP authorization completed..."
      break
    else
      sleep 1
    fi
  done
  /usr/bin/logger -st "pfatt" "em0 should now be available to configure as your WAN..."
  /usr/bin/logger -st "pfatt" "done!"
else
  /usr/bin/logger -st "pfatt" "error: unknown EAP_MODE. '$EAP_MODE' is not valid. exiting..."
  exit 1
fi

Set em0 as your wan, DHCP, mac spoof (RG of cert MAC address)
Voila!

I think this works because ESXI will strip and add VLAN0 tags on the port group so no need netgraph business. I don't think this would work by plugging into my Cisco SG500x because I can't define VLAN0 and so the switch would just drop everything. Too bad! Let me know if anyone has any ideas to improve on things.

t41k2m3

Would @GoldServe (or others) know how to work a similar scenario (without netgraph) with a physical switch (e.g. cisco), instead of an ESXi virtual switch (ONT --> Switch --> pfSense WAN)? Switch should do VLAN0 tagging via dot1p. Is that possible and what (affordable) switches could do that?

t41k2m3

@bulldog5 sounds like you are looking for the supplicant branch:
https://github.com/aus/pfatt/tree/supplicant
Edit pfatt.sh to use EAP_MODE="supplicant" - that should create a simpler netgraph and call wpa_supplicant.
What netgear switch are you using and does it do outgoing VLAN0 tagging?

bulldog5

@t41k2m3
I'm using a GS105Ev2 switch currently. This switch handles the VLAN0 fine, which is why the esxi method also works. But i'm running my pfsense on baremetal, so that option doesn't really apply to me. It would be "nice" to eliminate the GS105E in the middle of my ONT and pfsense WAN. I'll give the link you sent a shot and see how it goes. Thanks

t41k2m3

@bulldog5 curious about the config for both pfS and GS105ev2 if you don't mind. Could not get it to work with latest firmware on GS108ev3 and bare metal pfS (should be pretty close to your GS105ev2 setup) - EAP would not go through and no DHCP or anything after. Is your pfS going out on WAN NIC or do you use VLAN (if so what #/priority)? Any special settings on the switch - not much seemed to be configurable beyond 802.1q tagging and (802.1p) CoS of 0?

bulldog5

@t41k2m3

GS105Ev2 settings

VLAN > 802.1Q TAB

Basic 802.1Q VLAN Status:

PORT 1 and 3 are both in VLAN ID 1. Those are my ONT and WAN ports.

Make sure you're not using Port Based and have that Disabled.

t41k2m3

@bulldog5 is it possible that pfS is doing the tagging (how is pfS setup?) or that no tagging is required at your location? It's a bit odd because it looks like the switch is just passing traffic through on native VLAN without tagging (802.1q or p).

bulldog5

@t41k2m3 you have your MAC Address of your gateway set as your WAN interface in pfsense right?