Scripting the behaviour of Status --> Interfaces --> DHCP renew button

bradtpt · Nov 17, 2018, 2:05 AM

Further to my quest to deliver robust 4G connectivity using a pfsense box have now now veered away from the PPP method I was pursing earlier with my post a few days ago Huawei E3372 will not reconnect PPP after manual disconnect under status-->interfaces and have managed to configure and successfully connect with the Huawei E3372 in ndis mode. This resolves the issue I had previously where once the PPP session disconnects the modem's port would become unresponsive. For those who might be interested, this is how I achieved a successful ndis connection using a Huawei E3372:

The E3372 must be flashed with stick mode firmware v21.xx
Port mode needs to be configured as:

    AT^SETPORT="FF;10,12,16"

The USB Ethernet interface the stick presents to pfsense (ue0) needs to be configured as a WAN gateway with connection type DHCP
The following needs to be setup in shellcmd as an earlyshellcmd. This sends the NDISUP string to the modem and instructs it to connect to the internet. By executing it as an earlyshellcmd, the connection is up and live by the time pfsense goes to bring the interface up and as such it is able to grab a DHCP lease immediately. Executing this command as a regular shellcmd sees it happen after pfsense has brought the interface up and as such pfsense hangs until the DHCP request times out for that interface on boot.

    echo -e 'AT^NDISDUP=1,1' > /dev/cuaU0.1

The same command string is also setup as a cron job that executes every minute. The idea behind this is if the stick is unplugged and reinserted it'll bring the stick back online within one minute.

    echo -e 'AT^NDISDUP=1,1' > /dev/cuaU0.1

This is the point where I have run into a problem. When the stick is removed the interface will go down as expected. When the stick is reinserted the LED on the stick blinks as it's booting up and once my cron job executes the NDISUP command the LED goes solid blue indicating it's connected to the internet which is great, all working as expected... only in pfsense the interface stays down. If I jump into Status --> Interfaces and click the renew button to the right of Status DHCP down it brings the interface up, gets an IP and LAN clients can access the internet. Excellent this what I need, so I figured I could script the same thing by simply executing dhclient ue0 and simply appending that to the end of my cron task. So I tested it in the shell by unplugging and replugging the stick, waited for it to turn solid blue after the cron job, then ran:

# dhclient ue0

and sure enough it got a lease. I thought this was it but unfortunately it gets a lease but it doesn't update pfsense's routing table. So the interface comes up with a valid IP, I can ping hosts on the internet from the pfsense box but my LAN clients don't have any internet access. If I am to repeat the test but instead of running dhclient from the shell I click the renew button under status-->interfaces it gets an ip just the same, but the routing gets updated and the LAN clients can get to the net. I assume because dhclient is being called outside of the pfsense core it doesn't know about it and as a result doesn't update the routes (despite the fact that within a second or so of running dhclient from the shell the interfaces monitor on the home screen updates with the new IP address). I've looked through the code in /usr/local/www/status_interfaces.php and whilst I've found the command it executes for doing a DHCP release I unfortunately can't work out what it's executing when it does a DHCP renew for an interface. It's clearly more than just dhclient [interface_name].

Not one to give up easily I tried a number of other approaches from the shell including ifconfig ue0 up which brings the interface up, at which point I was hopeful that pfsense would see an interface status change from down to up and then automatically poll for a DHCP lease. Unfortunately the interface status changes from to but it never does a DHCP call on its own and as such the interface sits there with no IP. If I hit the renewal button it gets a lease and updates the routes and clients can connect immediately, if I manually execute dhclient ue0 it gets a lease straight away but the routes aren't updated and as such despite the fact it's online and I can ping public IP addresses from that interface on the pfsense box, none of my LAN clients can reach the internet. I even tried via an AT command AT^DHCP? which returned an address in the console session with the modem, but it wasn't passed to the operating system which isn't surprising.

This last obstacle is the only thing stopping me deploying this solution, keen for any feedback or direction anyone might have. In ndis mode my testing has shown speeds exceeding what I was able to achieve using PPP which seemed to tap out at about 45mbps so I'm keen to get this working. Thanks!

bradtpt · Nov 17, 2018, 2:06 AM

This post is deleted!

stephenw10 · Nov 18, 2018, 11:41 PM

I expect it to run that the dhcpclient when the shows as up. That's what the behaviour would be if it was a USB Ethernet device and you connected the cable. Or any Ethernet interface for that matter.

You might be able to do it using /etc/rc.linkup start wan but you would need to make that conditional. You wouldn't want that running every minute, it does a lot of things. One of them is running the dhcp client though.

Another issue you will hit here is that if the modem is unplugged or not in ndis mode for some reason pfSense will not boot fully if ue0 is not detected. It will wait at the interfaces assign screen on the console. That's the main reason why ppp is currently preferred for direct conected modems and an external Ethernet connected device is the best option.

Steve

jly2680 · Aug 1, 2019, 7:48 AM

Did you found any solution for this??Im deploying one of my pfsense box with this modem on my remote site where 4g is the only last viable net solution..Will the modem will automatically get a new dchp address when ISP renew its address??

bradtpt · Aug 1, 2019, 8:57 AM

@jly2680 unfortunately no.

On other client sites I've used a netgear nighthawk in bridge mode for this and it's a solid solution... although for that to work you need to make pfsense override the subnet mask the dhcp passthrough from the nighthawk issues, it issues a /32 and needs to be changed to a /24 otherwise it can't contact the default gateway to actually get traffic out. To do this you need to add "supersede subnet-mask 255.255.255.0" to the "DHCP Option Modifiers" under the "Lease Requirements and Requests" portion of your interface connected to the nighthawk.

jly2680 · Aug 7, 2019, 4:37 PM

i got my e3372 and switch its firmware to stick mode..but my pfsense is based on 2.3.5 running on alix 2d13..I check the dmesg when I put my modem:

ugen1.2: <HUAWEIMOBILE> at usbus1 (disconnected)
u3g0: at uhub1, port 1, addr 2 (disconnected)
ugen1.2: <HUAWEIMOBILE> at usbus1
u3g0: <HUAWEIMOBILE HUAWEIMOBILE, class 0/0, rev 2.10/1.02, addr 2> on usbus1
u3g0: Found 2 ports.

no ue0 device at /dev .. doesnt show the NCM interface..Can I load the NCM driver using 2.3.5??

stephenw10 · Aug 7, 2019, 6:28 PM

Check the USB config output to see if it actually presents as an Ethernet interface there.

Looks like u3g found ppp interfaces though, you can probably connect that way.

Steve

jly2680 · Aug 7, 2019, 6:40 PM

[2.3.5-RELEASE][root@pfSense.local]/root: usbconfig -d ugen1.2 dump_all_config_d esc
ugen1.2: <HUAWEIMOBILE HUAWEIMOBILE> at usbus1, cfg=0 md=HOST spd=HIGH (480Mbps) pwr=ON (2mA)

Configuration index 0

bLength = 0x0009
bDescriptorType = 0x0002
wTotalLength = 0x00af
bNumInterfaces = 0x0003
bConfigurationValue = 0x0001
iConfiguration = 0x0000  <no string>
bmAttributes = 0x0080
bMaxPower = 0x0001

Interface 0
  bLength = 0x0009
  bDescriptorType = 0x0004
  bInterfaceNumber = 0x0000
  bAlternateSetting = 0x0000
  bNumEndpoints = 0x0002
  bInterfaceClass = 0x00ff  <Vendor specific>
  bInterfaceSubClass = 0x0003
  bInterfaceProtocol = 0x0012
  iInterface = 0x0000  <no string>

  Additional Descriptor

  bLength = 0x05
  bDescriptorType = 0x24
  bDescriptorSubType = 0x00
   RAW dump:
   0x00 | 0x05, 0x24, 0x00, 0x10, 0x01


  Additional Descriptor

  bLength = 0x04
  bDescriptorType = 0x24
  bDescriptorSubType = 0x02
   RAW dump:
   0x00 | 0x04, 0x24, 0x02, 0x02


  Additional Descriptor

  bLength = 0x05
  bDescriptorType = 0x24
  bDescriptorSubType = 0x01
   RAW dump:
   0x00 | 0x05, 0x24, 0x01, 0x00, 0x00


  Additional Descriptor

  bLength = 0x05
  bDescriptorType = 0x24
  bDescriptorSubType = 0x06
   RAW dump:
   0x00 | 0x05, 0x24, 0x06, 0x00, 0x00


 Endpoint 0
    bLength = 0x0007
    bDescriptorType = 0x0005
    bEndpointAddress = 0x0081  <IN>
    bmAttributes = 0x0002  <BULK>
    wMaxPacketSize = 0x0200
    bInterval = 0x0000
    bRefresh = 0x0000
    bSynchAddress = 0x0000

 Endpoint 1
    bLength = 0x0007
    bDescriptorType = 0x0005
    bEndpointAddress = 0x0001  <OUT>
    bmAttributes = 0x0002  <BULK>
    wMaxPacketSize = 0x0200
    bInterval = 0x0000
    bRefresh = 0x0000
    bSynchAddress = 0x0000


Interface 1
  bLength = 0x0009
  bDescriptorType = 0x0004
  bInterfaceNumber = 0x0001
  bAlternateSetting = 0x0000
  bNumEndpoints = 0x0003
  bInterfaceClass = 0x00ff  <Vendor specific>
  bInterfaceSubClass = 0x0003
  bInterfaceProtocol = 0x0010
  iInterface = 0x0000  <no string>

  Additional Descriptor

  bLength = 0x05
  bDescriptorType = 0x24
  bDescriptorSubType = 0x00
   RAW dump:
   0x00 | 0x05, 0x24, 0x00, 0x10, 0x01


  Additional Descriptor

  bLength = 0x04
  bDescriptorType = 0x24
  bDescriptorSubType = 0x02
   RAW dump:
   0x00 | 0x04, 0x24, 0x02, 0x02


  Additional Descriptor

  bLength = 0x05
  bDescriptorType = 0x24
  bDescriptorSubType = 0x01
   RAW dump:
   0x00 | 0x05, 0x24, 0x01, 0x00, 0x01


  Additional Descriptor

  bLength = 0x05
  bDescriptorType = 0x24
  bDescriptorSubType = 0x06
   RAW dump:
   0x00 | 0x05, 0x24, 0x06, 0x00, 0x00


 Endpoint 0
    bLength = 0x0007
    bDescriptorType = 0x0005
    bEndpointAddress = 0x0083  <IN>
    bmAttributes = 0x0003  <INTERRUPT>
    wMaxPacketSize = 0x000a
    bInterval = 0x0009
    bRefresh = 0x0000
    bSynchAddress = 0x0000

 Endpoint 1
    bLength = 0x0007
    bDescriptorType = 0x0005
    bEndpointAddress = 0x0082  <IN>
    bmAttributes = 0x0002  <BULK>
    wMaxPacketSize = 0x0200
    bInterval = 0x0000
    bRefresh = 0x0000
    bSynchAddress = 0x0000

 Endpoint 2
    bLength = 0x0007
    bDescriptorType = 0x0005
    bEndpointAddress = 0x0002  <OUT>
    bmAttributes = 0x0002  <BULK>
    wMaxPacketSize = 0x0200
    bInterval = 0x0000
    bRefresh = 0x0000
    bSynchAddress = 0x0000


Interface 2
  bLength = 0x0009
  bDescriptorType = 0x0004
  bInterfaceNumber = 0x0002
  bAlternateSetting = 0x0000
  bNumEndpoints = 0x0001
  bInterfaceClass = 0x00ff  <Vendor specific>
  bInterfaceSubClass = 0x0003
  bInterfaceProtocol = 0x0016
  iInterface = 0x0007  <NCM Network Control Model>

  Additional Descriptor

  bLength = 0x05
  bDescriptorType = 0x24
  bDescriptorSubType = 0x00
   RAW dump:
   0x00 | 0x05, 0x24, 0x00, 0x10, 0x01


  Additional Descriptor

  bLength = 0x06
  bDescriptorType = 0x24
  bDescriptorSubType = 0x1a
   RAW dump:
   0x00 | 0x06, 0x24, 0x1a, 0x00, 0x01, 0x1f


  Additional Descriptor

  bLength = 0x0d
  bDescriptorType = 0x24
  bDescriptorSubType = 0x0f
   RAW dump:
   0x00 | 0x0d, 0x24, 0x0f, 0x09, 0x0f, 0x00, 0x00, 0x00,
   0x08 | 0xea, 0x05, 0x03, 0x00, 0x01


  Additional Descriptor

  bLength = 0x05
  bDescriptorType = 0x24
  bDescriptorSubType = 0x06
   RAW dump:
   0x00 | 0x05, 0x24, 0x06, 0x02, 0x02


 Endpoint 0
    bLength = 0x0007
    bDescriptorType = 0x0005
    bEndpointAddress = 0x0085  <IN>
    bmAttributes = 0x0003  <INTERRUPT>
    wMaxPacketSize = 0x0010
    bInterval = 0x0005
    bRefresh = 0x0000
    bSynchAddress = 0x0000


Interface 2 Alt 1
  bLength = 0x0009
  bDescriptorType = 0x0004
  bInterfaceNumber = 0x0002
  bAlternateSetting = 0x0001
  bNumEndpoints = 0x0003
  bInterfaceClass = 0x00ff  <Vendor specific>
  bInterfaceSubClass = 0x0003
  bInterfaceProtocol = 0x0016
  iInterface = 0x0008  <CDC Network Data>

 Endpoint 0
    bLength = 0x0007
    bDescriptorType = 0x0005
    bEndpointAddress = 0x0085  <IN>
    bmAttributes = 0x0003  <INTERRUPT>
    wMaxPacketSize = 0x0010
    bInterval = 0x0005
    bRefresh = 0x0000
    bSynchAddress = 0x0000

 Endpoint 1
    bLength = 0x0007
    bDescriptorType = 0x0005
    bEndpointAddress = 0x0084  <IN>
    bmAttributes = 0x0002  <BULK>
    wMaxPacketSize = 0x0200
    bInterval = 0x0000
    bRefresh = 0x0000
    bSynchAddress = 0x0000

 Endpoint 2
    bLength = 0x0007
    bDescriptorType = 0x0005
    bEndpointAddress = 0x0003  <OUT>
    bmAttributes = 0x0002  <BULK>
    wMaxPacketSize = 0x0200
    bInterval = 0x0000
    bRefresh = 0x0000
    bSynchAddress = 0x0000

stephenw10 · Aug 7, 2019, 6:49 PM

Looks promising. Can you test it on something 64bit?

Or boot FreeBSD 11.3 on the ALIX maybe?

You might get it working in 2.3.5 with the right FreeBSD 10 kernel module. Really you should upgrade though. 2.3.X is unmaintained and should be considered insecure as a result.

Steve

jly2680 · Aug 7, 2019, 7:01 PM

@stephenw10 said in Scripting the behaviour of Status --> Interfaces --> DHCP renew button:

Looks promising. Can you test it on something 64bit?

Or boot FreeBSD 11.3 on the ALIX maybe?

You might get it working in 2.3.5 with the right FreeBSD 10 kernel module. Really you should upgrade though. 2.3.X is unmaintained and should be considered insecure as a result.

Steve

I would prefer loading kernel module on 2.3.5,, but I dont have any links to dowload modules

stephenw10 · Aug 7, 2019, 7:18 PM

Get them from the corresponding FreeBSD iso. 10.3 in this case.

https://docs.netgate.com/pfsense/en/latest/releases/versions-of-pfsense-and-freebsd.html

Steve

jly2680 · Aug 7, 2019, 8:34 PM

@stephenw10 said in Scripting the behaviour of Status --> Interfaces --> DHCP renew button:

Looks promising. Can you test it on something 64bit?

Or boot FreeBSD 11.3 on the ALIX maybe?

You might get it working in 2.3.5 with the right FreeBSD 10 kernel module. Really you should upgrade though. 2.3.X is unmaintained and should be considered insecure as a result.

Steve

@stephenw10 said in Scripting the behaviour of Status --> Interfaces --> DHCP renew button:

Get them from the corresponding FreeBSD iso. 10.3 in this case.

https://docs.netgate.com/pfsense/en/latest/releases/versions-of-pfsense-and-freebsd.html

Steve

@stephenw10 said in Scripting the behaviour of Status --> Interfaces --> DHCP renew button:

Get them from the corresponding FreeBSD iso. 10.3 in this case.

https://docs.netgate.com/pfsense/en/latest/releases/versions-of-pfsense-and-freebsd.html

Steve

I did try it on vmware lastest pfsense , NCM and ue0 interface showed..can you guide which module are needed to load on 2.3.5??

But when I load Freebsd 10.3 also on vmware,,no ncm and ue0 int showed,,

stephenw10 · Aug 7, 2019, 9:28 PM

@jly2680 said in Scripting the behaviour of Status --> Interfaces --> DHCP renew button:

But when I load Freebsd 10.3 also on vmware,,no ncm and ue0 int showed,,

That doesn't bode well. It's probably just not supported there then.

You can use kldstat to see what kernel modules are being used.

Steve

bradtpt · Aug 7, 2019, 11:56 PM

@jly2680 said in Scripting the behaviour of Status --> Interfaces --> DHCP renew button:

i got my e3372 and switch its firmware to stick mode..but my pfsense is based on 2.3.5 running on alix 2d13..I check the dmesg when I put my modem:
ugen1.2: <HUAWEIMOBILE> at usbus1 (disconnected)
u3g0: at uhub1, port 1, addr 2 (disconnected)
ugen1.2: <HUAWEIMOBILE> at usbus1
u3g0: <HUAWEIMOBILE HUAWEIMOBILE, class 0/0, rev 2.10/1.02, addr 2> on usbus1
u3g0: Found 2 ports.
no ue0 device at /dev .. doesnt show the NCM interface..Can I load the NCM driver using 2.3.5??

Yes you can, you need to use the setport command from the AT command set for the modem to enable the NCM interface. I have another post with some details thrown in contained here: https://forum.netgate.com/topic/137637/huawei-e3372-will-not-reconnect-ppp-after-manual-disconnect-under-status-interfaces

In summary what you need to do is jump into the console and talk directly to the modem via the emulated serial port by running: cu -l /dev/cuaU0.0 or cu -l /dev/cuaU0.1 depending on which port it created and wants to talk on. Once in there run the command AT^SETPORT="FF;10,12,16" and then AT^RESET to reboot the stick. After having done this once, from herein the stick should present the NCM interface which is defined by the 16 in the command above.

PS - I can confirm this works on 2.3.x as I had a client on that version a while ago and utilized ones of these.

stephenw10 · Aug 8, 2019, 11:40 AM

Nice!

jly2680 · Aug 8, 2019, 12:37 PM

@bradtpt said in Scripting the behaviour of Status --> Interfaces --> DHCP renew button:

@jly2680 said in Scripting the behaviour of Status --> Interfaces --> DHCP renew button:
i got my e3372 and switch its firmware to stick mode..but my pfsense is based on 2.3.5 running on alix 2d13..I check the dmesg when I put my modem:
ugen1.2: <HUAWEIMOBILE> at usbus1 (disconnected)
u3g0: at uhub1, port 1, addr 2 (disconnected)
ugen1.2: <HUAWEIMOBILE> at usbus1
u3g0: <HUAWEIMOBILE HUAWEIMOBILE, class 0/0, rev 2.10/1.02, addr 2> on usbus1
u3g0: Found 2 ports.
no ue0 device at /dev .. doesnt show the NCM interface..Can I load the NCM driver using 2.3.5??
Yes you can, you need to use the setport command from the AT command set for the modem to enable the NCM interface. I have another post with some details thrown in contained here: https://forum.netgate.com/topic/137637/huawei-e3372-will-not-reconnect-ppp-after-manual-disconnect-under-status-interfaces

In summary what you need to do is jump into the console and talk directly to the modem via the emulated serial port by running: cu -l /dev/cuaU0.0 or cu -l /dev/cuaU0.1 depending on which port it created and wants to talk on. Once in there run the command AT^SETPORT="FF;10,12,16" and then AT^RESET to reboot the stick. After having done this once, from herein the stick should present the NCM interface which is defined by the 16 in the command above.

PS - I can confirm this works on 2.3.x as I had a client on that version a while ago and utilized ones of these.

[2.3.5-RELEASE][root@pfSense.local]/root: cu -l /dev/cuaU0.0
Connected

Manufacturer: huawei
Model: E3372
Revision: 21.200.07.00.805
IMEI: 866785033259671
+GCAP: +CGSM,+DS,+ES

OK

OK

OK
ugen1.2: <HUAWEIMOBILE> at usbus1 (disconnected)
u3g0: at uhub1, port 1, addr 2 (disconnected)
[2.3.5-RELEASE][root@pfSense.local]/root: ugen1.2: <HUAWEIMOBILE> at usbus1
u3g0: <HUAWEIMOBILE HUAWEIMOBILE, class 0/0, rev 2.10/1.02, addr 2> on usbus1
u3g0: Found 2 ports.

Same no ncm interface after issuing these commands..

bradtpt · Aug 8, 2019, 1:01 PM

Ok can you show me the output of the following:

cu -l /dev/cuaU0.0
AT^SETPORT=?
AT^SETPORT?

Unfortunately I don't have one with me to check the config directly, however have you modeswitched the device out of mass storage mode? If not, you'll need to download and install usbmodeswitch by running this (from memory 2.3.x is on FreeBSD10 so this should be the correct version):
pkg add http://pkg.freebsd.org/FreeBSD:10:amd64/release_4/All/usb_modeswitch-2.5.0.txz
and then running this:
/usr/local/sbin/usb_modeswitch -J -v0x12d1 -p0x1f01

Running that will cause the device to drop out of mass storage mode and should present the interfaces. Let us know how you go. If I had a stick with me I'd go through the motions and post it up for you but unfortunately I can only go off some notes I have here from the last one I did quite a while back.

jly2680 · Aug 8, 2019, 1:52 PM

[2.3.5-RELEASE][root@pfSense.local]/root: cu -l /dev/cuaU0.0
Connected

ERROR

^SETPORT:FF;10,12,16

OK

Issuing this AT^SETPORT=? command got an error answer from modem..here's the message after usbmodeswitch

[2.3.5-RELEASE][root@pfSense.local]/root: usb_modeswitch -J -v0x12d1 -p0x1506   Look for default devices ...
   product ID matched
 Found devices in default mode (1)
Access device 002 on bus 001
Get the current device configuration ...
Current configuration number is 1
Use interface number 0
 with class 255
Error: can't use storage command in MessageContent with interface 0; interface class is 255, expected 8. Abort

echo 'AT^NDISDUP=1,1,"jawalnet.com.sa"' > /dev/cuaU0.1

this modem command gave me a solid blue light on the modem,,but no ue0 or cde interface showed up..

bradtpt · Aug 8, 2019, 2:07 PM

The error from usbmodeswitch makes sense now I think about it, you have the stick firmware loaded so as such the device when inserted won't present as a mass storage device. I have an e3372 with the stick firmware loaded at work, if I get a chance tomorrow I'll fire up a machine with 2.3.5 and see if I can make it work. As I'm not familiar with your Alix hardware the closest I can go to replicating your environment would be to install an archived version of pfSense-CE-memstick-2.3.3-RELEASE-amd64.img I have here on an amd machine. Is your alix similar or a totally different platform?

jly2680 · Aug 8, 2019, 2:29 PM

@bradtpt said in Scripting the behaviour of Status --> Interfaces --> DHCP renew button:

The error from usbmodeswitch makes sense now I think about it, you have the stick firmware loaded so as such the device when inserted won't present as a mass storage device. I have an e3372 with the stick firmware loaded at work, if I get a chance tomorrow I'll fire up a machine with 2.3.5 and see if I can make it work. As I'm not familiar with your Alix hardware the closest I can go to replicating your environment would be to install an archived version of pfSense-CE-memstick-2.3.3-RELEASE-amd64.img I have here on an amd machine. Is your alix similar or a totally different platform?

Oh thanks..its an old alix 2d13 i386..I think 2.3.5 doesnt work with 3372 in stick mode