How to configure RNDIS on E3372

stephenw10

The ppp link is limited by the fact it is ppp. I believe it maybe limited to 42Mbps.
The ue0 interface is not limited by that but support for it is not well integrated with pfSense. You can assign the interface and use the connection but should you ever remove the modem or it powers off for some reason pfSense will not fully boot (if it's rebooted) until that interface is reassigned.

If you can live with the PPP speed I would use that. That's what I do personally as a backup connection.

Steve

RobertW

@stephenw10

I did PPP tests, the best result I got was 62 Mbps (download): http://www.speedtest.net/pl/result/7754164121

This is my primary connection (WAN). Is there any idea to support the ue0 (in the near future) ?

Robert

stephenw10

I'm not aware of anything.

62Mbps is pretty good though. Kills my 42Mbps theory!

If you need to use LTE by far the best solution is an Ethernet terminating external modem like:
https://www.netgear.com/home/products/mobile-broadband/lte-modems/LB1120.aspx

Steve

RobertW

This speed is different, once it is 62 Mbps, the second time is 51 Mbps... 30 Mbps, 20 Mbps, 10 Mbps, dependably on the BTS load, weather ?

I like my home system configurations: pfsense on VMware ESXi 6.7 (installed on HP MicroServer Gen8 + pass through USB 3.0 chip for the best performance with E3372 modem + external dual LTE MIMO antenna and OS in storage SSD).

I will wait until the ue0 problem is solved, always ready for testing (VM backup).

Best Reagards,
Robert

Dohmar

@robertw said in How to configure RNDIS on E3372:

@stephenw10

I did PPP tests, the best result I got was 62 Mbps (download): http://www.speedtest.net/pl/result/7754164121

This is my primary connection (WAN). Is there any idea to support the ue0 (in the near future) ?

Robert

It can work. Here is a little guide I've just written as I needed to automate it for pfsense. This can hopefully give others a little bit of background on the Hilink vs stick huaweis:

There has been a lot of conflicting documentation around older versions of pfSense/FreeBSD and people trying to get usb 4g dongles working in NDIS mode.
The advantages of running in NDIS mode is twofold. Firstly you'll be able to avoid being double NAT'ed as the Hilink dongle does.
Secondly, you'll be able to avoid being forced to use the PPP mode and hence lower connection speed that the Stick dongle generally uses.

For this example, I am using the Huawei e3372 4G dongle working directly in stick mode.
e3372h is sold as a hilink dongle, e3372s is sold as a stick dongle. It is the stick mode that we want for this purpose.
If you have a hilink dongle, you'll need to flash it to the firmware 21.x.x (22.x.x is Hilink).
All e3372s can be flashed, however it is critical that you flash the correct firmware for your particular version.
If you get it wrong, you can still recover your device by using a pin/shorting tool to boot the dongle into recovery mode.
I will not go into the details here as there are many other resources for you to use. Suffice to say it is complicated.
If you can buy the dongle in stick mode you can save yourself a lot of work.
Once your modem is in stick mode and you plug it in to pfSense, you'll see the following in the dmesg once it has initialized;

uhub0: <0x8086 XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus0
uhub0: 7 ports with 7 removable, self powered
ugen0.2: <HUAWEIMOBILE HUAWEIMOBILE> at usbus0
ugen0.2: <HUAWEIMOBILE HUAWEIMOBILE> at usbus0 (disconnected)
ugen0.2: <HUAWEIMOBILE HUAWEIMOBILE> at usbus0
u3g0 on uhub0
u3g0: <HUAWEIMOBILE HUAWEIMOBILE, class 0/0, rev 2.10/1.02, addr 2> on usbus0
u3g0: Found 2 ports.
cdce0 on uhub0
cdce0: <NCM Network Control Model> on usbus0
cdce0: faking MAC address
ue0: <USB Ethernet> on cdce0
ue0: Ethernet address: 00:00:00:00:00:00
umass0 on uhub0
umass0: <Mass Storage> on usbus0
umass1 on uhub0
umass1: <Mass Storage> on usbus0
cd0 at umass-sim0 bus 0 scbus1 target 0 lun 0
cd0: <HUAWEI Mass Storage 2.31> Removable CD-ROM SCSI-2 device
cd0: 40.000MB/s transfers
cd0: Attempt to query device size failed: NOT READY, Medium not present
cd0: quirks=0x10<10_BYTE_ONLY>
da0 at umass-sim1 bus 1 scbus2 target 0 lun 0
da0: <HUAWEI TF CARD Storage 2.31> Removable Direct Access SCSI-2 device
da0: 40.000MB/s transfers
da0: Attempt to query device size failed: NOT READY, Medium not present
da0: quirks=0x2<NO_6_BYTE>

The critical thing here is the ue0 interface. This is the NDIS interface.
If you have another brand of wireless broadband modem, as long as it presents a usb ethernet (ueX) interface then you will be able to follow the rest of this guide

The stick should also create a couple of serial interfaces. One is for serial AT commands. It is this one we need to use.
To get this, you must find out what usb port your device is on. You can check this by typing 'ls -l /dev/cuaU*'

crw-rw---- 1 uucp dialer 0x5f Dec 2 22:40 /dev/cuaU0.0
crw-rw---- 1 uucp dialer 0x60 Dec 2 20:36 /dev/cuaU0.0.init
crw-rw---- 1 uucp dialer 0x61 Dec 2 20:36 /dev/cuaU0.0.lock
crw-rw---- 1 uucp dialer 0x65 Dec 2 20:36 /dev/cuaU0.1
crw-rw---- 1 uucp dialer 0x66 Dec 2 20:36 /dev/cuaU0.1.init
crw-rw---- 1 uucp dialer 0x67 Dec 2 20:36 /dev/cuaU0.1.lock

Here I have 2 devices /dev/cuaU0.0 and /dev/cuaU0.1
In my case the serial port to communicate with the modem is /dev/cuaU0.0
If you type cu /dev/cuaU0.0 you should get into a serial console. The modem will dump its signal to noise reports constantly while you are connected.
If you type ATZ to reset the modem and get an OK, then you know you have the right port.

You'll need to create a script and set it executable in order to tell the stick to bring the NDIS interface up and tell the modem what its APN is.
This will then allow the modem to form a bridge between the ue0 interface and the ISP's wireless network.
For the sake of cutting down on log messages relating to medium not present, I also disable the cd0 and da0 mass storage.
I keep my script in /usr/local/bin/e3372.sh :

#/bin/sh
echo Initializing e3372 dongle
camcontrol eject cd0
camcontrol eject da0
echo 'AT^NDISDUP=1,1,"YOURAPN"' > /dev/cuaU0.0

You must replace YOURAPN with whatever your ISP required.
You must replace /dev/cuaU0.0 with whatever the serial interface the USB modem uses.

Once the ue0 is present and the script has ran and told the modem what APN to use, pfSense will be able to run DHCPD on the ue0 interface.
It is after this script is ran that the green led on the stick should stop blinking and remain constantly green.
Once you see a constant green led then you know the APN has been accepted by the ISP.
After this, you will need to disable and then re-enable ue0 to allow DHCPd to attempt getting an address from your carrier.

AUTOMATION
To automate this stage for the sake of booting, use the shellcmd pkg and create a link to /usr/local/bin/e3372.sh as an earlyshellcmd
This will allow the modem to come up before pfSense tries to get DHCP

Hope this helps someone else out there. It took me a while to get it working and after I did, I wasn't sure what of the dozens of commands I'd tried actually worked.
I've tested this from a full power off too so it works just fine on booting from scratch.

Cheers
-D

Edited - shell script missed some formatting

stephenw10

@dohmar said in How to configure RNDIS on E3372:

Suffice to say it is complicated.

Nice.

There is still the very real potential issue here that if the modem is unplugged or fails then pfSense will not boot correctly if rebooted as one of it's assigned interfaces will have disappeared. It is possible to omit ue from the list of checked interfaces but that still leaves the firewall with an undefined interface.

Steve

Dohmar

@stephenw10 said in How to configure RNDIS on E3372:

@dohmar said in How to configure RNDIS on E3372:

Suffice to say it is complicated.

Nice.

There is still the very real potential issue here that if the modem is unplugged or fails then pfSense will not boot correctly if rebooted as one of it's assigned interfaces will have disappeared. It is possible to omit ue from the list of checked interfaces but that still leaves the firewall with an undefined interface.

Steve

This is correct, I found the same issue mentioned with my Minisys PC when I was troubleshooting a boot issue. Do you know if there is a workaround for this aside from preventing the check? How do I disable the check for the interface?

The boot issue I faced was the bootloader crashing when trying to boot. The BIOS was seeing the TF/SD reader and attempting to hand off XHCI to the OS. To prevent the bootloader from crashing when trying to boot while the dongle is plugged in, you must go into the bios and disable both the XHCI option as well as the USB mass storage. This should prevent the BIOS boot options from being able to see the tf reader/sd reader.

I've got no idea how to fix the re-config getting triggered when a defined interface isn't detected on boot. If anyone has any suggestions I'll add it to my guide.

Edit - Is it possible to create a virtual ethernet port that gets checked at boot, that is then linked to ue0? something like a symbolic link that isnt going to make pfSense crap itself on boot if there is an issue with ue0 ?

-D

stephenw10

Add it here:
https://github.com/pfsense/pfsense/blob/1b988ed0e7168ada9e6260274f63fd84b15873a1/src/etc/inc/util.inc#L2201

But as I say whilst that will allow it to boot it then ends up with an interface defined that doesn't exist.

Steve

Dohmar

@stephenw10 said in How to configure RNDIS on E3372:

Add it here:
https://github.com/pfsense/pfsense/blob/1b988ed0e7168ada9e6260274f63fd84b15873a1/src/etc/inc/util.inc#L2201

But as I say whilst that will allow it to boot it then ends up with an interface defined that doesn't exist.

Steve

Ahh. That kind of defeats the purpose all together. I'll skip that I think
Cheers
-D

stephenw10

It only affects anything if the interface is removed and then pfSense reboots. In which case adding ue to that list allows it to boot fully so you have access to the gui etc. Otherwise you would need to use the console to re-assign the interfaces without the ue device.
Other users have done after finding exactly that happened when they lost power or the modem was removed as a troubleshooting step for example. It should never happen normally.

Steve

Dohmar

@stephenw10

OK good idea, thanks for the tip. Have modified my config to suit
Cheers
-D

RobertW

@stephenw10 said in How to configure RNDIS on E3372:

I'm not aware of anything.

62Mbps is pretty good though. Kills my 42Mbps theory!

If you need to use LTE by far the best solution is an Ethernet terminating external modem like:
https://www.netgear.com/home/products/mobile-broadband/lte-modems/LB1120.aspx

Steve

New record on ppp ~74 MBps (better BTS ?):
https://www.speedtest.net/result/8498127448

Best Regards,
Robert