IPMI access over pfsense OpenVPN?

miles267

Has anyone managed to access IPMI over pfsense OpenVPN? I can access my IPMI when on my LAN at home from any client via a 192.x.x.x address. However when connected remotely via pfsense OpenVPN my clients 10.0.8.0/24 are unable to connect to that 192.x.x.x address.

dreamslacker

Did you add a firewall rule on the OpenVPN tab allowing traffic from the OpenVPN subnet as source and LAN subnet as destination?

If not, the firewall will block OVPN to LAN traffic by default.

Cino

I believe the issue is with the IPMI hardware itself. I've have the same issue, even tho you have a gateway IP address; it doesn't use it for some reason. Doesn't work for me even when routing between different subnets

Nachtfalke

IPMI is working for me on three machines without problems.

I setup static IP, SM, GW on IPMI interface.

Cino

@Nachtfalke:

IPMI is working for me on three machines without problems.

I setup static IP, SM, GW on IPMI interface.

mine is static also… but dont get any replies from it when i'm routing from different subnets... one day i'll do a packet capture to be fully sure.. i am using ipmi that is built into my d510 motherboard

miles267

@dreamslacker:

Did you add a firewall rule on the OpenVPN tab allowing traffic from the OpenVPN subnet as source and LAN subnet as destination?

If not, the firewall will block OVPN to LAN traffic by default.

OK - I can access my IPMI from any client PC on my LAN. IPMI info is:
ip: 192.168.0.2
gw: 192.168.0.1
mask: 255.255.255.0

I created an OpenVPN rule as:
action: pass
proto: any
source - type: network, 10.0.8.0/24 (this is my OpenVPN)
destination - type: LAN subnet

But when I connect to OpenVPN and receive, for example, an IP of 10.0.8.6, I am unable to open my browser and go to http://192.168.0.2 like I can when at home on my LAN. But I am able to access other client PCs on my LAN such as \192.168.0.100, etc.

Am I doing something wrong?

torontob

I remember to have the same issues but don't remember how I solved it but I was working for me using SuperMicro boards. I think it was a port forward issue:

UDP Port 623 for IPMI protocol
TCP Port 80 for Web User Interface
TCP Port 443 for KVM Over Lan
On the Client side you need to open UDP Port 6666 for Text Console Redirection

Try a search on Google for proper ports and forward them regardless of the OpenVPN tunnel because Java still needs those to forward to those ports….

miles267

@torontob:

I remember to have the same issues but don't remember how I solved it but I was working for me using SuperMicro boards. I think it was a port forward issue:

UDP Port 623 for IPMI protocol
TCP Port 80 for Web User Interface
TCP Port 443 for KVM Over Lan
On the Client side you need to open UDP Port 6666 for Text Console Redirection

Try a search on Google for proper ports and forward them regardless of the OpenVPN tunnel because Java still needs those to forward to those ports….

Cool - thanks. I found a lost of IPMI ports (I too am using a Supermicro X7SPA board)
HTTP: 80 (TCP)
HTTPS: 443 (TCP)
IPMI: 623 (UDP)
Remote console: 5900 (TCP)
Virtual media: 623 (TCP)
Text Console Redirection: 6666 (UDP)

I have tried creating each of these rules under FIREWALL > RULES on either the OpenVPN (tab) or the LAN (tab). For example:

Action: Pass
Interface: LAN
Proto: UDP
Source: Any
Dest: Single Host - 192.168.0.2 (this is the LAN IP of my IPMI)
Dest Port Range: 623 to 623

And so on. Feel like I'm getting closer, but still no luck. Thanks so much for the hints you're providing.

Nachtfalke

I do not have any firewall restrictions for the connections.

So perhaps you try first with an "allow any to any" rule for your openvpn to LAN and for your IPMI to any.
You can log this traffic in firewall rule and check what is happening.

wallabybob

@miles267:

But when I connect to OpenVPN and receive, for example, an IP of 10.0.8.6, I am unable to open my browser and go to http://192.168.0.2 like I can when at home on my LAN.

Does the browser report anything when you attempt that?

miles267

@Nachtfalke:

I do not have any firewall restrictions for the connections.

So perhaps you try first with an "allow any to any" rule for your openvpn to LAN and for your IPMI to any.
You can log this traffic in firewall rule and check what is happening.

I created in the Firewall > Rules > OpenVPN tab an 'allow any to any' rule from OpenVPN to LAN subnet. Still unable to connect to http://192.168.0.2 after I establish an OpenVPN connection.

Not sure what 'IPMI to any' meant.

Nachtfalke

Your IPMI interface is probably on the LAN subnet. Your IPMI interface has an IP address.
create a firewall rule which allows all traffic from your IPMI interface to any.

So you need to rule:
One from OpenVPN to IPMI and
one from IPMI to OpenVPN

We need to find out if it is a firewall "problem" with any port or protocol.

miles267

@Nachtfalke:

Your IPMI interface is probably on the LAN subnet. Your IPMI interface has an IP address.
create a firewall rule which allows all traffic from your IPMI interface to any.

So you need to rule:
One from OpenVPN to IPMI and
one from IPMI to OpenVPN

We need to find out if it is a firewall "problem" with any port or protocol.

It appears I already have a LAN rule that passes (allows) any traffic from LAN subnet to any destination. And my IPMI is located on the LAN subnet. When I attempt to add an OpenVPN rule to allows any traffic from openvpn interface to any on LAN subnet, it still doesn't appear to work. Am stumped.

vassilis

I can confirm this issue with my boxes too.
The IPMI interface is not reachable over the VPN or over other OPT networks.

Even the firewall itself cant "see" the IPMI IP Address.

(Supermicro X7SPE-HF, X7SPA-HF)

Could it be because the NIC is shared? One could try using the NIC as a dedicated IPMI port and see if that solves the issue.
Of course then he/she would loose a NIC port..

torontob

Create an OpenVPN interface by adding interface and then I think you will see two tabs for OpenVPN under Firewall rules. Set both to * * * * OPEN ALL basically.

Then reset all the states and try again.

To debug this, you can input the port numbers or IP of the IPMI in States under Diagnostics > States. You want to see MULTIPLE:MULTIPLE for each connection else you have a problem on one side if you have a SINGLE which you have to look into. It will for sure tell you where the problem is.

vassilis

Here we go..
I didnt test it over a VPN connection, I simply tried to connect to the IPMI interface on the LAN side through a system on the OPT1 side.

OPT1 has access to that IP which is confirmed in the firewall log
pass Jun 5 08:57:00 ORANGE 172.29.xxxx 10.xxxxx:80 TCP:S

In the states there are two entries
tcp 10.xxxxx:80 <- 172.29.xxxx CLOSED:SYN_SENT
tcp 172.29.xxxx -> 10.xxxx:80 SYN_SENT:CLOSED

The IPMI interface by itself does not block any connections originating by different IP ranges. I have confirmed that with the manufacturer and a second IPMI interface I have that is not on the firewall is reachable (with the correct firewall rules)

cmb

@vassilis:

In the states there are two entries
tcp 10.xxxxx:80 <- 172.29.xxxx CLOSED:SYN_SENT
tcp 172.29.xxxx -> 10.xxxx:80 SYN_SENT:CLOSED

That just indicates you sent the SYN to open the connection, and nothing responded. Can confirm that by packet capturing on the internal interface. If the IP configuration, mask and gateway are definitely correct on the IPMI, and it has no IP restrictions, then it's just yet another embedded device that refuses to use its default gateway (which isn't extremely uncommon, though IMPI's generally work). Can source NAT the traffic destined to that host to work around using advanced outbound NAT.

vassilis

on the LAN side:

09:40:19.582103 00:25:90:03:XX:XX > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Ethernet (len 6), IPv4 (len 4), Request who-has 10.XXX.XXX.XXX tell 10.XXX.XXX.1, length 28
09:40:22.576018 00:25:90:03:XX:XX > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Ethernet (len 6), IPv4 (len 4), Request who-has 10.XXX.XXX.XXX tell 10.XXX.XXX.1, length 28
09:40:28.572483 00:25:90:03:XX:XX > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Ethernet (len 6), IPv4 (len 4), Request who-has 10.XXX.XXX.XXX tell 10.XXX.XXX.1, length 28

on the OPT side:

09:37:49.082986 52:54:00:77:XX:XX > 00:25:90:03:XX:XX, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 3586, offset 0, flags [DF], proto TCP (6), length 60)
    172.XXX.XXX.XXX.55370 > 10.XXX.XXX.XXX.80: Flags [s], cksum 0xac95 (correct), seq 622805277, win 5840, options [mss 1460,sackOK,TS val 1421299487 ecr 0,nop,wscale 6], length 0
09:37:52.078438 52:54:00:77:XX:XX > 00:25:90:03:XX:XX, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 3587, offset 0, flags [DF], proto TCP (6), length 60)
    172.XXX.XXX.XXX.55370 > 10.XXX.XXX.XXX.80: Flags [s], cksum 0xa9a7 (correct), seq 622805277, win 5840, options [mss 1460,sackOK,TS val 1421300237 ecr 0,nop,wscale 6], length 0
09:37:58.077919 52:54:00:77:XX:XX > 00:25:90:03:XX:XX, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 3588, offset 0, flags [DF], proto TCP (6), length 60)
    172.XXX.XXX.XXX.55370 > 10.XXX.XXX.XXX.80: Flags [s], cksum 0xa3cb (correct), seq 622805277, win 5840, options [mss 1460,sackOK,TS val 1421301737 ecr 0,nop,wscale 6], length 0

What I did after was a packet capture from my workstation through a Site-to-Site OpenVPN connection to a IPMI interface that is _not_ on the firewall:

[code]10:31:17.364698 00:1f:c6:e9:XX:XX > 00:25:90:03:XX:XX, ethertype IPv4 (0x0800), length 62: (tos 0x0, ttl 128, id 3326, offset 0, flags [DF], proto TCP (6), length 48)
    10.106.XXX.XXX.1746 > 10.90.XXX.XXX.80: Flags [s], cksum 0x4652 (correct), seq 2007364705, win 64380, options [mss 1460,nop,nop,sackOK], length 0
10:31:17.498489 00:25:90:03:XX:XX > 00:1f:c6:e9:XX:XX, ethertype IPv4 (0x0800), length 62: (tos 0x0, ttl 62, id 0, offset 0, flags [DF], proto TCP (6), length 48)
    10.90.XXX.XXX.80 > 10.106.XXX.XXX.1746: Flags [S.], cksum 0xe22f (correct), seq 3626201350, ack 2007364706, win 5840, options [mss 1352,nop,nop,sackOK], length 0
10:31:17.498675 00:1f:c6:e9:XX:XX > 00:25:90:03:XX:XX, ethertype IPv4 (0x0800), length 60: (tos 0x0, ttl 128, id 3329, offset 0, flags [DF], proto TCP (6), length 40)
    10.106.XXX.XXX.1746 > 10.90.XXX.XXX.80: Flags [.], cksum 0x29db (correct), seq 1, ack 1, win 64380, length 0
10:31:17.499657 00:1f:c6:e9:XX:XX > 00:25:90:03:XX:XX, ethertype IPv4 (0x0800), length 394: (tos 0x0, ttl 128, id 3330, offset 0, flags [DF], proto TCP (6), length 380)
    10.106.XXX.XXX.1746 > 10.90.XXX.XXX.80: Flags [P.], cksum 0x2e32 (correct), seq 1:341, ack 1, win 64380, length 340
10:31:17.638551 00:25:90:03:XX:XX > 00:1f:c6:e9:XX:XX, ethertype IPv4 (0x0800), length 54: (tos 0x0, ttl 62, id 24317, offset 0, flags [DF], proto TCP (6), length 40)
    10.90.XXX.XXX.80 > 10.106.XXX.XXX.1746: Flags [.], cksum 0x0ae4 (correct), seq 1, ack 341, win 6432, length 0

As you can see, there all works perfectly.. Doesnt that mean that the IPMI does work correctly with the gateway setting?

Trying a ping from the firewall on the LAN interface to the IPMI got this:
[code]PING 10.XXX.XXX.9 (10.XXX.XXX.9) from 10.XXX.XXX.1: 56 data bytes

--- 10.XXX.XXX.9 ping statistics ---
3 packets transmitted, 0 packets received, 100.0% packet loss[/code]

So even on the same subnet the firewall cant ping the IPMI. From any other system on that network I can ping the IPMI

I also spoke to Supermicro about this and they said that there are no restrictions.
[/s][/code][/s][/s][/s]

torontob

Do you have an OpenVPN interface? go to add interface and add OpenVPN interface by pressing + sign. If you see a + sign there you have an interface that is not configured like OpenVPN tunnel interface. Please follow my last post.

Then try the test again and go to Diagnostic > States and then put the IP of the IPMI and see if any requests come in. If not, that is because you are missing OpenVPN interface and firewall rules in it.

vassilis

@torontob:

Do you have an OpenVPN interface? go to add interface and add OpenVPN interface by pressing + sign. If you see a + sign there you have an interface that is not configured like OpenVPN tunnel interface. Please follow my last post.

Then try the test again and go to Diagnostic > States and then put the IP of the IPMI and see if any requests come in. If not, that is because you are missing OpenVPN interface and firewall rules in it.

Do you think there is a chance it will work this way even though it did not for the tests above?
(setup above: LAN interface to OPT1 interface)