IPMI access over pfsense OpenVPN?
-
I remember to have the same issues but don't remember how I solved it but I was working for me using SuperMicro boards. I think it was a port forward issue:
UDP Port 623 for IPMI protocol
TCP Port 80 for Web User Interface
TCP Port 443 for KVM Over Lan
On the Client side you need to open UDP Port 6666 for Text Console RedirectionTry a search on Google for proper ports and forward them regardless of the OpenVPN tunnel because Java still needs those to forward to those ports….
-
I remember to have the same issues but don't remember how I solved it but I was working for me using SuperMicro boards. I think it was a port forward issue:
UDP Port 623 for IPMI protocol
TCP Port 80 for Web User Interface
TCP Port 443 for KVM Over Lan
On the Client side you need to open UDP Port 6666 for Text Console RedirectionTry a search on Google for proper ports and forward them regardless of the OpenVPN tunnel because Java still needs those to forward to those ports….
Cool - thanks. I found a lost of IPMI ports (I too am using a Supermicro X7SPA board)
HTTP: 80 (TCP)
HTTPS: 443 (TCP)
IPMI: 623 (UDP)
Remote console: 5900 (TCP)
Virtual media: 623 (TCP)
Text Console Redirection: 6666 (UDP)I have tried creating each of these rules under FIREWALL > RULES on either the OpenVPN (tab) or the LAN (tab). For example:
Action: Pass
Interface: LAN
Proto: UDP
Source: Any
Dest: Single Host - 192.168.0.2 (this is the LAN IP of my IPMI)
Dest Port Range: 623 to 623And so on. Feel like I'm getting closer, but still no luck. Thanks so much for the hints you're providing.
-
I do not have any firewall restrictions for the connections.
So perhaps you try first with an "allow any to any" rule for your openvpn to LAN and for your IPMI to any.
You can log this traffic in firewall rule and check what is happening. -
But when I connect to OpenVPN and receive, for example, an IP of 10.0.8.6, I am unable to open my browser and go to http://192.168.0.2 like I can when at home on my LAN.
Does the browser report anything when you attempt that?
-
I do not have any firewall restrictions for the connections.
So perhaps you try first with an "allow any to any" rule for your openvpn to LAN and for your IPMI to any.
You can log this traffic in firewall rule and check what is happening.I created in the Firewall > Rules > OpenVPN tab an 'allow any to any' rule from OpenVPN to LAN subnet. Still unable to connect to http://192.168.0.2 after I establish an OpenVPN connection.
Not sure what 'IPMI to any' meant.
-
Your IPMI interface is probably on the LAN subnet. Your IPMI interface has an IP address.
create a firewall rule which allows all traffic from your IPMI interface to any.So you need to rule:
One from OpenVPN to IPMI and
one from IPMI to OpenVPNWe need to find out if it is a firewall "problem" with any port or protocol.
-
Your IPMI interface is probably on the LAN subnet. Your IPMI interface has an IP address.
create a firewall rule which allows all traffic from your IPMI interface to any.So you need to rule:
One from OpenVPN to IPMI and
one from IPMI to OpenVPNWe need to find out if it is a firewall "problem" with any port or protocol.
It appears I already have a LAN rule that passes (allows) any traffic from LAN subnet to any destination. And my IPMI is located on the LAN subnet. When I attempt to add an OpenVPN rule to allows any traffic from openvpn interface to any on LAN subnet, it still doesn't appear to work. Am stumped.
-
I can confirm this issue with my boxes too.
The IPMI interface is not reachable over the VPN or over other OPT networks.Even the firewall itself cant "see" the IPMI IP Address.
(Supermicro X7SPE-HF, X7SPA-HF)
Could it be because the NIC is shared? One could try using the NIC as a dedicated IPMI port and see if that solves the issue.
Of course then he/she would loose a NIC port.. -
Create an OpenVPN interface by adding interface and then I think you will see two tabs for OpenVPN under Firewall rules. Set both to * * * * OPEN ALL basically.
Then reset all the states and try again.
To debug this, you can input the port numbers or IP of the IPMI in States under Diagnostics > States. You want to see MULTIPLE:MULTIPLE for each connection else you have a problem on one side if you have a SINGLE which you have to look into. It will for sure tell you where the problem is.
-
Here we go..
I didnt test it over a VPN connection, I simply tried to connect to the IPMI interface on the LAN side through a system on the OPT1 side.OPT1 has access to that IP which is confirmed in the firewall log
pass Jun 5 08:57:00 ORANGE 172.29.xxxx 10.xxxxx:80 TCP:SIn the states there are two entries
tcp 10.xxxxx:80 <- 172.29.xxxx CLOSED:SYN_SENT
tcp 172.29.xxxx -> 10.xxxx:80 SYN_SENT:CLOSEDThe IPMI interface by itself does not block any connections originating by different IP ranges. I have confirmed that with the manufacturer and a second IPMI interface I have that is not on the firewall is reachable (with the correct firewall rules)
-
In the states there are two entries
tcp 10.xxxxx:80 <- 172.29.xxxx CLOSED:SYN_SENT
tcp 172.29.xxxx -> 10.xxxx:80 SYN_SENT:CLOSEDThat just indicates you sent the SYN to open the connection, and nothing responded. Can confirm that by packet capturing on the internal interface. If the IP configuration, mask and gateway are definitely correct on the IPMI, and it has no IP restrictions, then it's just yet another embedded device that refuses to use its default gateway (which isn't extremely uncommon, though IMPI's generally work). Can source NAT the traffic destined to that host to work around using advanced outbound NAT.
-
on the LAN side:
09:40:19.582103 00:25:90:03:XX:XX > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Ethernet (len 6), IPv4 (len 4), Request who-has 10.XXX.XXX.XXX tell 10.XXX.XXX.1, length 28 09:40:22.576018 00:25:90:03:XX:XX > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Ethernet (len 6), IPv4 (len 4), Request who-has 10.XXX.XXX.XXX tell 10.XXX.XXX.1, length 28 09:40:28.572483 00:25:90:03:XX:XX > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Ethernet (len 6), IPv4 (len 4), Request who-has 10.XXX.XXX.XXX tell 10.XXX.XXX.1, length 28on the OPT side:
09:37:49.082986 52:54:00:77:XX:XX > 00:25:90:03:XX:XX, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 3586, offset 0, flags [DF], proto TCP (6), length 60) 172.XXX.XXX.XXX.55370 > 10.XXX.XXX.XXX.80: Flags [s], cksum 0xac95 (correct), seq 622805277, win 5840, options [mss 1460,sackOK,TS val 1421299487 ecr 0,nop,wscale 6], length 0 09:37:52.078438 52:54:00:77:XX:XX > 00:25:90:03:XX:XX, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 3587, offset 0, flags [DF], proto TCP (6), length 60) 172.XXX.XXX.XXX.55370 > 10.XXX.XXX.XXX.80: Flags [s], cksum 0xa9a7 (correct), seq 622805277, win 5840, options [mss 1460,sackOK,TS val 1421300237 ecr 0,nop,wscale 6], length 0 09:37:58.077919 52:54:00:77:XX:XX > 00:25:90:03:XX:XX, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 3588, offset 0, flags [DF], proto TCP (6), length 60) 172.XXX.XXX.XXX.55370 > 10.XXX.XXX.XXX.80: Flags [s], cksum 0xa3cb (correct), seq 622805277, win 5840, options [mss 1460,sackOK,TS val 1421301737 ecr 0,nop,wscale 6], length 0 What I did after was a packet capture from my workstation through a Site-to-Site OpenVPN connection to a IPMI interface that is _not_ on the firewall: [code]10:31:17.364698 00:1f:c6:e9:XX:XX > 00:25:90:03:XX:XX, ethertype IPv4 (0x0800), length 62: (tos 0x0, ttl 128, id 3326, offset 0, flags [DF], proto TCP (6), length 48) 10.106.XXX.XXX.1746 > 10.90.XXX.XXX.80: Flags [s], cksum 0x4652 (correct), seq 2007364705, win 64380, options [mss 1460,nop,nop,sackOK], length 0 10:31:17.498489 00:25:90:03:XX:XX > 00:1f:c6:e9:XX:XX, ethertype IPv4 (0x0800), length 62: (tos 0x0, ttl 62, id 0, offset 0, flags [DF], proto TCP (6), length 48) 10.90.XXX.XXX.80 > 10.106.XXX.XXX.1746: Flags [S.], cksum 0xe22f (correct), seq 3626201350, ack 2007364706, win 5840, options [mss 1352,nop,nop,sackOK], length 0 10:31:17.498675 00:1f:c6:e9:XX:XX > 00:25:90:03:XX:XX, ethertype IPv4 (0x0800), length 60: (tos 0x0, ttl 128, id 3329, offset 0, flags [DF], proto TCP (6), length 40) 10.106.XXX.XXX.1746 > 10.90.XXX.XXX.80: Flags [.], cksum 0x29db (correct), seq 1, ack 1, win 64380, length 0 10:31:17.499657 00:1f:c6:e9:XX:XX > 00:25:90:03:XX:XX, ethertype IPv4 (0x0800), length 394: (tos 0x0, ttl 128, id 3330, offset 0, flags [DF], proto TCP (6), length 380) 10.106.XXX.XXX.1746 > 10.90.XXX.XXX.80: Flags [P.], cksum 0x2e32 (correct), seq 1:341, ack 1, win 64380, length 340 10:31:17.638551 00:25:90:03:XX:XX > 00:1f:c6:e9:XX:XX, ethertype IPv4 (0x0800), length 54: (tos 0x0, ttl 62, id 24317, offset 0, flags [DF], proto TCP (6), length 40) 10.90.XXX.XXX.80 > 10.106.XXX.XXX.1746: Flags [.], cksum 0x0ae4 (correct), seq 1, ack 341, win 6432, length 0 As you can see, there all works perfectly.. Doesnt that mean that the IPMI does work correctly with the gateway setting? Trying a ping from the firewall on the LAN interface to the IPMI got this: [code]PING 10.XXX.XXX.9 (10.XXX.XXX.9) from 10.XXX.XXX.1: 56 data bytes --- 10.XXX.XXX.9 ping statistics --- 3 packets transmitted, 0 packets received, 100.0% packet loss[/code] So even on the same subnet the firewall cant ping the IPMI. From any other system on that network I can ping the IPMI I also spoke to Supermicro about this and they said that there are no restrictions. [/s][/code][/s][/s][/s] -
Do you have an OpenVPN interface? go to add interface and add OpenVPN interface by pressing + sign. If you see a + sign there you have an interface that is not configured like OpenVPN tunnel interface. Please follow my last post.
Then try the test again and go to Diagnostic > States and then put the IP of the IPMI and see if any requests come in. If not, that is because you are missing OpenVPN interface and firewall rules in it.
-
Do you have an OpenVPN interface? go to add interface and add OpenVPN interface by pressing + sign. If you see a + sign there you have an interface that is not configured like OpenVPN tunnel interface. Please follow my last post.
Then try the test again and go to Diagnostic > States and then put the IP of the IPMI and see if any requests come in. If not, that is because you are missing OpenVPN interface and firewall rules in it.
Do you think there is a chance it will work this way even though it did not for the tests above?
(setup above: LAN interface to OPT1 interface) -
It MUST work. I am not sure what OPT1 interface is as I don't know your hardware type.
Bridge OPT1 with LAN interface and it will work. Then separate the subnets and still should work. If your problem is reaching through OpenVPN then the right method is to create a tunnel interface and firewall it accordingly (also makes life easier for future vpn tunnel management)
-
How about not bridging or doing anything fancy on the VPN side at all.
Firewall > NAT - Outbound tab
Switch to Manual
Add a rule - LAN interface, source VPN network, destination IPMI IP, translate to the interface address.Save/apply, access it fine.
-
an OPT1 interface is just a third NIC on the firewall. first is WAN, second is LAN, third is OPT1 (generic pfsense name).
This way you can give your LAN 192.168.0.0/24 and your OPT1 interface 192.168.1.0/24 subnets. Over the firewall rules you can control access between the networks. Its basically the same as making an interface out of a ovpnsX or ovpncX adapter.In my test environment both interfaces have full * rules. Anyone can go anywhere. I can confirm that by reaching any system on either network. The exception is the IPMI interface.
My problem is that I cannot reach the IPMI over any sort of "tunnel" through the firewall. The interface is only reachable from within the network. Not even the firewall itself(!) can reach the interface.
That is very odd and from my knowledge cannot be explained by routing or firewall rule errors.I have the feeling that the reason for this not working is the fact that the IPMI interface is on the same NIC as LAN and for some reason the requests are not going out the firewall to get back to the same NIC on a different IP.
I cannot image it being a routing/firewall rules issue. After setting up some VPN networks, this is the only single IP that is always not reachable.
-
0
How about not bridging or doing anything fancy on the VPN side at all.Firewall > NAT - Outbound tab
Switch to Manual
Add a rule - LAN interface, source VPN network, destination IPMI IP, translate to the interface address.Save/apply, access it fine.
Shouldn't it be working without that if its not a VPN network but a second/third NIC on the same firewall?
(as long as there is a firewall rule allowing it) -
That depends on whether or not the IPMI is actually respecting its default gateway, but if it is a shared NIC you may be right it may just not be picking up the packets as they leave, and no amount of trickery on the firewall can help it. You might need to setup a simple bounce daemon on an internal server to reflect the ipmi port back, then connect from the vpn to that port on the internal server.
-
That depends on whether or not the IPMI is actually respecting its default gateway, but if it is a shared NIC you may be right it may just not be picking up the packets as they leave, and no amount of trickery on the firewall can help it. You might need to setup a simple bounce daemon on an internal server to reflect the ipmi port back, then connect from the vpn to that port on the internal server.
Thats exactly what I suspect aswell..
About not respecting the default gateway: Does it not show that its working when I can access the IPMI interface over a site-to-site VPN when the IPMI is not on the firewall itself but on a server within that network?
