Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Rancher node ipsec behind pfsense

    Scheduled Pinned Locked Moved Routing and Multi WAN
    1 Posts 1 Posters 283 Views 1 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • M Offline
      mrgizmo
      last edited by mrgizmo

      Hello,
      I have a system using rancher 1.6 and pfsense routers.

      I have 3 datacenters, each one has one pfsense and they are connected each-other by an ipsec connection.

      Behind each pfsense I have a rancher node machine. Because Rancher create a ipsec connection between nodes, on each pfsense I have NAT rules in ports 500 and 4500 to forward traffic for the right node. Everything works great.

      Now a add a second node on one of the datacenters, behind the pfsense that were there before, I have created a second public ip in pfsense (virtual ip) and create the same NAT rules, from that node I can ping the other nodes on the other datacenters but not the node on the same datacenter, I've done lots of tests and it something on the pfsense that is blocking, anyone has a clue on what I'm missing or doing wrong here?
      rancherpfsense.png

      Note: The IP's are representative

      		Interface 	Protocol 	Source Address 	Source Ports 	Dest. Address 	Dest. Ports 	NAT IP 	NAT Ports 	Description 

		WAN 	UDP 	* 	* 	1.1.1.1 	500 (ISAKMP) 	192.168.1.1 	500 (ISAKMP) 	rancher-node-1 	
		WAN 	UDP 	* 	* 	1.1.1.2 	500 (ISAKMP) 	192.168.1.4 	500 (ISAKMP) 	rancher-node-4 	
		WAN 	UDP 	* 	* 	1.1.1.1 	4500 (IPsec NAT-T) 	192.168.1.1 	4500 (IPsec NAT-T) 	rancher-node-1 	
		WAN 	UDP 	* 	* 	1.1.1.2 	4500 (IPsec NAT-T) 	192.168.1.4 	4500 (IPsec NAT-T) 	rancher-node-4

      Outbound Mappings
      Interface Source Source Port Destination Destination Port NAT Address NAT Port Static Port Description Actions

      	WAN 	192.168.1.1/32 	* 	* 	* 	1.1.1.1 	* 		rancher-node-1 	
	WAN 	192.168.1.4/32 	* 	* 	* 	1.1.1.2 	* 		rancher-node-4

      In pfsense1 ipsec log I see messages like this:
      Jun 5 12:35:35 charon 13[NET] received packet: from 192.168.1.1[1024] to 1.1.1.2[500] (500 bytes)
      Jun 5 12:35:35 charon 13[NET] <291> received packet: from 192.168.1.1[1024] to 1.1.1.2[500] (500 bytes)
      Jun 5 12:35:35 charon 13[ENC] parsed IKE_SA_INIT request 0 [ SA KE No N(NATD_S_IP) N(NATD_D_IP) N(FRAG_SUP) N(HASH_ALG) N(REDIR_SUP) ]
      Jun 5 12:35:35 charon 13[ENC] <291> parsed IKE_SA_INIT request 0 [ SA KE No N(NATD_S_IP) N(NATD_D_IP) N(FRAG_SUP) N(HASH_ALG) N(REDIR_SUP) ]
      Jun 5 12:35:35 charon 13[CFG] looking for an IKEv2 config for 1.1.1.2...192.168.1.1
      Jun 5 12:35:35 charon 13[CFG] <291> looking for an IKEv2 config for 1.1.1.2...192.168.1.1
      Jun 5 12:35:35 charon 13[IKE] no IKE config found for 1.1.1.2...192.168.1.1, sending NO_PROPOSAL_CHOSEN
      Jun 5 12:35:35 charon 13[IKE] <291> no IKE config found for 1.1.1.2...192.168.1.1, sending NO_PROPOSAL_CHOSEN
      Jun 5 12:35:35 charon 13[ENC] generating IKE_SA_INIT response 0 [ N(NO_PROP) ]
      Jun 5 12:35:35 charon 13[ENC] <291> generating IKE_SA_INIT response 0 [ N(NO_PROP) ]
      Jun 5 12:35:35 charon 13[NET] sending packet: from 1.1.1.2[500] to 192.168.1.1[1024] (36 bytes)
      Jun 5 12:35:35 charon 13[NET] <291> sending packet: from 1.1.1.2[500] to 192.168.1.1[1024] (36 bytes)
      Jun 5 12:35:35 charon 13[IKE] IKE_SA (unnamed)[291] state change: CREATED => DESTROYING
      Jun 5 12:35:35 charon 13[IKE] <291> IKE_SA (unnamed)[291] state change: CREATED => DESTROYING
      Jun 5 12:36:34 charon 14[NET] received packet: from 192.168.1.4[1024] to 1.1.1.1[500] (500 bytes)
      Jun 5 12:36:34 charon 14[NET] <292> received packet: from 192.168.1.4[1024] to 1.1.1.1[500] (500 bytes)
      Jun 5 12:36:34 charon 14[ENC] parsed IKE_SA_INIT request 0 [ SA KE No N(NATD_S_IP) N(NATD_D_IP) N(FRAG_SUP) N(HASH_ALG) N(REDIR_SUP) ]
      Jun 5 12:36:34 charon 14[ENC] <292> parsed IKE_SA_INIT request 0 [ SA KE No N(NATD_S_IP) N(NATD_D_IP) N(FRAG_SUP) N(HASH_ALG) N(REDIR_SUP) ]
      Jun 5 12:36:34 charon 14[CFG] looking for an IKEv2 config for 1.1.1.1...192.168.1.4
      Jun 5 12:36:34 charon 14[CFG] <292> looking for an IKEv2 config for 1.1.1.1...192.168.1.4
      Jun 5 12:36:34 charon 14[IKE] no IKE config found for 1.1.1.1...192.168.1.4, sending NO_PROPOSAL_CHOSEN
      Jun 5 12:36:34 charon 14[IKE] <292> no IKE config found for 1.1.1.1...192.168.1.4, sending NO_PROPOSAL_CHOSEN
      Jun 5 12:36:34 charon 14[ENC] generating IKE_SA_INIT response 0 [ N(NO_PROP) ]
      Jun 5 12:36:34 charon 14[ENC] <292> generating IKE_SA_INIT response 0 [ N(NO_PROP) ]
      Jun 5 12:36:34 charon 14[NET] sending packet: from 1.1.1.1[500] to 192.168.1.4[1024] (36 bytes)
      Jun 5 12:36:34 charon 14[NET] <292> sending packet: from 1.1.1.1[500] to 192.168.1.4[1024] (36 bytes)
      Jun 5 12:36:34 charon 14[IKE] IKE_SA (unnamed)[292] state change: CREATED => DESTROYING
      Jun 5 12:36:34 charon 14[IKE] <292> IKE_SA (unnamed)[292] state change: CREATED => DESTROYING

      1 Reply Last reply Reply Quote 0
      • First post
        Last post
      Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.