Can't get port forward to work correctly.

[undertaker666]

@viragomann Those packets are rare, that's usually 0/0.
I check the “is my port open” website, and it showed up as open.

The BitTorrent client can download fine, and that traffic is not counted towards that NAT rule, however it will almost never upload.

I've seen suggestions to create an outbound NAT rule and try a lot of different things, but none of them worked.

I will also mention in case I forgot, I have Squid running MITM SSL Cache, Snort, and pfBlockerNG. But I don't see massive blocked traffic in the latter two.

[crucialguy]

@undertaker666 If an external port scan is seeing the port as open, then the NAT/Port Forward is operating ok, like viragomann said this points to a host problem.

With Torrent applications they can sometimes 'randomise' the port on startup, that's not happening here is it? Are you sure the application is set to use the same port every time it starts up?

I've never had to do anything more than you've done in order to get these sorts of scenarios working, just need to make sure the client it's self isn't trying to do UPnP or randomising the port on startup.

[undertaker666]

@crucialguy the problem is not with the host, the host was uploading fine before I switched to pfSense.

It is set to a specific port, otherwise I would not have set it to the number in the rule.

Again, you may be missing the point here, traffic in works fine, I can download at somewhat reasonable speeds, but there are no uploads happening at all.

[undertaker666]

@undertaker666 I forgot to mention, I also tried a few different torrent clients, all failed.

[viragomann]

@undertaker666 said in Can't get port forward to work correctly.:

the problem is not with the host, the host was uploading fine before I switched to pfSense.

Maybe your former router did masquerading on incoming forwarded packets.

Again, to get a step further sniff the traffic and show what you get.

[undertaker666]

@viragomann said in Can't get port forward to work correctly.:

Maybe your former router did masquerading on incoming forwarded packets.
Again, to get a step further sniff the traffic and show what you get.

Actually I didn't even have a port forward on my previous router, I had UPnP enabled on my client and everything worked.

Since moving to pfSense I disabled that, so that's not the cause.

As for sniffing, I'll try what you suggested and update this post.

[undertaker666]

@undertaker666

    92.14.118.168.34931 > 192.168.55.100.59372: [udp sum ok] UDP, length 1425
18:52:09.786523 ethertype IPv4 (0x0800), length 62: (tos 0x0, ttl 64, id 44038, offset 0, flags [none], proto UDP (17), length 48)
    192.168.55.100.59372 > 92.14.118.168.34931: [udp sum ok] UDP, length 20
18:52:09.787142 , ethertype IPv4 (0x0800), length 590: (tos 0x48, ttl 53, id 45721, offset 0, flags [none], proto UDP (17), length 576)
    181.214.206.157.46682 > 192.168.55.100.59372: [udp sum ok] UDP, length 548
18:52:09.787395  ethertype IPv4 (0x0800), length 70: (tos 0x0, ttl 64, id 32769, offset 0, flags [none], proto UDP (17), length 56)
    192.168.55.100.59372 > 181.214.206.157.46682: [udp sum ok] UDP, length 28
18:52:09.788122  ethertype IPv4 (0x0800), length 590: (tos 0x48, ttl 53, id 45722, offset 0, flags [none], proto UDP (17), length 576)
    181.214.206.157.46682 > 192.168.55.100.59372: [udp sum ok] UDP, length 548

Here's a capture of the traffic coming in to that machine on that port. TCP is non-existent, it's all UDP for some reason. Even though the client is set to use TCP and UTP (which is UDP).
Other than that, I don't see anything else out of this capture.

[viragomann]

@undertaker666
So the traffic seems to flow well. That's all pfSense can do.

Actually I didn't even have a port forward on my previous router, I had UPnP enabled on my client and everything worked.

Would be worth to mention. Maybe UPnP opens more than only one port.

I'm using a BitTorrent client behind pfSense. I've forwarded two ports, one is TCP for talking to other client, the other UDP for the tracker, and it works flawlessly.

However, it's also possible to enable UPnP on pfsense, but that's only recommended if you know what you do and restrict the access to known clients only.

[undertaker666]

@viragomann Yeah UPnP is not secure, and it's better to stick to just 1 port.

I use QBittorrent, and there's only one port that can be set, so both use the same port, but only UDP is used for some reason.

[undertaker666]

@undertaker666 OK, did a bit more digging, and found out that pfBlockerNG was ignoring the rule order, and was still blocking the traffic that matched that NAT Rule.

Turned it off, but still not seeding well, so did a port test from both pfSense and outside the network using an online service.

External service said the port was closed, pfSense said the port was closed when the destination was the WAN IP, but when it was the LAN IP it said it was open.

Now why would that be?

[lolipoplo]

@undertaker666

easiest way to debug, run tcpdump on pflog0, you are going to see all of the blocked packets and according to which rule they are blocked, if they are indeed blocked.

tcpdump -nettti pflog0 port 8010 and then run an external syn scan on 8010

[viragomann]

@undertaker666 said in Can't get port forward to work correctly.:

Turned it off, but still not seeding well

Maybe QBittorrent needs the outbound port to be static.
Many consumer routers does this, but pfSense use random outbound ports by default. You may have to add an outbound NAT rule to achieve this.

pfSense said the port was closed when the destination was the WAN IP, but when it was the LAN IP it said it was open.

You cannot use the port check on pfSense for the firewall itself. You can only check other destinations.

[lolipoplo]

@viragomann if you have port forwarding working, outbound NAT doesn't matter. Making it static just helps with hole punching

[viragomann]

@lolipoplo said in Can't get port forward to work correctly.:

if you have port forwarding working, outbound NAT doesn't matter.

Some programs need this like several games. Maybe QBittorrent as well. I don't know how it works, as I mentioned above.
But it's for sure that QBittorrent also make upstream connections and these have nothing to do with forwarding at all.

So a presume, you're knowing well QBittorrent and can possibly give more reliable infos.

[undertaker666]

@viragomann @lolipoplo said in Can't get port forward to work correctly.:

tcpdump -nettti pflog0 port 8010 and then run an external syn scan on 8010

Actually, once I turned off pfBlockerNG, parsec managed to connect to a host game. So those ports are fine.

The problem is with the torrent, it's better, it's actually seeding now, and it actually reached 300 KB/s, but it does not stay at those speeds, and there's more downtime than actual seeding.

Maybe QBittorrent as well

Well, I already had outbound set up, so that's not what's stopping it. The question is why is pfBlockerNG ignoring the rule order, and even with it turned off, why are connections not sticking as they used to?

[lolipoplo]

@undertaker666
So have you tried pflog as I suggested?

Actually run tcpdump on wan port at the same time to compare incoming and rule matching

[undertaker666]

@lolipoplo No, because you suggested running the tcpdump on the parsec port, and that is solved by just turning off pfBlockerNG.

I could run the tcpdump on the qbittorrent port and see what happens.

[undertaker666]

@undertaker666

tcpdump -nettti pppoe0 port 59372 -vv
tcpdump: listening on pppoe0, link-type NULL (BSD loopback), capture size 262144 bytes
 00:00:00.000000 AF IPv4 (2), length 56: (tos 0x2,ECT(0), ttl 63, id 28582, offset 0, flags [none], proto TCP (6), length 52)
    My-WAN.31533 > 172.16.1.0.59372: Flags [SEW], cksum 0xbecc (correct), seq 2428356104, win 62720, options [mss 8960,nop,wscale 8,nop,nop,sackOK], length 0
 00:00:00.013465 AF IPv4 (2), length 48: (tos 0x2,ECT(0), ttl 254, id 6409, offset 0, flags [DF], proto TCP (6), length 44)
    172.16.1.0.59372 > My-WAN.31533: Flags [S.], cksum 0x9772 (correct), seq 4044710442, ack 2428356105, win 4200, options [mss 1400], length 0
 00:00:00.000241 AF IPv4 (2), length 44: (tos 0x0, ttl 63, id 28583, offset 0, flags [none], proto TCP (6), length 40)
    My-WAN.31533 > 172.16.1.0.59372: Flags [.], cksum 0xc3ca (correct), seq 1, ack 1, win 64400, length 0
 00:00:00.003992 AF IPv4 (2), length 205: (tos 0x0, ttl 63, id 28584, offset 0, flags [none], proto TCP (6), length 201)
    My-WAN.31533 > 172.16.1.0.59372: Flags [P.], cksum 0xc832 (correct), seq 1:162, ack 1, win 64400, length 161
 00:00:00.302961 AF IPv4 (2), length 205: (tos 0x0, ttl 63, id 28585, offset 0, flags [none], proto TCP (6), length 201)
    My-WAN.31533 > 172.16.1.0.59372: Flags [P.], cksum 0xc832 (correct), seq 1:162, ack 1, win 64400, length 161
 00:00:00.394904 AF IPv4 (2), length 205: (tos 0x0, ttl 63, id 28586, offset 0, flags [none], proto TCP (6), length 201)
    My-WAN.31533 > 172.16.1.0.59372: Flags [P.], cksum 0xc832 (correct), seq 1:162, ack 1, win 64400, length 161
 00:00:00.699004 AF IPv4 (2), length 205: (tos 0x0, ttl 63, id 28587, offset 0, flags [none], proto TCP (6), length 201)
    My-WAN.31533 > 172.16.1.0.59372: Flags [P.], cksum 0xc832 (correct), seq 1:162, ack 1, win 64400, length 161
 00:00:00.589524 AF IPv4 (2), length 56: (tos 0x2,ECT(0), ttl 63, id 36468, offset 0, flags [none], proto TCP (6), length 52)
    My-WAN.60033 > 172.16.2.0.59372: Flags [SEW], cksum 0x442d (correct), seq 3544223184, win 62720, options [mss 8960,nop,wscale 8,nop,nop,sackOK], length 0
 00:00:00.013934 AF IPv4 (2), length 48: (tos 0x2,ECT(0), ttl 254, id 9436, offset 0, flags [DF], proto TCP (6), length 44)
    172.16.2.0.59372 > My-WAN.60033: Flags [S.], cksum 0x3e54 (correct), seq 3004006065, ack 3544223185, win 4200, options [mss 1400], length 0
 00:00:00.000178 AF IPv4 (2), length 44: (tos 0x0, ttl 63, id 36469, offset 0, flags [none], proto TCP (6), length 40)
    My-WAN.60033 > 172.16.2.0.59372: Flags [.], cksum 0x6aac (correct), seq 1, ack 1, win 64400, length 0
 00:00:00.005371 AF IPv4 (2), length 497: (tos 0x0, ttl 63, id 36470, offset 0, flags [none], proto TCP (6), length 493)
    My-WAN.60033 > 172.16.2.0.59372: Flags [P.], cksum 0x62af (correct), seq 1:454, ack 1, win 64400, length 453
 00:00:00.385924 AF IPv4 (2), length 497: (tos 0x0, ttl 63, id 36471, offset 0, flags [none], proto TCP (6), length 493)
    My-WAN.60033 > 172.16.2.0.59372: Flags [P.], cksum 0x62af (correct), seq 1:454, ack 1, win 64400, length 453
 00:00:00.304076 AF IPv4 (2), length 205: (tos 0x0, ttl 63, id 28593, offset 0, flags [none], proto TCP (6), length 201)
    My-WAN.31533 > 172.16.1.0.59372: Flags [P.], cksum 0xc832 (correct), seq 1:162, ack 1, win 64400, length 161
 00:00:00.095936 AF IPv4 (2), length 497: (tos 0x0, ttl 63, id 36472, offset 0, flags [none], proto TCP (6), length 493)
    My-WAN.60033 > 172.16.2.0.59372: Flags [P.], cksum 0x62af (correct), seq 1:454, ack 1, win 64400, length 453
 00:00:00.218016 AF IPv4 (2), length 44: (tos 0x0, ttl 254, id 10875, offset 0, flags [DF], proto TCP (6), length 40)
    172.16.1.0.59372 > My-WAN.31533: Flags [R.], cksum 0xbf57 (correct), seq 1, ack 1, win 0, length 0
 00:00:00.476972 AF IPv4 (2), length 497: (tos 0x0, ttl 63, id 36477, offset 0, flags [none], proto TCP (6), length 493)
    My-WAN.60033 > 172.16.2.0.59372: Flags [P.], cksum 0x62af (correct), seq 1:454, ack 1, win 64400, length 453
 00:00:01.204971 AF IPv4 (2), length 497: (tos 0x0, ttl 63, id 36480, offset 0, flags [none], proto TCP (6), length 493)
    My-WAN.60033 > 172.16.2.0.59372: Flags [P.], cksum 0x62af (correct), seq 1:454, ack 1, win 64400, length 453
 00:00:00.323059 AF IPv4 (2), length 44: (tos 0x0, ttl 254, id 13669, offset 0, flags [DF], proto TCP (6), length 40)
    172.16.2.0.59372 > My-WAN.60033: Flags [R.], cksum 0x6639 (correct), seq 1, ack 1, win 0, length 0

Here's a small output of that command.

The weird thing is, when I do canyouseeme, it shows me another public IP than what is shown in the WAN interface on my pfSense box.
Both are public IPs, and I tested both, and now they say closed.

[lolipoplo]

@undertaker666

if you aren't willing to do tcpdump on pflog0 you can't see how your packets get blocked

[undertaker666]

@lolipoplo Look up