"Google 1e100 addresses" & Google invaled certificates "Common Name invalid2.invalid"

louis2

More and more my firewall (pfSense), is flooded by tcp-streams towards "Google 1e100 addresses" resulting in lots of error messages related to those tcp-tranfers. All ending with as reason "RA / PA / FPA".

To an unknown extend this problem is related to an invalid certificate problem, where the invalid certificate with as common factor "Common Name invalid2.invalid".

What I know, however which does not clarify the issue is
"1e100.net is a Google-owned domain name used to identify the servers in our network"

https://www.spinics.net/lists/squid/msg93771.html
Many google services IP addresses returns invalid2.invalid CN and Error negotiating SSL connection on FD

Two problems with this:

what (&why) is happening here !?
(and why that many errors?)
it is flooding my message log, in which the important messages "disappear"

stephenw10

Certificate errors have nothing to do with blocked traffic entries in the firewall log, which is what I assume you are referring to.
TCP ack flagged traffic like that is blocked when the state that initially allowed it has been closed. You would expect to see some blocked traffic like that in a normal firewall.

https://docs.netgate.com/pfsense/en/latest/troubleshooting/log-filter-blocked.html

Are you running Squid?

Steve

louis2

@stephenw10

A few clarification:

I am not using any "intrusion detection packages"
the traffic type towards the involved destinations is https / 443
There is no filter towards the involved broad rage of destinations

The traffic just fails and that with two/three common factors:

the IP-addresses are all google 1e100 addresses
if I take one of these addresses and enter it in the browser, I see that the connection is refused due to an invalid certificate type "Common Name invalid2.invalid"
I also note that there is a lot of traffic towards that kind "google 1e100" destinations, not so clear for which reason. So given the amount of traffic, to a certain extend you can expect it to fail (some times)

About my firewall rule sets:

There is a ruleset for each vlan and at the end of the rule set there is a rule "what did I block" with logging to show what has been unintentionally blocked
according to that principle, I tried to add a rule to stop this records to enter the log ...... I have no idea how to do that ......
In other situations or perhaps even in this one ..... I would like to have the option to only log a summary of this kind of repeating problems (destination not reachable 100 times in state 100 messages)

johnpoz

@louis2 said in "Google 1e100 addresses" & Google invaled certificates "Common Name invalid2.invalid":

kind "google 1e100" destinations,

What is the website your trying to go to? That is not a fqdn you would go that is ptr for a google owned IP would be where you would see that.

Where you got that 1e100.net is google owned - states as well.
"Most typical Internet users will never see 1e100.net"

Trying to go to a fqdn that is hosted on google server, ptr would come back with that fqdn on it. But you shouldn't be going to that fqdn..

;; QUESTION SECTION:
;www.google.com.                        IN      A

;; ANSWER SECTION:
www.google.com.         1041    IN      A       172.217.4.36

;; QUESTION SECTION:
;36.4.217.172.in-addr.arpa.     IN      PTR

;; ANSWER SECTION:
36.4.217.172.in-addr.arpa. 86400 IN     PTR     lga15s46-in-f36.1e100.net.
36.4.217.172.in-addr.arpa. 86400 IN     PTR     lga15s46-in-f4.1e100.net.
36.4.217.172.in-addr.arpa. 86400 IN     PTR     ord38s18-in-f4.1e100.net.

What is not working? What specific fqdn are you trying to reach? You shouldn't be trying to go to https://1e100.net anything.. So yeah there would be an invalid cert with that fqdn..

RA PA FPA are out of state - so yeah pfsense would block those...

How about you actually post up your firewall log to what your actually seeing.. And we can try and work out what is the actual problem. But you would prob never hit some fqdn 1e100.net in your browser..

louis2

@johnpoz

John, I see this issue at the moment only in conjunction with android devices (phones and tablets).

Those devices are trying to reach there ^boss^, without any action or influence from my site, even if those devices are supposed to sleep.

The addresses which they try to reach (for unkown reason) are often in the google or amazon cloud.

Of course I do not like computer programs to access the outside world out of my control but that apart.

So trying to understand and solve the issue I did make some traces and screenshots.

The "sleeping" tablet causing this in this example

The wireshark trace

The pfsense log

So,

how to get rid of this frequently occurring problem
how to get rid of the alarms polluting the logs
(note I would like to see it, but e.g. only once in e.g. 15 min, with a count value or so)

johnpoz

@louis2 said in "Google 1e100 addresses" & Google invaled certificates "Common Name invalid2.invalid":

how to get rid of this frequently occurring problem

I just wouldn't log that noise.. Such traffic from sleeping devices is the devices problem.. Device A creates a session to somewhere that is allowed, ie state is created..

As long as that state is active the connection is allowed. Now if this device say goes to sleep and couple of hours later wakes up and still thinks it can use that same session, its going to fail with a out of state block like that PA, FPA - those are all acks and firewall blocks because there is no state. State is created from a SYN [S]

Yeah this is common sort of log that can show up with say wireless devices, that switch from say cell to wireless and try and maintain same session. Or devices that go to sleep and try and wake up and reuse some old session.

If something wants to talk to something else, and old session fails - it will create a new session, ie it will send syn. What your seeing almost always points to out of state sessions, or asymmetrical traffic - where pfsense didn't see the start of the conversation, ie the syn. But those are normally seen as SYN,ACK blocks [SA].

If your concern is just all the blocks in the log, then don't log them. You can just log SYN traffic if you want. I log that on my wan. This is not really a pfsense issue, this is a stateful firewall thing with devices too stupid to think hey I haven't used this session in X amount of time.. Should prob just start a new one.

You could up the timeout on sessions, but these could lead to state exhaustion and then you got bigger problems. I don't log default blocks for pretty much this very reason - keep my logs cleaner and easier to read.. If I feel maybe something is being blocked, I can always turn default logging back on with a click to troubleshoot.

I log only syn and some common udp ports on my wan - because its interesting to see such traffic. I log specific rules, some allowed some blocked for the traffic I want to see. But I don't care at all if my phone thinks it can try and use some old session. If its wants to talk to the mother ship and its old session isn't working - then it will try a new session, etc.

stephenw10

Yeah, you could try setting the firewall optimisation to conservative if you want to allow that traffic:
https://docs.netgate.com/pfsense/en/latest/config/advanced-firewall-nat.html#firewall-optimization-options

Or you can block it without logging if you really don't want to see it.

As stated though it's not really a problem.

You can always just disable wifi in standby in Android.

Steve

louis2

@johnpoz, @stephenw10,

I tried to get rid of the messages by adding two rules at the end of the firewall. The disadvantage of all those messages, is regretfully more important than the extra info ....

It looks like this now

The first row should filter on TCP-S and log
The second row is there to show for test purposes, to log what is blocked <> TCP-S
The third row is my normal end-rule showing what has not on purpose being blocked.

One problem .... it does not work ..... the first two rules, do not capture any passing tcp frames ...... which are there ....

In the first row I am doing something which might be wrong ....
I tried this, but I am not sure it is correct ...

Below part of the logging showing that the failing tcp-package did not trigger one of the tcp-rules ..... but the final blocking rule ........

So:

why is the second rule selecting on just TCP not triggered !?
is the rule intended to filter tcp-s correct !?

stephenw10

The first rule should never match anything. New connections, TCP:SYN, should always be passed.

You want to block without logging TCP:ACK.

Steve

louis2

@stephenw10

Stephen,

I just not know, in principle everything which is allowed to pass is/should be passed via rules above these final rules.

I would like to log all things which are not passed by the rules above these end rules, with the exception of the tcp-rules having states like "RA / PA / FPA"

So for that reason three end rules:

first one intended to log tcp-S (because I would like to see what I perhaps should have passed)
the second rule is "not to log" the rest of the blocked tcp (the RA / PA / FPA etc). (for this test the log is on)
and the final rule is to log the rest (not being tcp)

As said my problem is that the first two rules, intended to deal with TCP, do not work !! WHY O WHY !?

stephenw10

What does the second rule actually look like?

The first rule should only match anything if the pass rules above are wrong. So that fact it isn't is a good thing!

You either need one rule to block anything TCP:ACK or you need three rules for each of those flah combinations. You can't match all three specific combinations with one rule.

Steve

louis2

@stephenw10

first rule

second rule

third rule

stephenw10

There are no flags set on that second rule. So it won't match.

Change the first rule from SYN to ACK. Remove the second rule.

louis2

@stephenw10

..... I do not understand any thing related to rules which have to capture a certain tcp state ....

I tried

three rules in a row where I did expect the first rule to trigger on tcp packages having "RA / PA / FPA"

I used

there and did also test with ^out of "ack"^. In both cases nothing triggered the rule

So I tried the second rule, setting ^TCP Flags^ to ^Any flags^. This rule seems to fetch the FA etc.

However, I really do not understand

stephenw10

Ah, OK. No that needs to be this:

Screenshot from 2022-05-26 13-47-40.png

It means: The ACK flag must be set, only check the ACK flag.

The rule that you have says: check all flags and match packets that have only ACK set. Which wouldn't include any of the blocked traffic you were seeing that all has multiple flags set.

Steve

louis2

@stephenw10

I am using these two rules now, which seems to work

First rule is defined like this (I did try that before, but not likely good enough)

Still wondering what "Any Flag" is supposed to do

johnpoz

@louis2 I personally wouldn't do it that way ;)

I would just turn off the default log. This removes all the other unwanted log stuff like multicast broadcasts and the like, stuff from link-local addresses, etc.

And then create a rules at the end to block, and only log stuff that has syn set. And then a rule to log any sort of common udp ports you want to see..

stephenw10

The 'Any flags' box will cause it to match TCP packets with any combination of flags. That means you can pass traffic that is asymmetric for example.

Steve

louis2

@stephenw10

The Filter TCP-ACK rule works, however ....... does change the firewall behavoir in a couple of ways. Suddenly new loggings occur which where not there before.
Conclusion is that this rule must be changing some FW internal state table ....

Let me start with the logging above. I did never see that message before, but apart from that, it is communication between two devices in the same subnet. And communication within one subnet normally does not pass the FW. Where this is of course a strange situation, since one of the communication parties is the FW itself.

Here we see a second effect. A message from the floating rule set "Default deny rule ipv4".

Here a set of the same "Default deny rule ipv4", however it is different since the source is somewhere on the internet (google / amazon)

Switching off the TCP-ACK filter rule stops this "new" FW behavoir, we are back to the same situation as before.

So this TCP-ACK filter, has unexpected and unwanted side effects .......

stephenw10

I suggest that all of those are because the IP you're testing from hit locked out of the firewall dues to excessive login attempts and the it's existing states were cleared. That applies before the user rules so it still hit and logged.

The arrow there shows it was blocked outbound on PCLAN_1G whicb is almost always out-of-state traffic because the state was closed.

The extra rule you have added does nothing more than block some traffic without logging before it hits your block everything rule anyway.

Steve