Microsoft 365 and pfSense

johnpoz

@mohkhalifa said in Microsoft 365 and pfSense:

and IPv6 in Alias

Not seeing this, how exactly are you trying to add them?

Once you put the alias in a rule - it should populate your table

Your wild cards like that are going to be almost impossible in an "alias" since what should it resolve to find an IP? A wild card like that is possible with dns, what you allow or block. So if say whatever.domain queried for, then it can either be looked up - or not.

But in an alias that does a query for a fqdn to get an IP or list of IPs how would that ever work? What does it query exactly to find the IP(s)? There is almost infinite possibilities that could fall under *.domain.tld since something.domain.tld, or other.whatever.something.domain.tld all fall under that wildcard.. With almost endless combinations.

Now with a query type of ANY you use to be able to get back any records that fall under that domain. But ANY type query has been deprecated for quite some time.
https://blog.cloudflare.com/what-happened-next-the-deprecation-of-any/

That is dated from back in 2016

Are you sure those are needed if you allow the IP ranges? My take on such a list is the domains are listed for dns type queries or proxy access, while the IP netblocks are given for what possible IPs something queried might come back with so you can allow the IPs

example:
the outlook.office.com since that is something I can actually query comes back in the 52.96.0.0/14 range given. What would be the point of a list of cidr blocks to allow, if the what you queried for came back with an IP outside of any of those ranges?

;; QUESTION SECTION:
;outlook.office.com.            IN      A

;; ANSWER SECTION:
outlook.office.com.     3473    IN      CNAME   substrate.office.com.
substrate.office.com.   3474    IN      CNAME   outlook.ha.office365.com.
outlook.ha.office365.com. 3474  IN      A       52.96.163.50
outlook.ha.office365.com. 3474  IN      A       52.96.79.178
outlook.ha.office365.com. 3474  IN      A       52.96.79.242
outlook.ha.office365.com. 3474  IN      A       52.96.164.178
outlook.ha.office365.com. 3474  IN      A       52.96.79.194
outlook.ha.office365.com. 3474  IN      A       52.96.226.162
outlook.ha.office365.com. 3474  IN      A       52.96.191.2
outlook.ha.office365.com. 3474  IN      A       52.96.226.178

For ipv6 it comes back in the 2603:1036::/36 listed

;; QUESTION SECTION:
;outlook.office.com.            IN      AAAA

;; ANSWER SECTION:
outlook.office.com.     3331    IN      CNAME   substrate.office.com.
substrate.office.com.   3332    IN      CNAME   outlook.ha.office365.com.
outlook.ha.office365.com. 3600  IN      CNAME   outlook.ms-acdc.office.com.
outlook.ms-acdc.office.com. 3600 IN     CNAME   MDW-efz.ms-acdc.office.com.
MDW-efz.ms-acdc.office.com. 3600 IN     AAAA    2603:1036:304:83f::2
MDW-efz.ms-acdc.office.com. 3600 IN     AAAA    2603:1036:304:2853::2
MDW-efz.ms-acdc.office.com. 3600 IN     AAAA    2603:1036:304:285a::2
MDW-efz.ms-acdc.office.com. 3600 IN     AAAA    2603:1036:304:284c::2

BTW? Are you actually using IPv6 - if not then none of those IPv6 ranges would be needed.

Now the biggest trick to these - is those can change. I believe they publish the updated list every 30 days or something.. So you will either have to manually adjust your cidr lists and possible dns queries allowed if going that route - or you will have to automate it. Aliases does have the ability of updating on a schedule a list of ips.. Pfblocker prob come in handy for something like that as well. Will either of those methods work with how MS publishes these lists - not sure..

You can see from the changelog they publish that they are making changes

<rss version="2.0">
  <channel>
    <title>Microsoft 365 URLs and IP Addresses - Office 365 Worldwide</title>
    <link>http://aka.ms/ipurlws</link>
    <description />
    <language>en-us</language>
    <lastBuildDate>Thu, 29 Jul 2021 00:00:00 Z</lastBuildDate>
    <item>
      <guid isPermaLink="false">2021072900</guid>
      <link>https://endpoints.office.com/changes/Worldwide/2021072900?singleVersion=true&amp;clientRequestId=b10c5ed1-bad1-445f-b386-b919946339a7</link>
      <title>2021072900 - Worldwide</title>
      <description>Version 2021072900 includes 3 changes. URLs: 2 added and 1 removed.</description>
      <pubDate>Thu, 29 Jul 2021 00:00:00 Z</pubDate>
    </item>
    <item>
      <guid isPermaLink="false">2021062800</guid>
      <link>https://endpoints.office.com/changes/Worldwide/2021062800?singleVersion=true&amp;clientRequestId=b10c5ed1-bad1-445f-b386-b919946339a7</link>
      <title>2021062800 - Worldwide</title>
      <description>Version 2021062800 includes 3 changes. URLs: 0 added and 4 removed. 7 other attributes changed.</description>
      <pubDate>Mon, 28 Jun 2021 00:00:00 Z</pubDate>
    </item>
    <item>
      <guid isPermaLink="false">2021052800</guid>
      <link>https://endpoints.office.com/changes/Worldwide/2021052800?singleVersion=true&amp;clientRequestId=b10c5ed1-bad1-445f-b386-b919946339a7</link>
      <title>2021052800 - Worldwide</title>
      <description>Version 2021052800 includes 4 changes. IPs: 2 added and 0 removed. URLs: 0 added and 6 removed.</description>
      <pubDate>Fri, 28 May 2021 00:00:00 Z</pubDate>
    </item>

mer

@gertjan Yep, I simply pulled the port 25 directly from his list where it seems like the intent is "allow my LAN clients to only do SMTP to the following destinations" (simply wrote it the other way in a default deny)
:)

mohkhalifa

Dear All,

Thanks for your kind care and try helping me. As @Gertjan said I'm limiting users for using the internet as I need to open the desired URLs and IPs for my LAN interface using MS 365.

@johnpoz we aren't using IPv6 in our environment.

So, I need to migrate very soon and don't know How I can solve this issue as Microsoft said, I need to allow all the following in their article

https://docs.microsoft.com/en-us/microsoft-365/enterprise/urls-and-ip-address-ranges?view=o365-worldwide

Thank you all and waiting your recommendations

johnpoz

@mohkhalifa said in Microsoft 365 and pfSense:

don't know How I can solve this issue as Microsoft said

I gave you all the info you need.. Since your not using IPv6 - you need to allow for the IPv4 ranges listed.. Is that simple.

If your doing any dns filtering.. Allow for those domains to be queried.

fqdn and domains are for dns/proxy use - IP ranges are for firewalls. It is not possible to allow a firewall to to resolve a wildcard domain to IPs. There is no firewall that could do this. A "firewall" doesn't see the dns query.. It only sees traffic to an IP on a port..

For such a thing to work, a UTM would also need to be running the dns. "AND" within the couple of ms since a client did a query for said fqdn, that would have to be put into the firewall rules to allow or not.

mohkhalifa

@johnpoz Thanks for your reply. I will try and let you know

johnpoz

@mohkhalifa just to go into a bit more detail on this. Even our big boy/name Multi K $ NG firewalls in our DCs do not support such a function of filtering on DNS name.. Since this is not really how firewalls function. Even though customers do ask for such nonsense quite often..

Proxies do that.. While yes many firewalls allow for aliases based on a fqdn they can query on some sort of timed schedule. And yeah sure many allow for loading a set of IPs or netblocks from some file hosted somewhere that change.

As I went over a wildcard domain would never be possible to use to populate a table of IPs or CIDR.. Since those are impossible to query before hand and get an IP or list of IPs..

When a client wants to go to some site on the public internet - it doesn't send that as a fqdn, it sends that to the gateway as IP:PORT or IP protocol like with icmp, etc.

Now while it "might" be possible for some UTM sort of device where clients as their gateway also use that devices IP as dns, and via some magic this device says ok client X from IP just did a query for sljfdsljsdlf.wildcardallowed.domain.tld which resolve to IP 1.2.3.4 - allow that.. I have never seen such a beast. MS would be having a really hard time getting their stuff to work if that was what was required for anyone running typical firewall..

What happens when the gateway is not the firewall? Most companies that use office365, can almost promise you there is a router internal to the network that clients point to for their gateway, which may or may not firewall internal networks and even external traffic - there is for sure going to be an actual edge firewall.. Many companies the client for sure is not using the firewall as their dns. MS shops for sure - can promise you most of those are pointing to their AD DNS, so how would the edge firewall know where the client actually wanted to go other than IP:Port?

Those fqdn and wildcards are listed for proxy and dns to use, while the cidr are given for firewalls that work based IP... Proxies for example are never really sent an IP, they are sent the fqdn the browser or application want to go.. The proxy then looks up where that fqdn resolves to and the proxy goes there for the client. Sure they can also work if client tries to go to https://1.2.3.4 - but have seen some balk at that sort of url.. Unless IPs are specifically allowed for by the proxy.

That is why proxies can use wildcard domains.. Because they can say oh client is looking for bla.bla.whatever.something.domain.tld, yeah that is allowed by *.domain.tld.. It then looks up bla.bla.whatever.something.domain.tld gets the specific IP for that and then the proxy goes there.. If that IP is allowed by the firewall upstream of the proxy that is ;)

mohkhalifa

Dear All,

Microsoft created a JSON file includes all M365 firewall rules. Is there any idea to add it to pfSense instead-of creating them manually as they are too much.

Thank you

https://endpoints.office.com/endpoints/worldwide?clientrequestid=b10c5ed1-bad1-445f-b386-b919946339a7

johnpoz

@mohkhalifa said in Microsoft 365 and pfSense:

https://endpoints.office.com/endpoints/worldwide?clientrequestid=b10c5ed1-bad1-445f-b386-b919946339a7

you can for sure take all of those networks and put them into a alias and allow..

But those ones that list *.domain.tld are impossible in a L3 firewall.. That ends up being almost infinite amount of IPs that could be resolved.. How could you put those into a list of networks or IPs?

Such listings only make sense for dns or proxy sort of blocking.

Notice says use our proxy PAC file ;)

mohkhalifa

@johnpoz Thanks for your reply. Can you advise with a good Proxy Virtual appliance ?

johnpoz

@mohkhalifa there are few proxy users on here, squid could be used like this.

But you can also just whitelist the networks listed.. Unless you were specifically blocking domains, ie already running a proxy - most of the domains they say to whitelist wouldn't be required.

tinfoilmatt

@mohkhalifa said in Microsoft 365 and pfSense:

Microsoft created a JSON file includes all M365 firewall rules. Is there any idea to add it to pfSense instead-of creating them manually as they are too much.

Thank you

https://endpoints.office.com/endpoints/worldwide?clientrequestid=b10c5ed1-bad1-445f-b386-b919946339a7

we use pfBlockerNG—specifically its (JSON, yes) parser and floating auto-rule creation functionality—to accomplish outbound IP whitelisting to M365/O365 endpoints.

any domains that may need to be whitelisted would need to be done so manually and depend entirely on what's otherwise DNSBL-blocked (if anything) in your environment.