Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Office 365 URLs and IP address ranges

    Scheduled Pinned Locked Moved Firewalling
    19 Posts 6 Posters 8.0k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • K
      kuberan
      last edited by

      What is the best way keep my Aliases updated from the published Office365 list of IPs?
      https://support.office.com/en-us/article/Office-365-URLs-and-IP-address-ranges-8548a211-3fe7-47cb-abb1-355ea5aa88a2
      The first time I did an Important adding the list of IPs all at once. Now how does one update that imported list. Is there a way to over write the list with all the current IPs publish. Also, is there a way to automate this process?

      Thanks

      T 1 Reply Last reply Reply Quote 0
      • D
        doktornotor Banned
        last edited by

        When they already have RSS, perhaps you could ask them to actually make use of it and publish the lists in non-retarded format?

        1 Reply Last reply Reply Quote 0
        • K
          kuberan
          last edited by

          I could request it published in a different way… what way do you suggest I request this... and how would my pfSense consume the published IPs and URL?
          Do you have an example and I will open a case with the MS Office team?

          Thanks for your help.

          1 Reply Last reply Reply Quote 0
          • BBcan177B
            BBcan177 Moderator
            last edited by

            If its a plain txt file, then you can use the pfSense URL Table aliases to collect these txt files on a schedule… Or use the pfBlockerNG package to parse the URL to collect the domains.... pfBlockerNG doesn't need the feed to be in plain text, and can parse the html to collect the IPs... You can then use the "Alias" in your firewall rules/nat etc as required...

            "Experience is something you don't get until just after you need it."

            Website: http://pfBlockerNG.com
            Twitter: @BBcan177  #pfBlockerNG
            Reddit: https://www.reddit.com/r/pfBlockerNG/new/

            1 Reply Last reply Reply Quote 0
            • D
              doktornotor Banned
              last edited by

              Even if it were RSS or HTML, things like pfBlockerNG could make sense of it. In the current form, it's just incredibly useless POS. Cannot believe then don't have anything in machine-readable format for enterprise use.

              1 Reply Last reply Reply Quote 0
              • K
                kuberan
                last edited by

                It is in XML format, that is how it is published, which is machine readable…..
                "Use the https://go.microsoft.com/fwlink/?LinkId=533185 to view a single list with all of the endpoints or to automatically process changes."

                1 Reply Last reply Reply Quote 0
                • D
                  daschenbrener
                  last edited by

                  @kuberan thanks for the info, how would you go about implementing this?

                  1 Reply Last reply Reply Quote 0
                  • K
                    kuberan
                    last edited by

                    I would like to automate this… but for now I edited the XML file Aliases and loaded that... I think pfBlockerNG  can be used.. but I need to test it first.... I was hoping there is a simpler solution

                    1 Reply Last reply Reply Quote 0
                    • K
                      kpa
                      last edited by

                      You won't achieve full automation easily because XML is essentially free form and to write a parser for an arbitrary XML file you need to know a bit about the structure of the file (called the schema). There are XML parsers for PHP but the parts that know about a particular XML schema (like this what MS uses) are not going be included in base pfSense, they are a clear case for an addon package.

                      1 Reply Last reply Reply Quote 0
                      • D
                        daschenbrener
                        last edited by

                        well it is working and updating from their files, so unless they change their format etc, Ill continue to use it.

                        1 Reply Last reply Reply Quote 0
                        • K
                          kuberan
                          last edited by

                          @daschenbrener what advice do you have. How did you do it?

                          1 Reply Last reply Reply Quote 0
                          • D
                            daschenbrener
                            last edited by

                            how i used it is through pfBlockerNG.

                            I added it to the rules

                            see picture attached

                            and it automatically parsed the file and is working

                            ![xml file.png](/public/imported_attachments/1/xml file.png)
                            ![xml file.png_thumb](/public/imported_attachments/1/xml file.png_thumb)

                            1 Reply Last reply Reply Quote 0
                            • K
                              kpa
                              last edited by

                              Is that a lucky accident or is PFBlockerNG smart enough to parse the XML properly?

                              1 Reply Last reply Reply Quote 0
                              • D
                                daschenbrener
                                last edited by

                                this is a first for me, but i believe that it is designed that way as pfsense can uses xml files already as well.

                                1 Reply Last reply Reply Quote 0
                                • D
                                  doktornotor Banned
                                  last edited by

                                  @kpa:

                                  Is that a lucky accident or is PFBlockerNG smart enough to parse the XML properly?

                                  It doesn't really parse it, it uses a bunch of crazy regexps to get IPs out of the XML/HTML/whatnot file.

                                  https://github.com/pfsense/FreeBSD-ports/blob/devel/net/pfSense-pkg-pfBlockerNG/files/usr/local/pkg/pfblockerng/pfblockerng.inc#L4037

                                  1 Reply Last reply Reply Quote 0
                                  • K
                                    kuberan
                                    last edited by

                                    @daschenbrener: thank you… I will give this a try.... :-)

                                    1 Reply Last reply Reply Quote 0
                                    • D
                                      daschenbrener
                                      last edited by

                                      my pleasure ,now to find files like this for other companies , hmm, maybe there is something here i can start, a list of the various locations for ip address . Just like the one for microsoft.

                                      1 Reply Last reply Reply Quote 0
                                      • K
                                        kuberan
                                        last edited by

                                        Good idea… it just needs to be a trusted list....

                                        1 Reply Last reply Reply Quote 0
                                        • T
                                          Tomoko @kuberan
                                          last edited by

                                          @kuberan
                                          I use the Powershell script from Microsoft. It generates the IPv4 addresses in a text file. This is uploaded daily to an FTP server via task scheduling. I have deactivated ipv6 and Urls in the Powershell-Script.
                                          Maybe this will help other users.

                                          <# Get-O365WebServiceUpdates.ps1
                                          From https://aka.ms/ipurlws
                                          v1.1 8/6/2019
                                          DESCRIPTION
                                          This script calls the REST API of the Office 365 IP and URL Web Service (Worldwide instance)
                                          and checks to see if there has been a new update since the version stored in an existing
                                          $Env:TEMP\O365_endpoints_latestversion.txt file in your user directory's temp folder
                                          (usually C:\Users\<username>\AppData\Local\Temp).
                                          If the file doesn't exist, or the latest version is newer than the current version in the
                                          file, the script returns IPs and/or URLs that have been changed, added or removed in the latest
                                          update and writes the new version and data to the output file $Env:TEMP\O365_endpoints_data.txt.
                                          
                                          USAGE
                                          Run as a scheduled task every 60 minutes.
                                          
                                          PARAMETERS
                                          n/a
                                          
                                          PREREQUISITES
                                          PS script execution policy: Bypass
                                          PowerShell 3.0 or later
                                          Does not require elevation
                                          https://docs.microsoft.com/en-us/microsoft-365/enterprise/microsoft-365-ip-web-service?view=o365-worldwide
                                          #>
                                          
                                          #Requires -Version 3.0
                                          
                                          # web service root URL
                                          $ws = "https://endpoints.office.com"
                                          # path where output files will be stored
                                          #$versionpath = $Env:TEMP + "\O365_endpoints_latestversion.txt"
                                          $versionpath = "C:\Powershell\O365_endpoints_latestversion.txt"
                                          #$datapath = $Env:TEMP + "\O365_endpoints_data.txt"
                                          $datapath = "C:\Powershell\O365_endpoints_data.txt"
                                          # fetch client ID and version if version file exists; otherwise create new file and client ID
                                          if (Test-Path $versionpath) {
                                              $content = Get-Content $versionpath
                                              $clientRequestId = $content[0]
                                              $lastVersion = $content[1]
                                              Write-Output ("Version file exists! Current version: " + $lastVersion)
                                          }
                                          else {
                                              Write-Output ("First run! Creating version file at " + $versionpath + ".")
                                              $clientRequestId = [GUID]::NewGuid().Guid
                                              $lastVersion = "0000000000"
                                              @($clientRequestId, $lastVersion) | Out-File $versionpath
                                          }
                                          
                                          # call version method to check the latest version, and pull new data if version number is different
                                          $version = Invoke-RestMethod -Uri ($ws + "/version/Worldwide?clientRequestId=" + $clientRequestId)
                                          if ($version.latest -gt $lastVersion) {
                                              Write-Host "New version of Office 365 worldwide commercial service instance endpoints detected"
                                              # write the new version number to the version file
                                              @($clientRequestId, $version.latest) | Out-File $versionpath
                                              # invoke endpoints method to get the new data
                                              $endpointSets = Invoke-RestMethod -Uri ($ws + "/endpoints/Worldwide?clientRequestId=" + $clientRequestId)
                                              # filter results for Allow and Optimize endpoints, and transform these into custom objects with port and category
                                              # URL results
                                              $flatUrls = $endpointSets | ForEach-Object {
                                                  $endpointSet = $_
                                                  $urls = $(if ($endpointSet.urls.Count -gt 0) { $endpointSet.urls } else { @() })
                                                  $urlCustomObjects = @()
                                                  if ($endpointSet.category -in ("Allow", "Optimize")) {
                                                      $urlCustomObjects = $urls | ForEach-Object {
                                                          [PSCustomObject]@{
                                                              category = $endpointSet.category;
                                                              url      = $_;
                                                              tcpPorts = $endpointSet.tcpPorts;
                                                              udpPorts = $endpointSet.udpPorts;
                                                          }
                                                      }
                                                  }
                                                  $urlCustomObjects
                                              }
                                              # IPv4 results
                                              $flatIp4s = $endpointSets | ForEach-Object {
                                                  $endpointSet = $_
                                                  $ips = $(if ($endpointSet.ips.Count -gt 0) { $endpointSet.ips } else { @() })
                                                  # IPv4 strings contain dots
                                                  $ip4s = $ips | Where-Object { $_ -like '*.*' }
                                                  $ip4CustomObjects = @()
                                                  if ($endpointSet.category -in ("Allow", "Optimize")) {
                                                      $ip4CustomObjects = $ip4s | ForEach-Object {
                                                          [PSCustomObject]@{
                                                              category = $endpointSet.category;
                                                              ip = $_;
                                                              tcpPorts = $endpointSet.tcpPorts;
                                                              udpPorts = $endpointSet.udpPorts;
                                                          }
                                                      }
                                                  }
                                                  $ip4CustomObjects
                                              }
                                              # IPv6 results
                                              $flatIp6s = $endpointSets | ForEach-Object {
                                                  $endpointSet = $_
                                                  $ips = $(if ($endpointSet.ips.Count -gt 0) { $endpointSet.ips } else { @() })
                                                  # IPv6 strings contain colons
                                                  $ip6s = $ips | Where-Object { $_ -like '*:*' }
                                                  $ip6CustomObjects = @()
                                                  if ($endpointSet.category -in ("Optimize")) {
                                                      $ip6CustomObjects = $ip6s | ForEach-Object {
                                                          [PSCustomObject]@{
                                                              category = $endpointSet.category;
                                                              ip = $_;
                                                              tcpPorts = $endpointSet.tcpPorts;
                                                              udpPorts = $endpointSet.udpPorts;
                                                          }
                                                      }
                                                  }
                                                  $ip6CustomObjects
                                              }
                                          
                                              # write output to screen
                                              Write-Output ("Client Request ID: " + $clientRequestId)
                                              Write-Output ("Last Version: " + $lastVersion)
                                              Write-Output ("New Version: " + $version.latest)
                                              Write-Output ""
                                              Write-Output "IPv4 Firewall IP Address Ranges"
                                              ($flatIp4s.ip | Sort-Object -Unique) -join "," | Out-String
                                              Write-Output "IPv6 Firewall IP Address Ranges"
                                              ($flatIp6s.ip | Sort-Object -Unique) -join "," | Out-String
                                              Write-Output "URLs for Proxy Server"
                                              ($flatUrls.url | Sort-Object -Unique) -join "," | Out-String
                                              Write-Output ("IP and URL data written to " + $datapath)
                                          
                                              # write output to data file
                                          #    Write-Output "Office 365 IP and UL Web Service data" | Out-File $datapath
                                          #    Write-Output "Worldwide instance" | Out-File $datapath -Append
                                          #    Write-Output "" | Out-File $datapath -Append
                                          #   Write-Output ("Version: " + $version.latest) | Out-File $datapath -Append
                                          #    Write-Output "" | Out-File $datapath -Append
                                          #    Write-Output "IPv4 Firewall IP Address Ranges" | Out-File $datapath -Append
                                              ($flatIp4s.ip | Sort-Object -Unique) -join "`r`n" | Out-File $datapath -Append
                                          #    Write-Output "" | Out-File $datapath -Append
                                          #    Write-Output "IPv6 Firewall IP Address Ranges" | Out-File $datapath -Append
                                          #    ($flatIp6s.ip | Sort-Object -Unique) -join "," | Out-File $datapath -Append
                                              Write-Output "" | Out-File $datapath -Append
                                          #    Write-Output "URLs for Proxy Server" | Out-File $datapath -Append
                                          #    ($flatUrls.url | Sort-Object -Unique) -join "," | Out-File $datapath -Append
                                          }
                                          else {
                                              Write-Host "Office 365 worldwide commercial service instance endpoints are up-to-date."
                                          }
                                          
                                          $file = "C:\Powershell\O365_endpoints_data.txt"
                                          # Do not use special characters # . / % & etc. for password!!!
                                          $ftpuri = "ftp://username:password@my-ftp-server.com/ms365.txt"
                                          $webclient = New-Object System.Net.WebClient
                                          $uri = New-Object System.Uri($ftpuri)
                                          $webclient.UploadFile($uri, $file)
                                          
                                          

                                          aliase

                                          1 Reply Last reply Reply Quote 1
                                          • First post
                                            Last post
                                          Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.