ROKU Issues with pfBlockerNG (CBS All Access, PBS, CNET) not working



  • I am having issues with my Roku 4 player not being able to stream videos on CBS All Access, PBS and CNET. I select a video, it starts to stream, then returns me to the home screen.  When I disable pfBlockerNG, the channels work.  Likewise, if I plug the ETH cable form the Roku 4 into my ASUS AC88U router, it works there too.  So I suspect pfBlockerNG is blocking traffic from these channels to their servers.

    How can I troubleshoot this issue? I perhaps need to see what server ip addresses these channels are connecting to and whitelist them?  I tried whitelisting cbs.com, pbs.org and cnet.com in DNSBL, Custom Domain Whitelist.  This did not fix the issue.

    Thanks for the help!



  • Sorry, I didn't document the sites but I had the same issue last fall.  If I remember correctly they use Fastly.com (AS54113).  I added that ASN to my pass list and it all streamed fine.  Take a close look at your alerts, I know they were in there.

    Know the IP of your RoKu and filter your deny alerts by that IP, same for DNSBL if you use it.



  • Thank you Ramosel,

    That gives me something to go on. I did see the entries

    dw.cbsi.com
    om.cbsi.com
    pubads.g.doubleclick.net
    b.scorecardresearch.com
    

    in the DNSBL alerts and added them to the DNSBL Custom Domain Whitelist.

    Using a AS number is a new one for me. I created a new IPv4 list called "Whitelist" and added the number 54113 in the Custom Address (es) field and checked the Enable Domain/AS box with List Action as Permit Both.

    I then ran the Force Update.  I still have the issue with the three channels. It is getting late in my time zone. I will take a break and pick it up tomorrow.  I appreciate the tips.



  • I've also had problems with some Roku channels (Smithsonian Channel, PBS, and a couple of oothers) not working when using DNSBL.  In my case I had to whitelist pubads.g.doubleclick.net.

    I have noted one really strange part of this problem.  If I use piHole as my DNS server I can block pubads.g.doubleclick.net and all the Roku channels work.  It is only when I use pfBlockerNG DNSBL that I have to whitelist pubads.g.doubleclick.net to get them to work.  It appears DNSBL and piHole are somehow answering the DNS query differently, but I haven't figured out what is happening yet.


  • Moderator

    @reisender:

    In my case I had to whitelist pubads.g.doubleclick.net.

    That domain also has a CNAME, so you might have to whitelist both of them… When you click on the "+" whitelist icon in the Alerts tab, it will automatically whitelist all CNAME options also.... but when you manually add the domain to the whitelist, you should run a similar drill command to add all CNAME options.  I am adding the @8.8.8.8 so that you bypass DNSBL since that will show the DNSBL VIP address if its being blocked.

    drill @8.8.8.8 pubads.g.doubleclick.net

    ;; ->>HEADER<<- opcode: QUERY, rcode: NOERROR, id: 3058
    ;; flags: qr rd ra ; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 0
    ;; QUESTION SECTION:
    ;; pubads.g.doubleclick.net.    IN      A

    ;; ANSWER SECTION:
    pubads.g.doubleclick.net.      86399  IN    CNAME  partnerad.l.doubleclick.net.
    partnerad.l.doubleclick.net.    299    IN      A            172.217.0.226


  • Moderator

    @Xentrk:

    Thank you Ramosel,

    That gives me something to go on. I did see the entries

    dw.cbsi.com
    om.cbsi.com
    pubads.g.doubleclick.net
    b.scorecardresearch.com
    

    in the DNSBL alerts and added them to the DNSBL Custom Domain Whitelist.

    Using a AS number is a new one for me. I created a new IPv4 list called "Whitelist" and added the number 54113 in the Custom Address (es) field and checked the Enable Domain/AS box with List Action as Permit Both.

    I then ran the Force Update.  I still have the issue with the three channels. It is getting late in my time zone. I will take a break and pick it up tomorrow.  I appreciate the tips.

    Just keep in mind that when something is blocked by DNSBL, you can't whitelist or permit it by using IP firewall rules…

    and vice-versa... They are two different processes...


  • Moderator

    @Xentrk:

    Using a AS number is a new one for me. I created a new IPv4 list called "Whitelist" and added the number 54113 in the Custom Address (es) field and checked the Enable Domain/AS box with List Action as Permit Both.

    Also note that you shouldn't use "Permit Both" unless you configure the "Adv. Inbound Settings" so that any unsolicited Inbound from these IPs is limited to the Open Ports and Destination IPs….

    Usually when there are no open WAN ports, you only need to use "Permit Outbound" so that the LAN has to initiate the request which will create a firewall state to allow those packets back thru the firewall (inbound)... ie: stateful firewall



  • Thank you everyone for your helpful replies and suggestions!

    I removed the IPv4 whitelist I created for fastly.com AS number 54113.  I then went back to the DNSBL Custom Domain Whitelist and added periods in front of the domain name:

    .dw.cbsi.com
    .om.cbsi.com
    .pubads.g.doubleclick.net
    .b.scorecardresearch.com
    

    I then saved, and ran update.  It works.  CBS All Access, PBS and CNET are now working. Wooot!

    Based on comments from reisender, I then removed all entries except for pubads.g.doubleclick.net.  CBS All Access, PBS and CNET worked.  So the solution is to whitelist pubads.g.doubleclick.net with a dot in front of it.

    I am very grateful for the help and suggestions.


  • Moderator

    Here are the potential domains:

    grep "pubads.g.doubleclick.net" /var/db/pfblockerng/dnsblorig/*

    /var/db/pfblockerng/dnsblorig/Cameleon.orig:127.0.0.1    pubads.g.doubleclick.net
        /var/db/pfblockerng/dnsblorig/Cameleon.orig:127.0.0.1   
        securepubads.g.doubleclick.net
        /var/db/pfblockerng/dnsblorig/MVPS.orig:0.0.0.0 pubads.g.doubleclick.net
        /var/db/pfblockerng/dnsblorig/MVPS.orig:0.0.0.0 securepubads.g.doubleclick.net
        /var/db/pfblockerng/dnsblorig/SWC.orig:#127.0.0.1 pubads.g.doubleclick.net      #interferes with video on cwtv.com
        /var/db/pfblockerng/dnsblorig/SWC.orig:127.0.0.1 securepubads.g.doubleclick.net
        /var/db/pfblockerng/dnsblorig/hpHosts_ads.orig:127.0.0.1        pubads.g.doubleclick.net
        /var/db/pfblockerng/dnsblorig/hpHosts_ads.orig:127.0.0.1        securepubads.g.doubleclick.net

    Here is the CNAME entry:

    grep "partnerad.l.doubleclick.net" /var/db/pfblockerng/dnsblorig/*

    /var/db/pfblockerng/dnsblorig/Cameleon.orig:127.0.0.1    partnerad.l.doubleclick.net
        /var/db/pfblockerng/dnsblorig/MVPS.orig:0.0.0.0 partnerad.l.doubleclick.net
        /var/db/pfblockerng/dnsblorig/hpHosts_ads.orig:127.0.0.1        partnerad.l.doubleclick.net

    I am not sure how prefixing the "period" to that domain removed any additional domains, as it wouldn't match the  securepubads.g.doubleclick.net  domain…

    This command will show all sub-domains:

    grep ".pubads.g.doubleclick.net" /var/db/pfblockerng/dnsblorig/*

    and it shows no domains listed… Maybe there was another reason for this issue? Maybe the device still had that domain cached after you whitelisted it...

    Either way glad that you got it working...



  • @BBcan177:

    @reisender:

    In my case I had to whitelist pubads.g.doubleclick.net.

    That domain also has a CNAME, so you might have to whitelist both of them…

    BBcan177, you are correct.  I also have the CNAME you provided whitelisted since I used the "+" icon to add it to the whitelist.  My whitelist does not have a "period" in front of pubads.

    Do you have any idea why the Roku channels work normally when I block pubads.g.doubleclick.net when using piHole for DNS, but fail when blocking this domain using pfBlockerNG?  It makes me think the channel doesn't depend on doubleclick.net since it works while blocked with piHole.  Something is different with the way DNSBL is handling it.  It isn't the end of the world to keep doubleclick whitelisted, but I prefer to block it if possible.



  • @reisender:

    @BBcan177:

    @reisender:

    In my case I had to whitelist pubads.g.doubleclick.net.

    That domain also has a CNAME, so you might have to whitelist both of them…

    BBcan177, you are correct.  I also have the CNAME you provided whitelisted since I used the "+" icon to add it to the whitelist.  My whitelist does not have a "period" in front of pubads.

    Do you have any idea why the Roku channels work normally when I block pubads.g.doubleclick.net when using piHole for DNS, but fail when blocking this domain using pfBlockerNG?  It makes me think the channel doesn't depend on doubleclick.net since it works while blocked with piHole.  Something is different with the way DNSBL is handling it.  It isn't the end of the world to keep doubleclick whitelisted, but I prefer to block it if possible.

    Thanks for the feedback. I made two changes today from my updates last night: 1 - removing the AS number entry 2 - adding periods to the whitelist entries.  I then got it to work. So I assumed it was the periods in front that did the trick.  I removed the period and saved. But I can't update DNSBL right now as an active cron job is running. I will report back and update with the results. You are right. The period should not be required.

    EDIT: I can confirm the leading dot is not required in front of the domain name.

    I should also report that I have two other routers running ASUS Merlin 380.66_2. On these routers I use an ad blocking solution https://www.ab-solution.info/ which also blacklists pubads.g.doubleclick.net.  But CBS, PBS and CNET work fine on those routers with the Roku 4 and I don't have to whitelist the pubads.g.doubleclick.net site on that ad blocking solution.


  • Moderator

    I can only assume that the Roku devices are attempting to connect to this domain via HTTPS, and are not timing out as expected…  DNSBL uses port forwards to redirect the DNS blocked request to the lighttod webserver. So for HTTPS blocked domains, the browser or app will see that the Certificate of the DNSBL Webserver doesn't match the requested domain and it should immediately timeout and close the connection....

    The next version of the package will have an option to utilize "0.0.0.0" instead of the DNSBL VIP... So that might address this issue for these particular situations. You could temporarily whitelist this domain in DNSBL and add a host override in Unbound for this domain and point it to "0.0.0.0"...


  • Moderator

    @Xentrk:

    I should also report that I have two other routers running ASUS Merlin 380.66_2. On these routers I use an ad blocking solution https://www.ab-solution.info/ which also blacklists pubads.g.doubleclick.net.  But CBS, PBS and CNET work fine on those routers with the Roku 4 and I don't have to whitelist the pubads.g.doubleclick.net site on that ad blocking solution.

    You can see from the website link that you provided above, that pixelserv is MITM'ing the domains by creating false certificates… obviously something that I want to avoid:

    pixelserv-tls.add: Added option to purge auto-generated pixelserv-tls certificates.



  • Thanks for the explanation BBcan117.  Your contribution to the pfSense community is huge!  I look forward to the next version.



  • A quick follow-up.  I followed BBCan117's suggestion to use a host override for pubads.g.doubleclick.net and the CNAME partnerads.l.doubleclick.net.

    The Roku channels still work, and the ads are not appearing.  Problem solved.



  • @reisender:

    A quick follow-up.  I followed BBCan117's suggestion to use a host override for pubads.g.doubleclick.net and the CNAME partnerads.l.doubleclick.net.

    The Roku channels still work, and the ads are not appearing.  Problem solved.

    Does my host overrides look correct (see attachment)?  This is a new one for me on pfSense.

    Also, did you keep the Custom Domain Whitelist entry for pubads.g.doubleclick.net in the DNSBL Configuration screen?

    Regards, Xentrk




  • Your host override isn't quite right.  Make it look like the attached image.  Keep the domains in your whitelist.  I kept the whitelist for both domains.  I'm not sure if DNSBL or the host override is processed first, but it seems to work this way.




  • @reisender:

    Your host override isn't quite right.  Make it look like the attached image.  Keep the domains in your whitelist.  I kept the whitelist for both domains.  I'm not sure if DNSBL or the host override is processed first, but it seems to work this way.

    Thanks you reisender and BBCan177 for the help.  With the DNS Resolver change, I no longer see ads on those channels. I will need to watch more to make sure. A nice benefit for sure.  I'm happy to get those channels working again though. I never thought it was my firewall blocking them for two months as they worked okay before the last part of April. I then saw on the Roku forums that others were having an issue as well. I posted the fix on the Roku forum site.  Having access to TV in USA enhances my expat life here in the land of smiles.

    Because of this issue, I learned features of pfBlockerNG and DNS Resolver that I was not aware of.  I appreciate your help and educating me along the way!  I am very grateful for the help you all provided.

    Regards, Xen


  • Moderator

    @Xentrk:

    Thanks you reisender and BBCan177 for the help.  With the DNS Resolver change, I no longer see ads on those channels. I will need to watch more to make sure. A nice benefit for sure.  I'm happy to get those channels working again though. I never thought it was my firewall blocking them for two months as they worked okay before the last part of April. I then saw on the Roku forums that others were having an issue as well. I posted the fix on the Roku forum site.  Having access to TV in USA enhances my expat life here in the land of smiles.

    Because of this issue, I learned features of pfBlockerNG and DNS Resolver that I was not aware of.  I appreciate your help and educating me along the way!  I am very grateful for the help you all provided.

    Regards, Xen

    NP your welcome…  ;)



  • @reisender:

    Your host override isn't quite right.  Make it look like the attached image.  Keep the domains in your whitelist.  I kept the whitelist for both domains.  I'm not sure if DNSBL or the host override is processed first, but it seems to work this way.

    reisender,

    Is CBS all access still working for you?  It stopped working for me last night.  When I select a video to watch, I get a little spinning symbol for a few seconds followed by a black screen.  I can watch live TV okay.  It is just the on demand videos that are the issue.  It works okay on my ASUS router using AB-Solution.info ad blocker. But I do see ads. Maybe it is time to pay the extra $$ for the ad free version.



  • @Xentrk:

    @reisender:

    Your host override isn't quite right.  Make it look like the attached image.  Keep the domains in your whitelist.  I kept the whitelist for both domains.  I'm not sure if DNSBL or the host override is processed first, but it seems to work this way.

    reisender,

    Is CBS all access still working for you?  It stopped working for me last night.  When I select a video to watch, I get a little spinning symbol for a few seconds followed by a black screen.  I can watch live TV okay.  It is just the on demand videos that are the issue.  It works okay on my ASUS router using AB-Solution.info ad blocker. But I do see ads. Maybe it is time to pay the extra $$ for the ad free version.

    I have it working again. I was unable to determine the domain or host file causing the issues. I did some testing with hosts file on my ASUS router using AB-Solution. I had issues when I went to higher levels of hosts files. I found the right combination that made it work on the ASUS. I replicated that on the pfSense. I started with this list:

    http://someonewhocares.org/hosts/hosts
    http://sysctl.org/cameleon/hosts
    http://winhelp2002.mvps.org/hosts.txt
    http://www.malekal.com/HOSTS_filtre/HOSTS.txt
    http://www.malwaredomainlist.com/hostslist/hosts.txt
    https://zeustracker.abuse.ch/blocklist.php?download=hostfile
    http://www.hostsfile.org/Downloads/hosts.txt
    http://www.securemecca.com/Downloads/hosts.txt
    http://hosts-file.net/exp.txt
    http://hosts-file.net/ad_servers.txt
    http://hosts-file.net/emd.txt
    http://hosts-file.net/hjk.txt
    http://hosts-file.net/fsa.txt
    http://hosts-file.net/grm.txt
    http://hosts-file.net/psh.txt
    http://hosts-file.net/mmt.txt
    http://hosts-file.net/hfs.txt
    http://hosts-file.net/pha.txt
    http://hosts-file.net/wrz.txt
    http://raw.githubusercontent.com/michaeltrimm/hosts-blocking/master/_hosts.txt
    

    And narrowed it down to this list

    https://adaway.org/hosts.txt
    http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts&showintro=0&mimetype=plaintext&useip=0.0.0.0
    http://someonewhocares.org/hosts/zero/hosts
    http://winhelp2002.mvps.org/hosts.txt
    http://www.malwaredomainlist.com/hostslist/hosts.txt
    http://hosts-file.net/ad_servers.txt
    http://hosts-file.net/emd.txt
    http://hosts-file.net/grm.txt
    http://hosts-file.net/mmt.txt
    
    

    Perhaps one of these days, I will take some the hosts files back in one by one until I determine which one caused me the grief.



  • The above solution was a false positive. It did not work.  I ended up removing the Host Overrides in DNS Resolver to get it working. However, ads are now appearing.  We'll, I am paying for the lower tier with ads. So I can live with it. Enjoying it ad free was nice while it lasted though.