DNS-blackhole / Adblock / hosts
-
DISCLAIMER: I have of course used the forum-search, as well as the almighty GOOGLE, but so far no dice.
I'm currently using the newest pfSense (2.3.2 (AMD64) nanobsd (4g) mounted RW), with Unbound as resolver, and I'm looking to dig a large black hole (0.0.0.0), into which I can put requests for a large number of ad-sites and everything else I might deem to be unwanted.
Currently I use: https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts (which covers about 26K+ domains) on a couple of machines, but I'd like to also have pfSense do it for phones, tablets etc. (all are static DHCP-clients due to NFS-shares etc.)
As far as I can gather, editing the /etc/hosts on a pfSense, doesn't really work, as it is constantly overwritten and nothing seems to stick.
I've tried copying the hosts-file from the link above, to the pfSense-router, (logged in as root via SSH/SCP), but again it doesn't seem to stick. (Any idea as to where to pu it, to make it stick?)
If I could get the file on to the machine, I suppose it would be no problem to just add it to DNS Resolver => General Settings => Custom options => server:include: /SOMEPLACE/github-hostsI've been toying around with pfBlockerNG, trying to figure out how to do what I want to. But have so far only gotten it to activate Easylists, and I can't say that I notice any difference.
I'm probably just doin' it wrong?!?
I appreciate any hints/input/pointers to get me to where I'd like to go.
/Bawl
-
I think pfblocker is overkill for this as well. I like its connection with country IP ranges for aliases - but latest version and change of databases really screwed the pooch with mem usage.
If all your looking to do is have domains resolve to something that blackholes them, I resolve them to 127.0.0.1
In the unbound options.
server:
include: /etc/unbound_ad_serversThen put what your wanting to block in there, I use a cron job to update my list..
example
local-zone: "zenzuu.com" redirect
local-data: "zenzuu.com A 127.0.0.1"
local-zone: "zeus.developershed.com" redirect
local-data: "zeus.developershed.com A 127.0.0.1"
local-zone: "zeusclicks.com" redirect
local-data: "zeusclicks.com A 127.0.0.1"
local-zone: "zintext.com" redirect
local-data: "zintext.com A 127.0.0.1"
local-zone: "zmedia.com" redirect
local-data: "zmedia.com A 127.0.0.1"
local-zone: "zv1.november-lax.com" redirect
local-data: "zv1.november-lax.com A 127.0.0.1"I grab list from here formatted for use with unbound
http://pgl.yoyo.org/adservers/serverlist.php?hostformat=unbound;showintro=0Simple cron
/usr/local/bin/curl -sS -L --compressed "http://pgl.yoyo.org/adservers/serverlist.php?hostformat=unbound;showintro=0" | sed -e 's/<[^>]*>//g' | sed -e 's/^Ad.*//g' > /etc/unbound_ad_servers
and then another one to reload unbound a few minutes later
/usr/local/sbin/unbound-control -c /var/unbound/unbound.conf reload -
I wish there was a simple anti-kitchensink package to do just that, I tried PfBlockerNG for a while but it just tries to do too much in one package.
-
^ yeah, bcan has done an amazing job with that package.. But be nice if he could break it up in to smaller pieces for those of that that don't want the whole kitchensink.. I would love to use his alias creation for geo ip blocks, etc. But I sure and the hell don't want it f'ing with my firewall rules directly, etc.
-
Simple cron
/usr/local/bin/curl -sS -L --compressed "http://pgl.yoyo.org/adservers/serverlist.php?hostformat=unbound;showintro=0" | sed -e 's/<[^>]*>//g' | sed -e 's/^Ad.*//g' > /etc/unbound_ad_servers
and then another one to reload unbound a few minutes later
/usr/local/sbin/unbound-control -c /var/unbound/unbound.conf reloadThis looks more or less spot on, and nice with the cron-job option. :)
example
local-zone: "zenzuu.com" redirect
local-data: "zenzuu.com A 127.0.0.1"
local-zone: "zeus.developershed.com" redirect
local-data: "zeus.developershed.com A 127.0.0.1"
local-zone: "zeusclicks.com" redirect
local-data: "zeusclicks.com A 127.0.0.1"
local-zone: "zintext.com" redirect
local-data: "zintext.com A 127.0.0.1"
local-zone: "zmedia.com" redirect
local-data: "zmedia.com A 127.0.0.1"
local-zone: "zv1.november-lax.com" redirect
local-data: "zv1.november-lax.com A 127.0.0.1"I grab list from here formatted for use with unbound
http://pgl.yoyo.org/adservers/serverlist.php?hostformat=unbound;showintro=0That's good, but as far as I can tell; Mine's bigger than your's. (My list, that is…) :D
https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts is 26,000+ entries, while the one from yoyo.org is 2700+.
I know quantity and quality isn't necessarily comparable, but I hope I would be forgiven for thinking the GitHub-list covered a few more bases!?!?
And thus my problem becomes the formatting of the GitHub-list.
I know they say, never give the Devil a finger, or he'll rip your arm of or eat you right up or some such. :)
But I'd appreciate a hint on how to use the GitHub-list instead of or together with the yoyo-list
@kpa:
I wish there was a simple anti-kitchensink package to do just that, I tried PfBlockerNG for a while but it just tries to do too much in one package.
This! :)
pfBlockerNG (and SquidGuard for that matter) look immensely powerful, but I'm not sure where to start, and all I'm looking for, (right now) is simple /etc/hosts-blocking… ;-) -
^ yeah, bcan has done an amazing job with that package.. But be nice if he could break it up in to smaller pieces for those of that that don't want the whole kitchensink.. I would love to use his alias creation for geo ip blocks, etc. But I sure and the hell don't want it f'ing with my firewall rules directly, etc.
All the features in pfBlockerNG are optional, you do not have to use all of them. You can manage you own FW rules when using Alias Deny instead of using Deny Both.
The issue with memory was triggered by the MaxMind database changing from Range to CIDR will eventually be fixed when BBcan177 is back from vacation.
He already has in his wishlist the sink-holing and skipping the webserver 1x1, but doing so will kill the alerts logging.
-
And I just notice that the GitHub, pulls from yoyo.org as well:
https://github.com/StevenBlack/hosts
And if you scroll down, (apparently a 1900x1200 monitor makes it look like the top is the full page) ::) there are multiple versions of the file, to suit whatever preferences you might have.
And even a tutorial for creating you own unified hosts-file, if you were so inclined.
Now we just need to make it play nice with Unbound. 8)
-
Did you look at pfBlockerNG v2.0 w/DNSBL
I not that difficult to setup for a few DNSBL feeds. -
And I just notice that the GitHub, pulls from yoyo.org as well:
https://github.com/StevenBlack/hosts
And if you scroll down, (apparently a 1900x1200 monitor makes it look like the top is the full page) ::) there are multiple versions of the file, to suit whatever preferences you might have.
And even a tutorial for creating you own unified hosts-file, if you were so inclined.
Now we just need to make it play nice with Unbound. 8)
Maybe this tool will work to convert it?
https://github.com/jodrell/unbound-block-hosts
-
While I agree the different parts in pfblocker are optional.. There is just so much in that package that it could for sure get overwhelming for a new user, etc. I look forward to discussing with bbcan when he gets back the possibility of a "lite" version if you will.
I was using it with just the stuff I wanted, the geoip stuff.. But vs doing the fix when the memory thing blew up I just uninstalled it and will wait for a fixed version or hopefully a slimmed down version, etc. Direct to just the feature of creating aliases from the geoip stuff I would for sure install in a heart beat.. Something like that would be nice to see just plain integrated into pfsense directly.
Kind of how unbound at one point was a package and now is just part of pfsense default install.
-
Maybe this tool will work to convert it?
https://github.com/jodrell/unbound-block-hosts
Now we're talking… 8) (Gotta love the power of the hive-mind)
Gotta study this one a little closer tomorrow,
I'll report back if/when I get it running.
-
I got unbound-block-hosts copied to the pfSense-router via SCP:
scp unbound-block-hosts root@[pfSense-IP]:/root/
I then logged in as root via ssh: (and found out that I've mistankenly created a /root/var-directory, which explains why nothing seemed to stick earlier) ::)
ssh root@[pfSense-IP]
I've checked that Perl is installed:
perl -v
Then I try to run the script by:
perl unbound-block-hosts ```(I'm in /root/ where I copied unbound-block-hosts to) and then I get the following error:
Can't locate LWP.pm in @INC (you may need to install the LWP module)
(@INC contains: /usr/local/lib/perl5/site_perl/mach/5.20 /usr/local/lib/perl5/site_perl /usr/local/lib/perl5/5.20/mach /usr/local/lib/perl5/5.20 /usr/local/lib/perl5/site_perl/5.20 /usr/local/lib/perl5/site_perl/5.20/mach .) at unbound-block-hosts line 5.
BEGIN failed--compilation aborted at unbound-block-hosts line 5.Skimming through the list of available packages in pfSense, I'm not really seeing anything that looks like Perl or LWP, so I'm now currently at a loss with regards to moving forward. I'll dig deeper, when work is out of the way… :) If anyone has a suggestion they're more than welcome.
-
LWP is not a standard module.. Normally you would just install via cpan.. But not sure it will work on pfsense?? I have never installed any extra perl modules on pfsense.
Typically its
cpan
install modulename
exit -
LWP is not a standard module.. Normally you would just install via cpan.. But not sure it will work on pfsense?? I have never installed any extra perl modules on pfsense.
Typically its
cpan
install modulename
exitcpan is apparently a thing in pfSense:
cpan
first asks if I want it to automagically configure settings, I answer yes.
I then type:
install lwp
I get an error:
Warning: Cannot install lwp, don't know what it is. Try the command i /lwp/ to find objects with matching identifiers.
I then try:
i /lwp/
And get the following laundry list of 325 items:
cpan[2]> i /lwp/ Bundle Bundle::LWP (GAAS/libwww-perl-5.837.tar.gz) Bundle Bundle::LWP5_837 (SPROUT/Bundle-LWP5_837-0.01.tar.gz) Distribution ADAMK/LWP-Online-1.08.tar.gz Distribution AGROLMS/LWP-Authen-Negotiate-0.08.tar.gz Distribution ALEXMV/LWP-UserAgent-Determined-1.07.tar.gz Distribution AUTRIJUS/LWP-Authen-Wsse-0.05.tar.gz Distribution BENNING/LWP-Protocol-connect-6.09.tar.gz Distribution BGILLS/JSON-RPC-LWP-0.007.tar.gz Distribution BURAK/Net-Hadoop-WebHDFS-LWP-0.007.tar.gz Distribution CLIFFORDJ/Mock-LWP-Request-0.01.tar.gz Distribution CREIN/LWPx-Profile-0.2.tar.gz Distribution CREIN/Plack-Middleware-Debug-LWP-0.2.tar.gz Distribution CREIN/Test-VCR-LWP-0.5.tar.gz Distribution DOMM/LWP-Authen-OAuth2-0.12.tar.gz Distribution EALLENIII/Test-LWP-Recorder-0.1.1.tar.gz Distribution ETHER/Test-LWP-UserAgent-0.030.tar.gz Distribution FLORA/LWP-Protocol-http-SocketUnix-0.02.tar.gz Distribution GAAS/LWP-MediaTypes-6.02.tar.gz Distribution GAAS/LWP-Protocol-http10-6.03.tar.gz Distribution GAAS/LWP-attic-1.00.tar.gz Distribution GDM/LWP-UserAgent-OfflineCache-0.02.tar.gz Distribution GONCALES/LWP-Simple-REST-0.092.tar.gz Distribution GOSHA/LWP-Protocol-http-SocksChain-1.7.tar.gz Distribution GOSHA/LWP-Protocol-http-SocksChain10-1.7.tar.gz Distribution GOSHA/LWP-Protocol-https-SocksChain-1.8.tar.gz Distribution GOSHA/LWP-Protocol-https-SocksChain10-1.7.tar.gz Distribution HVALVERDE/LWP-UserAgent-RandomProxyConnect-1.10.tar.gz Distribution IKEGAMI/LWP-Protocol-AnyEvent-http-v1.8.0.tar.gz Distribution IKEGAMI/LWP-Protocol-Coro-http-v1.8.0.tar.gz Distribution ITUB/IO-All-LWP-0.12.tar.gz Distribution ITUB/IO-All-LWP-0.14.tar.gz Distribution JJORE/LWP-Simple-Cookies-0.01.tar.gz Distribution KJETILK/LWP-UserAgent-CHICaching-0.04.tar.gz Distribution KRYDE/LWP-Protocol-rsync-1.tar.gz Distribution LBROCARD/LWP-ConnCache-MaxKeepAliveRequests-0.33.tar.gz Distribution LEEYM/LWP-Simple-WithCache-0.03.tar.gz Distribution LORN/LWP-Curl-0.14.tar.gz Distribution LUKEC/Test-Mock-LWP-0.08.tar.gz Distribution LUSHE/Egg-Plugin-LWP-3.01.tar.gz Distribution MANWAR/LWP-UserAgent-Anonymous-0.09.tar.gz Distribution MARCEL/LWP-UserAgent-ProgressBar-1.100810.tar.gz Distribution MARROTTE/LWP-UserAgent-ExponentialBackoff-004/LWP-UserAgent-ExponentialBackoff-0.04.tar.gz Distribution MASAKI/LWP-UserAgent-DNS-Hosts-0.08.tar.gz Distribution MASAKI/Test-Mock-LWP-Conditional-0.03.tar.gz Distribution MASAKI/Test-Mock-LWP-Conditional-0.04.tar.gz Distribution MBARBON/Wx-Perl-FSHandler-LWP-0.03.tar.gz Distribution MIYAGAWA/LWP-Protocol-PSGI-0.09.tar.gz Distribution MIYAGAWA/LWP-UserAgent-Keychain-0.01.tar.gz Distribution MJEMMESON/LWP-UserAgent-Mockable-1.16.tar.gz Distribution MJGARDNER/LWPx-UserAgent-Cached-0.007.tar.gz Distribution MNUNBERG/Test-LWP-MockSocket-http-0.05.tar.gz Distribution MOTEMEN/LWPx-Record-DataSection-0.01.tar.gz Distribution MSCHILLI/LWP-Protocol-https-6.06.tar.gz Distribution MSCHILLI/LWP-UserAgent-POE-0.05.tar.gz Distribution NIKOLAS/LWP-Protocol-UWSGI-v1.1.8.tar.gz Distribution NPW/LWP-UserAgent-Snapshot-v0.2.tar.gz Distribution OALDERS/LWP-ConsoleLogger-0.000028.tar.gz Distribution OLEG/LWP-UserAgent-Cached-0.06.tar.gz Distribution PERLANCAR/LWP-UserAgent-Patch-FilterMirror-0.05.tar.gz Distribution PERLANCAR/LWP-UserAgent-Patch-HTTPSHardTimeout-0.05.tar.gz Distribution PERLANCAR/LWP-UserAgent-Patch-LogRequestContent-0.02.tar.gz Distribution PERLANCAR/LWP-UserAgent-Patch-LogResponse-0.09.tar.gz Distribution PERLANCAR/LWP-UserAgent-Patch-Retry-0.03.tar.gz Distribution PERLANCAR/LWP-UserAgent-ProgressAny-0.06.tar.gz Distribution PERLANCAR/Log-Any-For-LWP-0.06.tar.gz Distribution QJZHOU/LWP-UserAgent-ProxyAny-1.01.tar.gz Distribution RPAUL/LWP-Protocol-virtual-0.02.tar.gz Distribution SALVA/LWP-Protocol-sftp-0.05.tar.gz Distribution SARGIE/LWP-Simple-Post-0.05.tar.gz Distribution SAXJAZMAN/lwp/LWPx-ParanoidAgent-1.10.tar.gz Distribution SCR/LWP-Protocol-socks-1.7.tar.gz Distribution SEKIMURA/LWP-UserAgent-WithCache-0.12.tar.gz Distribution SERGEYCHE/LWP-ConnCache-Resolving-0.02.tar.gz Distribution SHARYANTO/LWP-Protocol-http-SocketUnixAlt-0.0204.tar.gz Distribution SHIRAIWA/LWP-UserAgent-Cache-Memcached-0.02.tar.gz Distribution SIMONW/LWPx-TimedHTTP-1.4.tar.gz Distribution SIMONW/LWPx-TimedHTTP-1.8.tar.gz Distribution SKINGTON/LWP-JSON-Tiny-0.007.tar.gz Distribution SPEBERN/LWP-UserAgent-Tor-0.05.tar.gz Distribution SPROUT/Bundle-LWP5_837-0.01.tar.gz Distribution SYP/LWP-Protocol-Net-Curl-0.023.tar.gz Distribution TADAM/Test-Mock-LWP-Dispatch-0.08.tar.gz Distribution TIMBRODY/LWP-Authen-OAuth-1.02.tar.gz Distribution TOKUHIROM/LWPx-ParanoidHandler-0.07.tar.gz Distribution TSIBLEY/LWP-UserAgent-Paranoid-0.97.tar.gz Distribution YAKEX/AnyEvent-HTTP-LWP-UserAgent-0.10.tar.gz Distribution YAKEX/AnyEvent-HTTP-LWP-UserAgent-Determined-v0.05.1.06.tar.gz Distribution ZOFFIX/LWP-UserAgent-ProxyHopper-0.004.tar.gz Distribution ZOFFIX/LWP-UserAgent-ProxyHopper-Base-0.003.tar.gz Module < Activiti::Rest::UserAgent::LWP (NJFRANCK/Activiti-Rest-Client-0.1253.tar.gz) Module < Amazon::DynamoDB::LWP (RCONOVER/Amazon-DynamoDB-0.35.tar.gz) Module < AnyEvent::HTTP::LWP::UserAgent (YAKEX/AnyEvent-HTTP-LWP-UserAgent-0.10.tar.gz) Module < AnyEvent::HTTP::LWP::UserAgent::Determined (YAKEX/AnyEvent-HTTP-LWP-UserAgent-Determined-v0.05.1.06.tar.gz) Module < Apache::Tika::Connection::LWP (CORION/Apache-Tika-Async-0.06.tar.gz) Module < App::ZofCMS::Plugin::BasicLWP (ZOFFIX/App-ZofCMS-1.001006.tar.gz) Module < ArangoDB2::HTTP::LWP (EWARNCKE/ArangoDB2-0.11.tar.gz) Module < Babble::Transport::LWP (ALGERNON/Babble-0.07.tar.gz) Module < CORION::Apache::Tika::Connection::LWP (CORION/Dancer-SearchApp-0.05.tar.gz) Module = CPAN::LWP::UserAgent (ANDK/CPAN-2.14.tar.gz) Module < Catmandu::AlephX::UserAgent::LWP (NJFRANCK/Catmandu-AlephX-1.065.tar.gz) Module < Coro::LWP (MLEHMANN/Coro-6.511.tar.gz) Module < Coro::LWP::Socket (MLEHMANN/Coro-6.511.tar.gz) Module < Coro::PatchSet::LWP (OLEG/Coro-PatchSet-0.13.tar.gz) Module < Egg::Plugin::LWP (LUSHE/Egg-Plugin-LWP-3.01.tar.gz) Module < Email::MIME::CreateHTML::Resolver::LWP (VANSTYN/Email-MIME-CreateHTML-1.041.tar.gz) Module < Gauge::LWP_Curl (SYP/AnyEvent-Net-Curl-Queued-0.047.tar.gz) Module < Gauge::LWP_Protocol_Net_Curl (SYP/AnyEvent-Net-Curl-Queued-0.047.tar.gz) Module < Gauge::LWP_UserAgent (SYP/AnyEvent-Net-Curl-Queued-0.047.tar.gz) Module < Gtk::LWP (MLEHMANN/Gtk-Perl-0.7010.tar.gz) Module < Gtk::LWP::http (MLEHMANN/Gtk-Perl-0.7010.tar.gz) Module < HTTP::Any::LWP (KNI/HTTP-Any-0.06.tar.gz) Module < HTTP::CookieJar::LWP (DAGOLDEN/HTTP-CookieJar-0.008.tar.gz) Module < HTTP::Tinyish::LWP (MIYAGAWA/HTTP-Tinyish-0.07.tar.gz) Module < IO::All::LWP (ITUB/IO-All-LWP-0.14.tar.gz) Module < JSON::RPC::LWP (BGILLS/JSON-RPC-LWP-0.007.tar.gz) Module < LWP (ETHER/libwww-perl-6.15.tar.gz) Module < LWP::Auth_UA (MIKEDLR/Link_Controller-0.037.tar.gz) Module < LWP::Authen::Basic (ETHER/libwww-perl-6.15.tar.gz) Module < LWP::Authen::Digest (ETHER/libwww-perl-6.15.tar.gz) Module < LWP::Authen::Gooddata (LKUNDRAK/WWW-GoodData-1.11.tar.gz) Module < LWP::Authen::Negotiate (AGROLMS/LWP-Authen-Negotiate-0.08.tar.gz) Module < LWP::Authen::Ntlm (ETHER/libwww-perl-6.15.tar.gz) Module < LWP::Authen::OAuth (TIMBRODY/LWP-Authen-OAuth-1.02.tar.gz) Module < LWP::Authen::OAuth2 (DOMM/LWP-Authen-OAuth2-0.12.tar.gz) Module < LWP::Authen::OAuth2::AccessToken (DOMM/LWP-Authen-OAuth2-0.12.tar.gz) Module < LWP::Authen::OAuth2::AccessToken::Bearer (DOMM/LWP-Authen-OAuth2-0.12.tar.gz) Module < LWP::Authen::OAuth2::Args (DOMM/LWP-Authen-OAuth2-0.12.tar.gz) Module < LWP::Authen::OAuth2::ServiceProvider (DOMM/LWP-Authen-OAuth2-0.12.tar.gz) Module < LWP::Authen::OAuth2::ServiceProvider::Dwolla (DOMM/LWP-Authen-OAuth2-0.12.tar.gz) Module < LWP::Authen::OAuth2::ServiceProvider::Google (DOMM/LWP-Authen-OAuth2-0.12.tar.gz) Module < LWP::Authen::OAuth2::ServiceProvider::Google::Device (DOMM/LWP-Authen-OAuth2-0.12.tar.gz) Module < LWP::Authen::OAuth2::ServiceProvider::Google::Installed (DOMM/LWP-Authen-OAuth2-0.12.tar.gz) Module < LWP::Authen::OAuth2::ServiceProvider::Google::Login (DOMM/LWP-Authen-OAuth2-0.12.tar.gz) Module < LWP::Authen::OAuth2::ServiceProvider::Google::Service (DOMM/LWP-Authen-OAuth2-0.12.tar.gz) Module < LWP::Authen::OAuth2::ServiceProvider::Google::WebServer (DOMM/LWP-Authen-OAuth2-0.12.tar.gz) Module < LWP::Authen::OAuth2::ServiceProvider::Strava (DOMM/LWP-Authen-OAuth2-0.12.tar.gz) Module < LWP::Authen::Wsse (AUTRIJUS/LWP-Authen-Wsse-0.05.tar.gz) Module < LWP::AuthenAgent (AWRIGLEY/sitemapper-1.019.tar.gz) Module < LWP::ConnCache (ETHER/libwww-perl-6.15.tar.gz) Module < LWP::ConnCache::MaxKeepAliveRequests (LBROCARD/LWP-ConnCache-MaxKeepAliveRequests-0.33.tar.gz) Module < LWP::ConnCache::Resolving (SERGEYCHE/LWP-ConnCache-Resolving-0.02.tar.gz) Module < LWP::ConsoleLogger (OALDERS/LWP-ConsoleLogger-0.000028.tar.gz) Module < LWP::ConsoleLogger::Easy (OALDERS/LWP-ConsoleLogger-0.000028.tar.gz) Module < LWP::Curl (LORN/LWP-Curl-0.14.tar.gz) Module < LWP::Debug (ETHER/libwww-perl-6.15.tar.gz) Module < LWP::DebugFile (ETHER/libwww-perl-6.15.tar.gz) Module < LWP::FastRobot (MIKEDLR/Link_Controller-0.037.tar.gz) Module < LWP::Iterator::UserAgent (EWILHELM/dotReader-v0.11.2.tar.gz) Module < LWP::JSON::Tiny (SKINGTON/LWP-JSON-Tiny-0.007.tar.gz) Module < LWP::MediaTypes (GAAS/LWP-MediaTypes-6.02.tar.gz) Module < LWP::MemberMixin (ETHER/libwww-perl-6.15.tar.gz) Module < LWP::NoStopRobot (MIKEDLR/Link_Controller-0.037.tar.gz) Module < LWP::Online (ADAMK/LWP-Online-1.08.tar.gz) Module < LWP::Parallel (MSOUTH/ParallelUserAgent-2.62.tgz) Module < LWP::Parallel::Protocol (MSOUTH/ParallelUserAgent-2.62.tgz) Module < LWP::Parallel::Protocol::MyFTP (MSOUTH/ParallelUserAgent-2.62.tgz) Module < LWP::Parallel::Protocol::file (MSOUTH/ParallelUserAgent-2.62.tgz) Module < LWP::Parallel::Protocol::ftp (MSOUTH/ParallelUserAgent-2.62.tgz) Module < LWP::Parallel::Protocol::http (MSOUTH/ParallelUserAgent-2.62.tgz) Module < LWP::Parallel::Protocol::http::Socket (MSOUTH/ParallelUserAgent-2.62.tgz) Module < LWP::Parallel::Protocol::http::SocketMethods (MSOUTH/ParallelUserAgent-2.62.tgz) Module < LWP::Parallel::Protocol::https (MSOUTH/ParallelUserAgent-2.62.tgz) Module < LWP::Parallel::Protocol::https::Socket (MSOUTH/ParallelUserAgent-2.62.tgz) Module < LWP::Parallel::RobotUA (MSOUTH/ParallelUserAgent-2.62.tgz) Module < LWP::Parallel::UserAgent (MSOUTH/ParallelUserAgent-2.62.tgz) Module < LWP::Parallel::UserAgent::Entry (MSOUTH/ParallelUserAgent-2.62.tgz) Module < LWP::ParallelUA (MSOUTH/ParallelUserAgent-2.62.tgz) Module < LWP::Protocol (ETHER/libwww-perl-6.15.tar.gz) Module < LWP::Protocol::AnyEvent::http (IKEGAMI/LWP-Protocol-AnyEvent-http-v1.8.0.tar.gz) Module < LWP::Protocol::Coro::http (IKEGAMI/LWP-Protocol-Coro-http-v1.8.0.tar.gz) Module < LWP::Protocol::GHTTP (ETHER/libwww-perl-6.15.tar.gz) Module < LWP::Protocol::MyFTP (ETHER/libwww-perl-6.15.tar.gz) Module < LWP::Protocol::Net::Curl (SYP/LWP-Protocol-Net-Curl-0.023.tar.gz) Module < LWP::Protocol::PSGI (MIYAGAWA/LWP-Protocol-PSGI-0.09.tar.gz) Module < LWP::Protocol::UWSGI (NIKOLAS/LWP-Protocol-UWSGI-v1.1.8.tar.gz) Module < LWP::Protocol::UWSGI::Socket (NIKOLAS/LWP-Protocol-UWSGI-v1.1.8.tar.gz) Module < LWP::Protocol::connect (BENNING/LWP-Protocol-connect-6.09.tar.gz) Module < LWP::Protocol::connect::Socket::Base (BENNING/LWP-Protocol-connect-6.09.tar.gz) Module < LWP::Protocol::cpan (ETHER/libwww-perl-6.15.tar.gz) Module < LWP::Protocol::data (ETHER/libwww-perl-6.15.tar.gz) Module < LWP::Protocol::file (ETHER/libwww-perl-6.15.tar.gz) Module < LWP::Protocol::ftp (ETHER/libwww-perl-6.15.tar.gz) Module < LWP::Protocol::gopher (ETHER/libwww-perl-6.15.tar.gz) Module < LWP::Protocol::http (ETHER/libwww-perl-6.15.tar.gz) Module < LWP::Protocol::http10 (GAAS/LWP-Protocol-http10-6.03.tar.gz) Module < LWP::Protocol::http::Socket (ETHER/libwww-perl-6.15.tar.gz) Module < LWP::Protocol::http::SocketMethods (ETHER/libwww-perl-6.15.tar.gz) Module < LWP::Protocol::http::SocketUnix (FLORA/LWP-Protocol-http-SocketUnix-0.02.tar.gz) Module < LWP::Protocol::http::SocketUnix::Socket (FLORA/LWP-Protocol-http-SocketUnix-0.02.tar.gz) Module < LWP::Protocol::http::SocketUnixAlt (SHARYANTO/LWP-Protocol-http-SocketUnixAlt-0.0204.tar.gz) Module < LWP::Protocol::http::SocketUnixAlt::Socket (SHARYANTO/LWP-Protocol-http-SocketUnixAlt-0.0204.tar.gz) Module < LWP::Protocol::http::SocksChain (GOSHA/LWP-Protocol-http-SocksChain-1.7.tar.gz) Module < LWP::Protocol::http::SocksChain10 (GOSHA/LWP-Protocol-http-SocksChain10-1.7.tar.gz) Module < LWP::Protocol::http::SocksChain::Socket (GOSHA/LWP-Protocol-http-SocksChain-1.7.tar.gz) Module < LWP::Protocol::http::connect (BENNING/LWP-Protocol-connect-6.09.tar.gz) Module < LWP::Protocol::http::connect::Socket (BENNING/LWP-Protocol-connect-6.09.tar.gz) Module < LWP::Protocol::http::hosts (MASAKI/LWP-UserAgent-DNS-Hosts-0.08.tar.gz) Module < LWP::Protocol::http::socks (SCR/LWP-Protocol-socks-1.7.tar.gz) Module < LWP::Protocol::http::socks::Socket (SCR/LWP-Protocol-socks-1.7.tar.gz) Module < LWP::Protocol::https (MSCHILLI/LWP-Protocol-https-6.06.tar.gz) Module < LWP::Protocol::https::Socket (MSCHILLI/LWP-Protocol-https-6.06.tar.gz) Module < LWP::Protocol::https::SocksChain (GOSHA/LWP-Protocol-https-SocksChain-1.8.tar.gz) Module < LWP::Protocol::https::SocksChain10 (GOSHA/LWP-Protocol-https-SocksChain10-1.7.tar.gz) Module < LWP::Protocol::https::SocksChain::Socket (GOSHA/LWP-Protocol-https-SocksChain-1.8.tar.gz) Module < LWP::Protocol::https::connect (BENNING/LWP-Protocol-connect-6.09.tar.gz) Module < LWP::Protocol::https::connect::Socket (BENNING/LWP-Protocol-connect-6.09.tar.gz) Module < LWP::Protocol::https::hosts (MASAKI/LWP-UserAgent-DNS-Hosts-0.08.tar.gz) Module < LWP::Protocol::https::socks (SCR/LWP-Protocol-socks-1.7.tar.gz) Module < LWP::Protocol::https::socks::Socket (SCR/LWP-Protocol-socks-1.7.tar.gz) Module < LWP::Protocol::ldap (MARSCHAP/perl-ldap-0.65.tar.gz) Module < LWP::Protocol::ldapi (MARSCHAP/perl-ldap-0.65.tar.gz) Module < LWP::Protocol::ldaps (MARSCHAP/perl-ldap-0.65.tar.gz) Module < LWP::Protocol::loopback (ETHER/libwww-perl-6.15.tar.gz) Module < LWP::Protocol::mailto (ETHER/libwww-perl-6.15.tar.gz) Module < LWP::Protocol::nntp (ETHER/libwww-perl-6.15.tar.gz) Module < LWP::Protocol::nogo (ETHER/libwww-perl-6.15.tar.gz) Module < LWP::Protocol::rsync (KRYDE/LWP-Protocol-rsync-1.tar.gz) Module < LWP::Protocol::sftp (SALVA/LWP-Protocol-sftp-0.05.tar.gz) Module < LWP::Protocol::socks (SCR/LWP-Protocol-socks-1.7.tar.gz) Module < LWP::Protocol::socks4 (SCR/LWP-Protocol-socks-1.7.tar.gz) Module < LWP::Protocol::virtual (RPAUL/LWP-Protocol-virtual-0.02.tar.gz) Module < LWP::RobotPUA (MSOUTH/ParallelUserAgent-2.62.tgz) Module < LWP::RobotUA (ETHER/libwww-perl-6.15.tar.gz) Module < LWP::SecureSocket (GAAS/LWP-attic-1.00.tar.gz) Module < LWP::Simple (ETHER/libwww-perl-6.15.tar.gz) Module < LWP::Simple::Cookies (JJORE/LWP-Simple-Cookies-0.01.tar.gz) Module < LWP::Simple::Post (SARGIE/LWP-Simple-Post-0.05.tar.gz) Module < LWP::Simple::REST (GONCALES/LWP-Simple-REST-0.092.tar.gz) Module < LWP::Simple::WithCache (LEEYM/LWP-Simple-WithCache-0.03.tar.gz) Module < LWP::Socket (GAAS/LWP-attic-1.00.tar.gz) Module < LWP::UserAgent (ETHER/libwww-perl-6.15.tar.gz) Module < LWP::UserAgent::AG (DRRHO/RDF-AllegroGraph-Easy-0.01.tar.gz) Module < LWP::UserAgent::Anonymous (MANWAR/LWP-UserAgent-Anonymous-0.09.tar.gz) Module < LWP::UserAgent::AtomClient (MIYAGAWA/XML-Atom-0.41.tar.gz) Module < LWP::UserAgent::CHICaching (KJETILK/LWP-UserAgent-CHICaching-0.04.tar.gz) Module < LWP::UserAgent::Cache::Memcached (SHIRAIWA/LWP-UserAgent-Cache-Memcached-0.02.tar.gz) Module < LWP::UserAgent::Cached (OLEG/LWP-UserAgent-Cached-0.06.tar.gz) Module < LWP::UserAgent::DNS::Hosts (MASAKI/LWP-UserAgent-DNS-Hosts-0.08.tar.gz) Module < LWP::UserAgent::Determined (ALEXMV/LWP-UserAgent-Determined-1.07.tar.gz) Module < LWP::UserAgent::ExponentialBackoff (MARROTTE/LWP-UserAgent-ExponentialBackoff-004/LWP-UserAgent-ExponentialBackoff-0.04.tar.gz) Module < LWP::UserAgent::FramesReady (DERHAAG/FramesReady-1.023.tar.gz) Module < LWP::UserAgent::JSON (SKINGTON/LWP-JSON-Tiny-0.007.tar.gz) Module < LWP::UserAgent::Keychain (MIYAGAWA/LWP-UserAgent-Keychain-0.01.tar.gz) Module < LWP::UserAgent::Mockable (MJEMMESON/LWP-UserAgent-Mockable-1.16.tar.gz) Module < LWP::UserAgent::OfflineCache (GDM/LWP-UserAgent-OfflineCache-0.02.tar.gz) Module < LWP::UserAgent::POE (MSCHILLI/LWP-UserAgent-POE-0.05.tar.gz) Module < LWP::UserAgent::Paranoid (TSIBLEY/LWP-UserAgent-Paranoid-0.97.tar.gz) Module < LWP::UserAgent::Paranoid::Compat (TSIBLEY/LWP-UserAgent-Paranoid-0.97.tar.gz) Module < LWP::UserAgent::Paranoid::Test (TSIBLEY/LWP-UserAgent-Paranoid-0.97.tar.gz) Module < LWP::UserAgent::Patch::FilterLcpan (PERLANCAR/App-lcpan-1.001.tar.gz) Module < LWP::UserAgent::Patch::FilterMirror (PERLANCAR/LWP-UserAgent-Patch-FilterMirror-0.05.tar.gz) Module < LWP::UserAgent::Patch::FilterMirrorMaxSize (PERLANCAR/LWP-UserAgent-Patch-FilterMirror-0.05.tar.gz) Module < LWP::UserAgent::Patch::HTTPSHardTimeout (PERLANCAR/LWP-UserAgent-Patch-HTTPSHardTimeout-0.05.tar.gz) Module < LWP::UserAgent::Patch::LogRequestContent (PERLANCAR/LWP-UserAgent-Patch-LogRequestContent-0.02.tar.gz) Module < LWP::UserAgent::Patch::LogResponse (PERLANCAR/LWP-UserAgent-Patch-LogResponse-0.09.tar.gz) Module < LWP::UserAgent::Patch::Retry (PERLANCAR/LWP-UserAgent-Patch-Retry-0.03.tar.gz) Module < LWP::UserAgent::ProgressAny (PERLANCAR/LWP-UserAgent-ProgressAny-0.06.tar.gz) Module < LWP::UserAgent::ProgressBar (MARCEL/LWP-UserAgent-ProgressBar-1.100810.tar.gz) Module < LWP::UserAgent::ProxyAny (QJZHOU/LWP-UserAgent-ProxyAny-1.01.tar.gz) Module < LWP::UserAgent::ProxyHopper (ZOFFIX/LWP-UserAgent-ProxyHopper-0.004.tar.gz) Module < LWP::UserAgent::ProxyHopper::Base (ZOFFIX/LWP-UserAgent-ProxyHopper-Base-0.003.tar.gz) Module < LWP::UserAgent::RTClient (AUTRIJUS/RT-Client-0.01.tar.gz) Module < LWP::UserAgent::RandomProxyConnect (HVALVERDE/LWP-UserAgent-RandomProxyConnect-1.10.tar.gz) Module < LWP::UserAgent::RedirectNotOk (ETJ/Mail-POP3-3.08.tar.gz) Module < LWP::UserAgent::Role::CHICaching (KJETILK/LWP-UserAgent-CHICaching-0.04.tar.gz) Module < LWP::UserAgent::Role::CHICaching::SimpleKeyGen (KJETILK/LWP-UserAgent-CHICaching-0.04.tar.gz) Module < LWP::UserAgent::Role::CHICaching::SimpleMungeResponse (KJETILK/LWP-UserAgent-CHICaching-0.04.tar.gz) Module < LWP::UserAgent::Role::CHICaching::VaryNotAsterisk (KJETILK/LWP-UserAgent-CHICaching-0.04.tar.gz) Module < LWP::UserAgent::SemWebCache (KJETILK/AtteanX-Query-Cache-0.002.tar.gz) Module < LWP::UserAgent::Snapshot (NPW/LWP-UserAgent-Snapshot-v0.2.tar.gz) Module < LWP::UserAgent::Tor (SPEBERN/LWP-UserAgent-Tor-0.05.tar.gz) Module < LWP::UserAgent::WithCache (SEKIMURA/LWP-UserAgent-WithCache-0.12.tar.gz) Module < LWPx::ParanoidAgent (SAXJAZMAN/lwp/LWPx-ParanoidAgent-1.10.tar.gz) Module < LWPx::ParanoidHandler (TOKUHIROM/LWPx-ParanoidHandler-0.07.tar.gz) Module < LWPx::Profile (CREIN/LWPx-Profile-0.2.tar.gz) Module < LWPx::Protocol::http_paranoid (SAXJAZMAN/lwp/LWPx-ParanoidAgent-1.10.tar.gz) Module < LWPx::Protocol::http_paranoid::Socket (SAXJAZMAN/lwp/LWPx-ParanoidAgent-1.10.tar.gz) Module < LWPx::Protocol::http_paranoid::SocketMethods (SAXJAZMAN/lwp/LWPx-ParanoidAgent-1.10.tar.gz) Module < LWPx::Protocol::https_paranoid (SAXJAZMAN/lwp/LWPx-ParanoidAgent-1.10.tar.gz) Module < LWPx::Protocol::https_paranoid::Socket (SAXJAZMAN/lwp/LWPx-ParanoidAgent-1.10.tar.gz) Module < LWPx::Record::DataSection (MOTEMEN/LWPx-Record-DataSection-0.01.tar.gz) Module < LWPx::TimedHTTP (SIMONW/LWPx-TimedHTTP-1.8.tar.gz) Module < LWPx::TimedHTTP::Socket (SIMONW/LWPx-TimedHTTP-1.4.tar.gz) Module < LWPx::TimedHTTP::https (SIMONW/LWPx-TimedHTTP-1.4.tar.gz) Module < LWPx::TimedHTTP::https::Socket (SIMONW/LWPx-TimedHTTP-1.4.tar.gz) Module < LWPx::UserAgent::Cached (MJGARDNER/LWPx-UserAgent-Cached-0.007.tar.gz) Module < Log::Any::For::LWP (PERLANCAR/Log-Any-For-LWP-0.06.tar.gz) Module < Meta::LWP::Simple (VELTZER/Meta-0.08.tar.gz) Module < Mock::LWP::Request (CLIFFORDJ/Mock-LWP-Request-0.01.tar.gz) Module < Net::Hadoop::WebHDFS::LWP (BURAK/Net-Hadoop-WebHDFS-LWP-0.007.tar.gz) Module < Net::NSS::SSL::LWPCompat (CLAESJAC/Crypt-NSS-0.04.tar.gz) Module < Net::SSLGlue::LWP (SULLR/Net-SSLGlue-1.058.tar.gz) Module < Net::SSLGlue::LWP::Socket (SULLR/Net-SSLGlue-1.058.tar.gz) Module < Nexmo::SMS::MockLWP (RENEEB/Nexmo-SMS-0.09.tar.gz) Module < PML::LWP (PJONES/PML-0.4.1.tar.gz) Module < Padre::Task::LWP (PLAVEN/Padre-1.00.tar.gz) Module < Paws::Net::LWPCaller (JLMARTIN/Paws-0.27.tar.gz) Module < Plack::App::Proxy::Backend::LWP (LEEDO/Plack-App-Proxy-0.29.tar.gz) Module < Plack::LWPish (MIYAGAWA/Plack-1.0039.tar.gz) Module < Plack::Middleware::Debug::LWP (CREIN/Plack-Middleware-Debug-LWP-0.2.tar.gz) Module < Plient::Handler::LWP (SUNNAVY/Plient-0.03.tar.gz) Module < Prophet::Replica::FS::Backend::LWP (IOANR/Prophet-0.751.tar.gz) Module < REST::Neo4p::Agent::LWP::UserAgent (MAJENSEN/REST-Neo4p-0.3012.tar.gz) Module < Search::Elasticsearch::Cxn::LWP (DRTECH/Search-Elasticsearch-2.03.tar.gz) Module < Test::LWP::MockSocket::http (MNUNBERG/Test-LWP-MockSocket-http-0.05.tar.gz) Module < Test::LWP::Recorder (EALLENIII/Test-LWP-Recorder-0.1.1.tar.gz) Module < Test::LWP::UserAgent (ETHER/Test-LWP-UserAgent-0.030.tar.gz) Module < Test::Mock::LWP (LUKEC/Test-Mock-LWP-0.08.tar.gz) Module < Test::Mock::LWP::Conditional (MASAKI/Test-Mock-LWP-Conditional-0.04.tar.gz) Module < Test::Mock::LWP::Conditional::Stubs (MASAKI/Test-Mock-LWP-Conditional-0.03.tar.gz) Module < Test::Mock::LWP::Dispatch (TADAM/Test-Mock-LWP-Dispatch-0.08.tar.gz) Module < Test::Mock::LWP::UserAgent (LUKEC/Test-Mock-LWP-0.08.tar.gz) Module < Test::Nginx::LWP (AGENT/Test-Nginx-0.25.tar.gz) Module < Test::Smoke::Poster::LWP_UserAgent (ABELTJE/Test-Smoke-1.70.tar.gz) Module < Test::VCR::LWP (CREIN/Test-VCR-LWP-0.5.tar.gz) Module < WWW::Chain::UA::LWP (GETTY/WWW-Chain-0.003.tar.gz) Module < WWW::IRail::API::Client::LWP (ESSELENS/WWW-IRail-API-0.003.tar.gz) Module < WWW::Kickstarter::HttpClient::Lwp (IKEGAMI/WWW-Kickstarter-v1.8.0.tar.gz) Module < WWW::ORCID::Transport::LWP (NICS/WWW-ORCID-0.0101.tar.gz) Module < WebService::Amazon::DynamoDB::LWP (TEAM/WebService-Amazon-DynamoDB-0.001.tar.gz) Module < WebService::Async::UserAgent::LWP (TEAM/WebService-Async-UserAgent-0.004.tar.gz) Module < Wx::Perl::FSHandler::LWP (MBARBON/Wx-Perl-FSHandler-LWP-0.03.tar.gz) Module < XML::Compile::SOAP::Daemon::LWPutil (MARKOV/XML-Compile-SOAP-Daemon-3.12.tar.gz) Module < XML::RPC::UA::LWP (MONS/XML-RPC-Fast-0.6.tar.gz) Author PRUIKLW ("lwp" <pruiklw@cpan.org>) 325 items found</pruiklw@cpan.org>
And I can't figure out if one or more fit the bill, a couple are also marked "perl", but I can't really be sure which one might be the right one.
Anyone? :o
-
It appears 'LWP' is a package, whereas 'lwp' is not.
So I just learned that cpan is case-sensitive.
Installing LWP gives a lot of "unsatisfied dependency" errors during install.
The installation seems big (/huge), and probably a bit more than I thought necessary.The last few lines of the install looks like this:
# Looks like you failed 3 tests of 63. t/local/http.t .......... Dubious, test returned 3 (wstat 768, 0x300) Failed 3/63 subtests t/local/protosub.t ...... ok t/net/http-get.t ........ skipped: no net config file t/net/http-post.t ....... skipped: no net config file t/net/http-timeout.t .... skipped: no net config file t/net/mirror.t .......... skipped: no net config file t/net/moved.t ........... skipped: no net config file t/net/proxy.t ........... skipped: no net config file t/robot/ua-get.t ........ ok t/robot/ua.t ............ ok Test Summary Report ------------------- t/base/ua.t (Wstat: 512 Tests: 35 Failed: 2) Failed tests: 13-14 Non-zero exit status: 2 t/local/autoload-get.t (Wstat: 0 Tests: 1 Failed: 1) Failed test: 1 t/local/autoload.t (Wstat: 0 Tests: 1 Failed: 1) Failed test: 1 t/local/http.t (Wstat: 768 Tests: 63 Failed: 3) Failed tests: 30-31, 63 Non-zero exit status: 3 Files=16, Tests=131, 27 wallclock secs ( 0.49 usr 0.15 sys + 13.19 cusr 1.58 csys = 15.41 CPU) Result: FAIL Failed 4/16 test programs. 7/131 subtests failed. *** Error code 255 Stop. make: stopped in /root/.cpan/build/libwww-perl-6.15-_FLDTd ETHER/libwww-perl-6.15.tar.gz 2 dependencies missing (HTML::HeadParser,HTML::Entities); additionally test harness failed /usr/bin/make test -- NOT OK //hint// to see the cpan-testers results for installing this module, try: reports ETHER/libwww-perl-6.15.tar.gz Failed during this command: GAAS/HTML-Parser-3.72.tar.gz : make NO (optional) MIKEM/Net-SSLeay-1.77.tar.gz : writemakefile NO -- No 'Makefile' created (optional) SULLR/IO-Socket-SSL-2.034.tar.gz : make_test NO ETHER/libwww-perl-6.15.tar.gz : make_test NO 2 dependencies missing (HTML::HeadParser,HTML::Entities); additionally test harness failed
After the install, I exit and come back to the the normal pfSense-cli, and I try:
perl unbound-block-hosts
and get the error:
Can't locate LWP.pm in @INC (you may need to install the LWP module) (@INC contains: /usr/local/lib/perl5/site_perl/mach/5.20 /usr/local/lib/perl5/site_perl /usr/local/lib/perl5/5.20/mach /usr/local/lib/perl5/5.20 /usr/local/lib/perl5/site_perl/5.20 /usr/local/lib/perl5/site_perl/5.20/mach .) at unbound-block-hosts line 5. BEGIN failed--compilation aborted at unbound-block-hosts line 5.
Which looks like I have gotten no further.
Now to find out how to undo what I just did in cpan… 8)
-
It appears that's not as easy as getting it in there. ???
(With cpan not being a package manager and all…) :)cpan
followed by:
cpan[1]> o conf cpan_home
gives:
cpan_home [/root/.cpan]
So I would assume that:
rm -R /root/.cpan
(Or some such)
would rid me of the installation of LWP, that didn't work (and possibly reset cpan in the process)?
-
yeah its not a package so much modules that are compiled..
I have not looked into what tweaks or differences that pfsense has done with their perl environment.. That would prevent the module from installing..
If you just delete the .cpan folder you created in /root that should delete the cpan configuration. Since it failed what do you think should have to be removed? I don't think a failed module install will cause any issues with perl and the modules that are installed, etc.
-
If you just delete the .cpan folder you created in /root that should delete the cpan configuration. Since it failed what do you think should have to be removed? I don't think a failed module install will cause any issues with perl and the modules that are installed, etc.
I just wanted to remove the failed module, as I suppose it still takes up space on my nanobsd-install. At least source-files etc. but I cant really find out if it only put something in the /root/.cpan-directory.
But I'm on new ground, as I usually tinker with linux-based systems, so though some things look the same, others surely don't.
Anyways, we're (I'm) veering of course. :)
I think I'll try firing up a VM with a full pfSense, to see if that makes any difference.
-
I've got 419,418 hosts across 9 lists, currently using pfBlockerNG DNSBL Feeds. I don't have a need for any of the other provided features aside from DNSBL. There must be an alternate method to block these hosts through Unbound?