download list of urls to whitelist



  • Hey,

    I've been trying to use pfblocker to create an alias that I can use in a rule to whitelist them.

    The URL's in question come from the following URL:
    https://www.archlinux.org/mirrorlist/?country=all&protocol=https&ip_version=4&use_mirror_status=on

    I was hoping to use pfblocker to create an alias that I could use my own rule.

    I am already using pfblocker this way with IP lists, I let it create aliases and I use them in my own rules.

    This list are URL's, I tried in DNSBL feed to create an entry, I get an alias that I can use in a rule, but its not working.

    In pfsense I can create an URL alias too, when I try to enter the URL there it says it is not a valid URL. When I use curl on the commandline, I get the complete list that I need.

    If anyone has any ideas I'd appreciate it.

    Thanks!


  • Moderator

    I have plans to add this at some point, so for now, you can manually add this to the DNSBL Whitelist customlist:

    fetch -o /tmp/file.txt "https://www.archlinux.org/mirrorlist/?country=all&protocol=https&ip_version=4&use_mirror_status=on"
    
    grep '=' /tmp/file.txt | cut -d '/' -f3
    
    mirror.koddos.net
    ftp.lysator.liu.se
    mirrors.kurnode.com
    mirrors.dotsrc.org
    mirrors.nix.org.ua
    archlinux.us-west.mirror.zoidplex.net
    arlm.tyzoid.com
    ftp.lanet.kr
    archimonde.ts.si
    mirror.wormhole.eu
    archlinux.mirror.wearetriple.com
    ftp.fau.de
    arch.eckner.net
    mirror.i3d.net
    mirrors.arnoldthebat.co.uk
    arch-mirror.wtako.net
    www.ratenzahlung.de
    archlinux.us-east.mirror.zoidplex.net
    mirror.hackingand.coffee
    mirror.metalgamer.eu
    mirror.lty.me
    mirror.smith.geek.nz
    mirror.fra10.de.leaseweb.net
    ftp.sh.cvut.cz
    fooo.biz
    arch.mirror.constant.com
    mirrors.rutgers.edu
    archlinux.mivzakim.net
    mirror.pseudoform.org
    archlinux.thelinuxnetworx.rocks
    mirror.system.is
    muug.ca
    mirrors.n-ix.net
    archlinux.beccacervello.it
    mirrors.cat.net
    repo.itmettke.de
    packages.oth-regensburg.de
    mirror.netcologne.de
    mirrors.phx.ms
    mirror.csclub.uwaterloo.ca
    sgp.mirror.pkgbuild.com
    pkg.adfinis-sygroup.ch
    mirror.hactar.xyz
    gluttony.sin.cvut.cz
    mirrors.neusoft.edu.cn
    ftp.heanet.ie
    arch.mirror.square-r00t.net
    mirrors.celianvdb.fr
    arch.yourlabs.org
    jpn.mirror.pkgbuild.com
    mirror.puzzle.ch
    linuxmirrors.ir
    repo.sadjad.ac.ir
    archlinux.mirror.pkern.at
    mirror.epiphyte.network
    mirror.dkm.cz
    mirror.aarnet.edu.au
    archlinux.eu.mirror.zoidplex.net
    mirrors.kernel.org
    ftp.jaist.ac.jp
    mirrors.ukfast.co.uk
    mirror.thomaskilian.net
    archlinux.dynamict.se
    mirror.srv.fail
    mirrors.nxthost.com
    mirror.sfo12.us.leaseweb.net
    mirrors.sjtug.sjtu.edu.cn
    mirror.lnx.sk
    ftp.rnl.tecnico.ulisboa.pt
    foss.aueb.gr
    mirror.osbeck.com
    mirror.sergal.org
    archlinux.layer8.fail
    mirrors.rit.edu
    mirrors.atviras.lt
    www.mirrorservice.org
    ftp.halifax.rwth-aachen.de
    mirrors.sonic.net
    archlinux.asia-east.mirror.zoidplex.net
    mirror.kku.ac.th
    mirrors.lug.mtu.edu
    archlinux.mailtunnel.eu
    mirror.reisenbauer.ee
    archlinux.nautile.nc
    archlinux.ip-connect.vn.ua
    ind.mirror.pkgbuild.com
    mirror.michael-eckert.net
    mirrors.niyawe.de
    mirror.yandex.ru
    mirror.f4st.host
    mirror.0x.sg
    archlinux.vi-di.fr
    ftp.spline.inf.fu-berlin.de
    mirror.ams1.nl.leaseweb.net
    mirrors.xjtu.edu.cn
    mirror.kaminski.io
    mirrors.uni-plovdiv.net
    mirror.bethselamin.de
    mirror.t-home.mk
    mirror.homelab.no
    arch.jensgutermuth.de
    mirror.ubrco.de
    ftp.wrz.de
    mirror.neostrada.nl
    mirror.armbrust.me
    ftp.myrveln.se
    mirror.xtom.com.hk
    mirror.neuf.no
    mirror.dal10.us.leaseweb.net
    mirrors.ocf.berkeley.edu
    k42.ch
    mirrors.ustc.edu.cn
    mirrors.pidginhost.com
    mirror.one.com
    archlinux.nullpointer.io
    mirror.orbit-os.com
    mirror.thekinrar.fr
    mirror.jankoppe.de
    mex.mirror.pkgbuild.com
    mirror.stephen304.com
    ftp.acc.umu.se
    mirror.aur.rocks
    mirrors.shu.edu.cn
    mirror.datacenterlight.ch
    archlinux.thaller.ws
    mirrors.tuna.tsinghua.edu.cn
    archlinux.us-central.mirror.zoidplex.net
    mirror.wdc1.us.leaseweb.net
    archlinux.surlyjake.com
    glua.ua.pt
    mirror.grig.io