Dual WAN conectivity issues (static and pppoe) without loadbalancing or failover
I have some weird problems, thought I'll figure this; but no, as usual, I suck at networking :) So I have 2 internet connections, from the same ISP, and I am trying to configure one pfSense box (an old HP server) for both. But I am not after failover or something, on the contrary! I don't need anything from my the subnet of the servers to get into my administration and home LAN and viceversa. I am after segregation of the two and you realise that I can't find anything usefull, everything Google spits usually regards loadbalancing, if you are searching "multi wan" and the sorts.
So, for the first pair of WAN (WANQ) and LAN: I have a subnet /27 of 32 ips routed by my ISP through an IP, working for years, with some servers in this "LAN" of static IPs, pfSense is their gateway, using the dual port gigabit card integrated in the motherboard. Here should be no NAT and these are public IPs as mentioned. Worked fine till now.
On the second pair of WAN (WANL) and LAN using another DP Gb card, added today, I have a pppoe WAN connection (no router/modem in front of pfSense from the ISP), and I just need a regular LAN, NATed, with DHCP so on for the machines at my home for internet access. Which should talk to the outside world through it's own gateway. BTW, I also connect the second NIC from my servers for administration and transfer purposes in the same switch (have different VLANS in the switch, of course) on the same 192.168.1.0/24 subnet. But shouldn't matter.
It is eating the heart of me. I finally made it work somehow… but man.... can't be sure of anything as I can't access my servers, nothing, through the internet, from my internal network. I don't know why is this happening, I can't ping them. The servers seem OK on the static WANQ, tests from the outside world tell me that that part it's still working; and I have internet on all the machines from the pppoe WANL (later edit: NOT ANYMORE!).
I just want to use the same reliable pfSense router for two Gb connections, isolated one from another, with their own roles, and not some extra crappy Linksys costing it's weight in gold (my EA6700 died on me, slowly in about 3 months of errors and weirdness, and it's speeds were... nevermind).
I am posting some screenshots to help a potential benevolent knowledgeable guy here understand the shit I've done. This is the only way I could make it work somehow.
Any advice would be most welcome. Or better, make it a screenshot, kinda' slow here, networking wise :D Tks!
This is my understanding of how to do that setup:
When adding the WAN with the routed subnet, do not enter an upstream gateway on the interface config page. Only add it as a gateway on the System > Routing page. This will make it not do NAT. Configure a LAN interface with the subnet provided by the ISP. On that LAN interface, all firewall rules that apply to outbound internet traffic must have that gateway manually specified in each firewall rule.
Add the other WAN as normal. Make that interface's gateway the default gateway in System > Routing. Configure your other LAN firewall rules as normal.
Was just about to post some new info, while you replied (thanks!).
BTW it worked for the subnet exactly as you described, no NAT, for a long time, until I messed things with the second pppoe connection. And the 10.0.0.1 gateway it's a weirdness of my ISPs pppoe, don't remember/care why, read a long time ago, it's fine - always had that, and so do other people, some posted here on the pfsense forums.
I know what you are saying - but, it didn't work, already tried adding the gateways for some policy routing, everything went down. Now I found that it is working only if I tick the "Default gateway"; if it is enabled just for the pppoe line… fails miserably otherwise in any other combo. And even so, I still can't access a single server in the subnet from the home LAN. For the pppoe I would expect that such a connection would exit via it's own gateway and interface... than comeback on the other one. It's like it would try internally to solve this? I am sure it's a rule or two but I have no idea for now how to do it. At least because I don't understand what's wrong.
I still can't access my websites - except this, it's fine for the most part. I have internet access in the home LAN, servers services are accessible for the outside world. But not by me, from the home LAN. Here are some new screenshots with the current setup. The big chunk of rules are in the floating area - but only for blacklists, whitelists, the pfblockerNG and so on, nothing it could interfere.
Later edit: NO IT WASN'T OK! Weird things happened, from random external locations no services were accessible from the servers. Had to drop the whole ideea for now.
Any thoughts, someone? It's the third day without access to the subnet… I really should do some work on my websites :D
PS: disabled Reserved networks on both WANs just for a test. The same, can't ping an IP.
Later edit: dropped the pppoe connection for now as I can't "function" this way - still waiting for some feedback... TKS!
This is the worst thread I ever started :D talking to myself and stuff :D
It's not interesting?
Is it plain stupid and/or un-achievable?
No one ever tried such a thing (I don't think it's THAT uncommon)?
Can someone at least give me a hint if it is possible and where/how should I start with this?
For example if I ask "Do I have to use VLANS in pfsense?" could someone just answer "Yes" or "No"? And so on :D ?
Bumping this one more time. Any ideeas?