Looking for a pfSense and/or Snort expert to configure my server
-
Hiya all,
I am in the early stages of moving to a pfSense based firewall solution, and want to make sure I get it right the first time - which means hiring an expert to get the initial setup done. Before we know if thats you, let me explain what I have now, and what I want moving forward.
I own a VPS hosting company and currently have 5 servers of various sizes at a single datacenter in the USA.
Currently the 5 servers each have their own network drop on 2 different Vlans (my oldest server is on a legacy set of IP addresses that need to change in order to get that server on to the same Vlan as the rest of my servers).
All 5 servers are also connected by a private static (no DHCP) LAN.
Currently I rely on Windows firewall for security, and am being hit by SQL brute force attacks on the SA account about 3-5 times per second from various IPs in China. I have a small C# app that runs on the SQL server that detects these attacks (by monitoring the windows event log for SQL failed login notices to user SA), and adds the attacking IP(s) to a windows firewall rule that blocks them. Moving forward I want all my servers protected on a higher level.
One of my servers is a 1U Pentium 4 duel core box with 2 NICs currently, one connected to the internet, one to my LAN. I had windows 2008 R2 installed on it to function as a backup storage server only, but have today moved it to VMWare Vsphere 5 and installed one windows VM that will do the backup work, and will make another VM for the firewall/IDS.
Now, I am fairly certain that I will need to add one more NIC to that box for a total of 3… but thats one of the things I want my "expert" to suggest.
Moving forward I want a single network drop in to this 1U server, and have my other servers connect transparently thru it via a new LAN that I will have set up.
Now, it is very important that this is transparent - meaning that the computers behind the "firewall" can access the Vlan that goes in to the firewall. Over my servers I have more than 200 IP addresses, and they are constantly being rotated between servers as needed - that functionality needs to remain - My whole business model relies on it.
Now, I need a firewall system that will either:
Detect SQL/DNS attacks/abuse and block them, or
Allow me to add IPs to the firewall block list remotely from the other servers as needed via an API of some sort.This is where the snort idea came in to play... I know nothing about either pfSense or Snort, and I am not a linux person so playing with them and learning is a lot more time consuming for me than I can afford. So... I want to hire someone from here to do it for me, as well as to suggest a network framework that would best support it.
For instance, my plan is to keep the current local static LAN (with hard-coded 10.5.. IP addresses and no DHCP), for internal server communication, and add a new LAN to handle internet/WAN traffic. If there is a better way, I am all ears :)
Please only reply if you have the experience I need - I do not need someone who has never set up a mission critical server. When I have my datacenter shut down the network and plug this box in to place, it needs to be perfect, or close enough to get to "perfect" within 20 minutes or so (in other words we can tweak things a little, but it really needs to be ready when we move to it).
I still need to order the new switch and likely a 3rd network card for this little server, so we have some time - a week or so, but I would like to find someone and begin planning/working right away, starting with a suggestion on if I should stick with stock pfSense or go with a Snort variant, etc.
If you are interested, please contact me here.
Thanks,
Dave (Frosty)
-
A word of caution on the snort package, you do need to monitor it closely as I've seen the process fail and not automatically resume.
It seems the snort package is more buggy than most and is in need of some more work. -
Currently I rely on Windows firewall for security, and am being hit by SQL brute force attacks on the SA account about 3-5 times per second from various IPs in China. I have a small C# app that runs on the SQL server that detects these attacks (by monitoring the windows event log for SQL failed login notices to user SA), and adds the attacking IP(s) to a windows firewall rule that blocks them. Moving forward I want all my servers protected on a higher level.
One of my servers is a 1U Pentium 4 duel core box with 2 NICs currently, one connected to the internet, one to my LAN. I had windows 2008 R2 installed on it to function as a backup storage server only, but have today moved it to VMWare Vsphere 5 and installed one windows VM that will do the backup work, and will make another VM for the firewall/IDS.
Now, I am fairly certain that I will need to add one more NIC to that box for a total of 3… but thats one of the things I want my "expert" to suggest.
Moving forward I want a single network drop in to this 1U server, and have my other servers connect transparently thru it via a new LAN that I will have set up.
Now, it is very important that this is transparent - meaning that the computers behind the "firewall" can access the Vlan that goes in to the firewall. Over my servers I have more than 200 IP addresses, and they are constantly being rotated between servers as needed - that functionality needs to remain - My whole business model relies on it.
Now, I need a firewall system that will either:
Detect SQL/DNS attacks/abuse and block them, or
Allow me to add IPs to the firewall block list remotely from the other servers as needed via an API of some sort.This is where the snort idea came in to play... I know nothing about either pfSense or Snort, and I am not a linux person so playing with them and learning is a lot more time consuming for me than I can afford. So... I want to hire someone from here to do it for me, as well as to suggest a network framework that would best support it.
For instance, my plan is to keep the current local static LAN (with hard-coded 10.5.. IP addresses and no DHCP), for internal server communication, and add a new LAN to handle internet/WAN traffic. If there is a better way, I am all ears :)
Please only reply if you have the experience I need - I do not need someone who has never set up a mission critical server. When I have my datacenter shut down the network and plug this box in to place, it needs to be perfect, or close enough to get to "perfect" within 20 minutes or so (in other words we can tweak things a little, but it really needs to be ready when we move to it).
I still need to order the new switch and likely a 3rd network card for this little server, so we have some time - a week or so, but I would like to find someone and begin planning/working right away, starting with a suggestion on if I should stick with stock pfSense or go with a Snort variant, etc.
If you are interested, please contact me here.
Thanks,
Dave (Frosty)
You dont say what SQL server you are running but I am guessing MS SQL server becuase of the SA reference?
If so can I ask why you are running SQL Server in mixed mode with the SA account enabled? Are you exposing the 1433/4 port direct to the net or do you have an interface (website) before it as well and if so is it handling SQL injection properly?
Likewise can you move the sql ports to a different address range to make it less likely for scanners to find your open ports?
What version of windows are you using as well? SBS variant or a straight server os of sorts?You can do some basic blocking in MS Firewall but it depends on which OS you are using, if using SBS for example you can albeit in a time consuming way put in ipblocks into isa server, but if you are using a std server OS, then its a little more basic.
I would still secure your server & sql server using whats available on those machines as a backup in the off chance something happens to pfSense/Snort, but thats just me ultra cautious.
Personally I would say familiarise yourself with pfsense, you can boot from the LiveCD on any PC so it runs in memory and you can backup your settings to a memory stick so you can come back to it and play with it again.
I dont think its very hard to setup pfsense at all compared to some of the other firewalls I have seen, its quite intuitive in lots of ways imo but we are all different so others might not agree, but bear in mind if you are managing the other hardware and something happens you might not always be able to get hold of an expert asap so some basic knowledge of pfsense is always good in my books.
Where are you based so others can see if you are in their neighbourhood or not and maybe able to help as its not the easiest of tasks to do remotely I would say. :-)
-
It is SQL Server, and it is exposed.
Yes, that is a newbie mistake, but one I cannot easily change right now. That server has been live for about 12 years, and 2 of my oldest customers (the reason I set that server up in the first place) have a multitude of applications that run with direct SQL communication using "sa". Back when we developed those applications we thought nothing of it… a few years later when I was doing some work on the server and noticed it, it was too late. I would have to re-write several applications - some of which were written in VB 6 which I do not have the platforms for anymore, to get even the port changed. Even worse, several of those apps run on administrator laptops that move from location to location, so no static IPs.
The only good thing about it is that the "sa" password is a friggin monster. No way I could ever remember it no matter how many times I type it (its VERY long and totally random upper case, lower case, numbers and symbols). So I cannot change the port or move to a different user account, even tho that would be best. I am trying to talk them in to letting us do a total re-write of their apps using new technology, but thats a big job - will take all my developers a year or so at a cost of 2m, give or take. It originally was just me and 2 others and it took us 3 years at a cost of over 1m back then. No, making changes will not be easy and will not happen quickly enough for me.
The SQL Server is running on Windows 2008 Standard, fully updated. My backup server is Windows 2008 R2 Standard, and my VPS servers run on Windows 2008 R2 Enterprise. I do currently use the windows firewall , and have a custom app I wrote that monitors the windows event log for failed logins to "sa" and blocks the IP that failed. I keep those blocks for 48 hours... and there are ~300 IPs in the block list at any given time.
In addition to the SQL issue, I run a DNS server as well, and have been getting hit on that up to 40 times per second from Chinese IPs on requests for "any". I block these out of hand, and block any IP that makes more than 10 requests per second, but want a better solution. My original thought (and how I found out about pfsense in the first place) was to block all chinese IPs period. I still plan to do that. I do not have any customers in China, and likely never will. China in my opinion is a security risk. I would say the same about S. Korea, since there are a lot of things coming from there too, but lots of good comes out of there as well. Nothing good (as far as the internet goes) comes from China... so I am going to block every subnet in china. I found pfsense and this forum by posting just that on hardforum. I am pretty sure that the hack attempts on my server (and the DNS attacks, if thats what they are - its not very strong for a DOS attack, and usually only 1 IP at a time, so its not a DDOS) will stop. On the day I made this decision (last week), I checked all the IPs in my block list. There were 318 of them, and all but 3 resolved to China. Before you ask, yes I have communicated this with my customers and they all say "go for it". The one guy that does travel to china says he has a VPN he uses while hes there anyway, so no harm no foul. When this gets set up, its primary function will be to block china from getting to my servers on any port. :)
I also agree that pfsense is pretty simple... but some of what I need done is not... like the transparency. Thats why I want to hire an expert. Better to pay someone to make sure it gets done right.
Frosty
-
It is SQL Server, and it is exposed.
Yes, that is a newbie mistake, but one I cannot easily change right now. That server has been live for about 12 years, and 2 of my oldest customers (the reason I set that server up in the first place) have a multitude of applications that run with direct SQL communication using "sa". Back when we developed those applications we thought nothing of it… a few years later when I was doing some work on the server and noticed it, it was too late. I would have to re-write several applications - some of which were written in VB 6 which I do not have the platforms for anymore, to get even the port changed. Even worse, several of those apps run on administrator laptops that move from location to location, so no static IPs.
In that case, can you use a VPN? That will allow you to restrict access to just the VPN and avoid the need to re-write the applications in the short term. Can you use the country block package to block countries your customers aren't in?
The only good thing about it is that the "sa" password is a friggin monster. No way I could ever remember it no matter how many times I type it (its VERY long and totally random upper case, lower case, numbers and symbols).
That doesn't stop it being brute forced. You do have protections in place to stop (or at least spot) that?
I also agree that pfsense is pretty simple… but some of what I need done is not... like the transparency. Thats why I want to hire an expert. Better to pay someone to make sure it gets done right.
Then you may want to use the consultancy option.
-
Yes, that is a newbie mistake, but one I cannot easily change right now. That server has been live for about 12 years, and 2 of my oldest customers (the reason I set that server up in the first place) have a multitude of applications that run with direct SQL communication using "sa". Back when we developed those applications we thought nothing of it… a few years later when I was doing some work on the server and noticed it, it was too late. I would have to re-write several applications - some of which were written in VB 6 which I do not have the platforms for anymore, to get even the port changed. Even worse, several of those apps run on administrator laptops that move from location to location, so no static IPs.
VMware or Virtualbox will let you run your old dev environments. I've got Dos & 16bit dev tools which dont run on my 64bit dev machine unless I use some sort of virtualisation software like the aforementioned so provided you have the source, you could bring back to life your old dev environments, virtualisation is also how some of my customers still run their dos apps when their old boxes have died and they needed to get them working on new machines. I've done very little VB programming in my time preferring other languages & tools but if you have hardwired the SQL ports into the SQL connection string then you can search and replace your code and change the port number that way, shouldnt take long to do at all! I dare say there will be more time spent setting up the environment again, making sure any addons you use are installed etc etc before compiling rovided you have all the code.
I am trying to talk them in to letting us do a total re-write of their apps using new technology, but thats a big job - will take all my developers a year or so at a cost of 2m, give or take. It originally was just me and 2 others and it took us 3 years at a cost of over 1m back then. No, making changes will not be easy and will not happen quickly enough for me.
I agree with Cry Havok, spend some money and also explore the VPN route as a secondary defense to protect your $/£/Eur1-2m investmen . With some of the stuff I have seen you want to protect your investment and time. Good luck!
On the subject of rewriting your apps, which new technology would you be looking at? I'm always looking for some new dev tools to play with and I do a bit of db work myself.
-
VMware or Virtualbox will let you run your old dev environments. I've got Dos & 16bit dev tools which dont run on my 64bit dev machine unless I use some sort of virtualisation software like the aforementioned so provided you have the source, you could bring back to life your old dev environments, virtualisation is also how some of my customers still run their dos apps when their old boxes have died and they needed to get them working on new machines. I've done very little VB programming in my time preferring other languages & tools but if you have hardwired the SQL ports into the SQL connection string then you can search and replace your code and change the port number that way, shouldnt take long to do at all! I dare say there will be more time spent setting up the environment again, making sure any addons you use are installed etc etc before compiling rovided you have all the code.
I agree with Cry Havok, spend some money and also explore the VPN route as a secondary defense to protect your $/£/Eur1-2m investmen . With some of the stuff I have seen you want to protect your investment and time. Good luck!
On the subject of rewriting your apps, which new technology would you be looking at? I'm always looking for some new dev tools to play with and I do a bit of db work myself.
I am a .NET Solution Architect, so I would re-write them all in .NET, mostly C#, with a WCF backend that will replace the need for direct SQL communication. Remember these were written 12+ years ago (development started about 14-15 years ago). At the time SQL attacks like this were rare, and since this was at that time a private server with only the 2 clients (not public - nobody knew about it), it was considered safe. The database has been upgraded, as has the server, but the applications have not. There are several of them that I do not have the source code for anymore, tho I do have most. The ones I do not are ones that were developed by 3rd parties, but still in use today. Its a mess.
If you are a developer, .NET is where its at. I started in .NET with version 1 years ago, and my profit margins went up considerably from that point. My average hourly rate now is $150, and I am semi-retired and living in Thailand doing my work remotely. I am sure that if .NET had not come out, I would still be plugging away at my Tulsa office with the rest of my employees lol. It takes 1 person with .NET a day to do what it would take a week for a team of 3 to do back then. Gotta love it :)
As for a VPN, thats a no go. I tried talking them in to that over a year ago, but they have too many locations, some of them mobile, and the employees that use them are idiots. The long and the short of it is I need to protect it as best I can with what I have… which means limiting the possibility of a brute force attack being successful. The less attacks that get thru, the better. My app manages that pretty well, but I would much rather have most of the blocking happen before it gets to the server, hence the move to a firewall appliance.
As I said, the vast majority of my issues are coming from China. This solution will do away with that... and perhaps give me the time I need to come up with a more viable solutions (like re-writing their apps). Its a contract I really want, so I am working on that... but the company is being a pain lol. Of course if their server was hacked maybe they would change their minds... but I would rather not put that to the test :)
Frosty
-
There are several of them that I do not have the source code for anymore, tho I do have most. The ones I do not are ones that were developed by 3rd parties, but still in use today. Its a mess.
Ah, thats never helpful when source has dissappeared.
As for a VPN, thats a no go. I tried talking them in to that over a year ago, but they have too many locations, some of them mobile, and the employees that use them are idiots. The long and the short of it is I need to protect it as best I can with what I have… which means limiting the possibility of a brute force attack being successful. The less attacks that get thru, the better. My app manages that pretty well, but I would much rather have most of the blocking happen before it gets to the server, hence the move to a firewall appliance.
As I said, the vast majority of my issues are coming from China. This solution will do away with that... and perhaps give me the time I need to come up with a more viable solutions (like re-writing their apps). Its a contract I really want, so I am working on that... but the company is being a pain lol. Of course if their server was hacked maybe they would change their minds... but I would rather not put that to the test :)
Frosty
One thought as another workaround although others would need to chip in as it depends on how controllable pfsense is in realtime, is to have the SQL ports shut by default, then give all your customers a small app (I'll call it the shell app for now) which connects to a secure port of your choice on your server to act as an authenticator(nightclub bouncer) service of sorts which then updates the rules in pfsense in realtime. At the same time the shell app then calls the main app the user wants to run using ShellExecuteEx or the equiv in .net. Think of the shell app like a menu interface app, ie they click one shortcut to load the shell app and then they choose which program they want to run (assuming the option to use a number of apps is available to them) from inside the shell app.
What this would give you is their current IP address thus getting around the variable IP address problem and you can authenticate them as genuine users that way.
You should be able to alter the Keep-Alive as well to maintain the ip-address and connection between your servers and your shell app, this is what active-sync (push email) does with smart phones connecting to MS Exchange server does so it works well as I see very little reconnects from mobile phones. In ISA server on some of my customers sbs machines, I can see numerous connections which have been alive for days and hours but your mileage might vary depending on your customers telecoms companies, some ISP might impose a disconnect after X number of hours on DSL and the same might apply to 3G/GPRS but I havent seen this here in the UK. The mobile telco's handle the connections between cells quite well if the user is moving around like on a train but altering the keep-alive is a large factor in making this work.
Anyway having got the users IP address you can then get the club bouncer service to update the rules in pfsense to open the SQL ports for your customers current ip address. When the connection between the shell app breaks, have the club bouncer service on your servers update pfsense so the rule is removed and (future) connection is closed until they initiate a connection again from your shell app.
What I dont know (havent tested) is if pfsense can kill the connection that might have moved onto another port after the initial connection/handshake has taken place on the ports 1433/4. I know with HTTP after the initial connection with port 80, the commmunication moves to another port and continues there but dont know if the same applies to SQL communication as I've never looked into that before.
Anyway a possible work around which would help make your server a little more secure imo although if anyone else begs to differ please chip in as I'm always learning myself and there might be something I'm not aware of. :)
-
I assume you also know how you can control a browser from another program/service to fill in forms from another program using a browser like IE, Firefox, and how to get the URL's out of them using COM or API's like FindWindowEx and SendMessage with WM_GetText as well.