AWS S3 Sync Not Working
-
Hi everyone,
I wonder if anyone has come across this issue before?
Background
We have well over 100 AWS accounts which we setup for customers who run our ERP software from within AWS. For a number of years we have been using a Vyatta instance for VPN connectivity when the customer has multiple branches, which is a lot cheaper than multiple AWS VPN's. The issue is that the old Vyatta doesn't support IKEv2, so I looked for a replacement.
Initially I looked at VyOS, which did seems to work OK apart from a few niggles, but then chose to adopt pfSense due to better OpenVPN support.The Issue
For one new customer, I'd already setup a VyOS device which was working just fine, and then replaced it with a pfSense device which all seems to be working fine, however.We have a script which backs up the PostgreSQL databases, and then uploads the backup files to an S3 bucket which has versioning enabled, so we can go back to various days in the event that historic data is required.
For some reason the 'aws s3 sync' command which uploads the files has stopped working. It was working just fine with a VyOS and Vyatta, and stopped working on the day I replaced it with a pfSense.
This is where it gets interesting as when you run the backup script manually (as the postgres) it runs just fine and the files upload, but when it runs via a daily cron job it doesn't upload :-(
Before you say it, this is not a pathing issue as the same script has been running OK for years on multiple systems.
The only change is the replacement of the VyOS with a pfSense. I know for a fact that if I replace the pfSense back with the VyOS it will start working again.So my question is, what is causing the problem? I've been looking at this for days and getting nowhere!
There is nothing in the Firewall log to suggest it's being blocked, but surely if it was a FW issue it would be blocked when running manually?
At some point we will need to replace all the Vyatta instances (over 100) with pfSense devices, so need to know these are as solid as the Vyatta appliances they will be replacing or I need to go back to a VyOS?
Thanks
Martin