Why not simply reconfigure those routers
Because some devices (not mine) directly connected to router 1 have in their routing table certain rules to redirect traffic through 10.1.0.4. Hence those routers need to be on the same subnet.
These routers are shared by around 20 people, in 4 rooms on single floor. Hence I cannot change settings on those routers.