In order for your roadwarrior clients to access resources @ site B, two things need to happen:
Site A's road warrior clients need to know that site B's LAN subnet should be routed down the tunnel
Site B needs to know where to send the return traffic for site A's road warrior clients
Based on the above, the following adjustments should be made to the configs:
Site A:
Road Warrior config should have "192.168.20.0/24, 192.168.10.0/24" on the
IPv4 Local network(s) line. (Remove 10.0.20.0/24).
Site B:
Re-verify the site-to-site config has "192.168.20.0/24, 10.0.20.0/24" on the
IPv4 Remote network(s) line
Once the site-to-site tunnel is re-established and the clients re-connect, you should be good to go.