Hello Micha,
Your issue seems very similar to mine. See the thread below.
http://forum.pfsense.org/index.php/topic,15300.0.html
So far, I have not been able to resolve the issue. However, I have made a similar discovery. I had a terminal ping going in a session for 36 hours. Once the tunnel is up it seems to be very stable. I too have to attempt multiple times. However, I did notice that while that one connection was active, I was much more likely to be able to establish a connection from another client on the first try.
I know that this information will not help you as such but it may be wise to watch each other's threads to see where things go.
I have actually decided to purchase a support contract and will be opening a case if I cannot figure this out. At this point it seems unlikely that I will resolve myself. I seem to have hit a brick wall due to my level of expertise it seems.
Good luck
Wayne