If I got it right, seems like a peculiar necessity.
Anyway, I'd go with nginx + varnish (if you need caching) combination, and multiple subdomains also. Taking into consideration that remembering an URL is simpler than remembering a port number when it's about end-user interaction.
But if you still want to go with varnish and in the way you want it, you need to put its daemon listening on all the ports you want (in your example, 80, 8080, 8081). Then define the all backends you have serving varnish. And, finally, you just set the flux and point the correct backend for the request, based on the server.port attribute. This last one done through the vcl_recv subroutine.
I have no means to test it, but should be something like:
sub vcl_recv {
if (server.port == 8080) {
set req.backend = webserver2;
}
elsif (server.port == 8081) {
set req.backend = webserver3;
}
else {
set req.backend = webserver1;
}
}
This documentation can help you with it: https://www.varnish-cache.org/docs/trunk/users-guide/vcl-backends.html
If it doesn't work, at least I hope it guides you to the right path. :)