The PAM error is usually from it just not matching the key. Either because the correct key isn't present, or what's in the file isn't valid.
The keys are on disk on the firewall in the expected location for a user, if you login with a password they would be in ~/.ssh/authorized_keys. If you login as admin or root, then use ~<username>/.ssh/authorized_keys, or look under /home/<username>/.ssh/authorized_keys.
The firewall will manage the content of that file and its permissions so those are not likely to be concerns.
These issues almost always boil down to a problem with the formatting of the key, either line breaks in the key, extraneous whitespace, or the wrong string pasted in (For example we've seen people paste in the fingerprint, not the public key)
There is also a slight chance that ssh-keygen in Cygwin is generating a bad key somehow. Using a more modern format like ssh-keygen -t ed25519 may help instead of using RSA format.