It is a very high time even for an authenticated email. Normally 1 to 4 seconds is reasonable but still uncomfortable. In the meantime the user is annoying and impatient. It has to update the page, makes duplicate submissions, etc, generating even more traffic.
One tactic to reduce this cost of waiting time on the user’s side is to generate a server schedule to send right away in the first minute. So the request will return to the user’s browser in a few milliseconds. Meanwhile, the scheduling on the server will "take care" of the execution. Alternatively to scheduling is to create a background run because schedules depend on hosting server permissions.
About the problem described in the question it is impossible to determine what really happens. It can be bad server environment configuration or simply a limited capacity.
Due to the high waiting time, I suggest you refer to server support as it is not a normal waiting time even on cheap shared servers.
Also be aware that email authentication depends on a third server, the SMTP server you are authenticating with, so it may be a failure or congestion on that SMTP server and it may be something momentary. You can test this by configuring the upload to other distinct SMTP servers. For example, if you are using gmail, test with live mail, yahoo or any other particular. If everyone else takes too long, then it is likely that there is something "abnormal" in the hosting server.
Anyway, sending email always costs a while. Try to create scheduling or background execution techniques (asynchronous).
For asynchronous execution, see: Run parallel process in PHP
The test you talk to with gmail, live mail and yahoo mail would be the recipients, right? because I’ve seen some examples of them being used as "host," I don’t quite understand
– Thiago
vc seems confused as it is mixing things up.. The test is to replace the sender authentication smtp..
– Daniel Omine