I have tried both Apache / Nginx and Openlitespeed (Both had been optimised). Apache / Nginx worked faster for me, you just have to tune it with caching, brotli / pagespeed / varnish or other modules.
You have Panel + OLS / LS Panel + SSH compared to Panel + SSH, even the open source panel for OLS (have used as well) does not integrate many of the basic features such as adding additional IP address or setting SSL ciphers.