I think you should not nurture the use of an IP for accessing your site at all. it is the wrong way to do.
as soon as you have two domains or different pages under that IP, how would you want to decide which one should appear under the IP? the times where one IP could only be used with one domain and page are long gone 
I understand that, if you run a very single domain/page on your server, one might think about what’s happening if someone types the IP address. and, while no one is going to do so in the first place, I think they should end up on a neutral page if at all and never on one of your websites.
you can’t use pure IPs with ssl properly and without browser warnings. also urls with IP instead of domain might even hurt SEO for your main page if ever bots stumble across it and so on.
leave the IP be just a placeholder and do not even promote it, but instead only ever link everything with your domain. imagine what happens if people bookmark your page by IP and then you switch servers…
if you really lose sleep over it, I’d then put an .htaccess 302 redirect into /var/www/html that points to your real domain.