Running a websockets server

I’m trying to create a websockets server on a subdomain.

As I understand it, I need a new template for nginx. I have:

.tpl

server {
        listen      %ip%:%proxy_port%;
        server_name %domain_idn% %alias_idn%;

        location / {
                proxy_pass http://%ip%:%web_port%;
                proxy_set_header Host $host;
                proxy_set_header X-Real-IP $remote_addr;
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        }

        location /ws/ {
                proxy_pass http://127.0.0.1:9090;
                proxy_http_version 1.1;
                proxy_set_header Upgrade $http_upgrade;
                proxy_set_header Connection "Upgrade, Keep-Alive";
                proxy_set_header Host $host;
                proxy_set_header X-Real-IP $remote_addr;
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        }
}

.stpl

server {
        listen      %ip%:%proxy_ssl_port% ssl;
        server_name %domain_idn% %alias_idn%;

        ssl_certificate     %ssl_pem%;
        ssl_certificate_key %ssl_key%;
        ssl_stapling        on;
        ssl_stapling_verify on;

        location / {
                proxy_pass http://%ip%:%web_port%;
                proxy_set_header Host $host;
                proxy_set_header X-Real-IP $remote_addr;
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        }

        location /ws/ {
                proxy_pass http://127.0.0.1:9090; # WebSocket server
                proxy_http_version 1.1;
                proxy_set_header Upgrade $http_upgrade;
                proxy_set_header Connection "Upgrade";
                proxy_set_header Host $host;
                proxy_set_header X-Real-IP $remote_addr;
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        }
}

I created a subdomain and set the template for the proxy server to the above. Also forced https and used Let’s Encrypt.

I dropped the followed server.php file in the subdomain’s public_html:

<?php
$host = '0.0.0.0'; // Listen on all available IPs
$port = 9090; // Change if needed

// SSL certificate and private key files
$certFile = '/usr/local/hestia/data/users/user/ssl/domain.crt';
$keyFile = '/usr/local/hestia/data/users/user/ssl/domain.key';
$caFile = '/usr/local/hestia/data/users/user/ssl/domain.ca'; 


// Create a secure WebSocket server
$context = stream_context_create([
    'ssl' => [
        'local_cert' => $certFile,
        'private_key' => $keyFile,
        'verify_peer' => true, // Only for development purposes; set to true for production
        'allow_self_signed' => false, // Only for development purposes
        'cafile' => $caFile,
    ]
]);

$server = stream_socket_server("ssl://$host:$port", $errno, $errstr, STREAM_SERVER_BIND | STREAM_SERVER_LISTEN, $context);

if (!$server) {
    die("Error: $errstr ($errno)\n");
}

echo "WebSocket Secure Server started on wss://$host:$port\n";

$clients = [];

while (true) {
    $read = $clients;
    $read[] = $server;

    // Check for new connections or messages
    stream_select($read, $write, $except, 0, 10);

    // Handle new connections
    if (in_array($server, $read)) {
        $client = stream_socket_accept($server);
        if ($client) {
            $clients[] = $client;
            echo "New client connected\n";
        }
        unset($read[array_search($server, $read)]);
    }

    // Handle existing client messages
    foreach ($read as $client) {
        $data = fread($client, 1024);
        if (!$data) {
            fclose($client);
            unset($clients[array_search($client, $clients)]);
            echo "Client disconnected\n";
            continue;
        }

        echo "Received data: $data\n"; // Log incoming data

        // WebSocket handshake
        if (strpos($data, "Upgrade: websocket") !== false) {
            performHandshake($client, $data);
        } else {
            // Decode and respond to WebSocket messages
            $message = decodeWebSocketFrame($data);
            echo "Received: $message\n";
            sendWebSocketMessage($client, "Server received: $message");
        }
    }
}


function performHandshake($client, $request) {
    if (preg_match("/Sec-WebSocket-Key: (.*)\r\n/", $request, $matches)) {
        $key = trim($matches[1]);
        $acceptKey = base64_encode(pack('H*', sha1($key . '258EAFA5-E914-47DA-95CA-C5AB0DC85B11')));

        // Forming a proper WebSocket handshake response
        $response = "HTTP/1.1 101 Switching Protocols\r\n";
        $response .= "Upgrade: websocket\r\n";
        $response .= "Connection: Upgrade\r\n";
        $response .= "Sec-WebSocket-Accept: $acceptKey\r\n";
        $response .= "Sec-WebSocket-Version: 13\r\n"; // WebSocket version 13 is the current standard
        $response .= "\r\n";

        fwrite($client, $response);
        echo "Handshake completed\n";
    } else {
        // If no Sec-WebSocket-Key was found, send a close frame and close the connection
        fwrite($client, chr(0x88) . chr(0x00)); // Sending close frame
        fclose($client);
        echo "Client failed handshake\n";
    }
}

function decodeWebSocketFrame($data) {
    $length = ord($data[1]) & 127;
    if ($length == 126) {
        $masks = substr($data, 4, 4);
        $message = substr($data, 8);
    } elseif ($length == 127) {
        $masks = substr($data, 10, 4);
        $message = substr($data, 14);
    } else {
        $masks = substr($data, 2, 4);
        $message = substr($data, 6);
    }

    $decoded = "";
    for ($i = 0; $i < strlen($message); $i++) {
        $decoded .= $message[$i] ^ $masks[$i % 4];
    }
    return $decoded;
}

function sendWebSocketMessage($client, $message) {
    $frame = chr(129) . chr(strlen($message)) . $message;
    fwrite($client, $frame);
}
?>

I grabbed this online because I just wanted something up and running. More fool me I suppose.

I also created a superviser conf that’s running:

[program:websocket]
command=/usr/bin/php /home/my user/web/domain/public_html/server.php
autostart=true
autorestart=true
user=admin
stderr_logfile=/var/log/websocket.err.log
stdout_logfile=/var/log/websocket.out.log

And then sudo netstat -tuln | grep :9090 shows me the port is being listened to.

However, all attempts at trying to connect to the server fail with a timeout.

Trying wscat -c wss://distribu.davidmurphy.nu:9090/ws/ directly on the server gives me:
error: read ECONNRESET

Which makes me think I’ve got two problems? One is the reverse proxy isn’t forwarding correctly, the other problem is maybe my server.php code is crap. I think this because:

let ws = new WebSocket("wss://mydomain:9090/ws/");
ws.onopen = () => console.log("Connected!");
ws.onmessage = (msg) => console.log("Received:", msg.data);
ws.onerror = (err) => console.error("Error:", err);
ws.onclose = () => console.log("Disconnected.");

in a local browser’s console should result in the same error as I see in wscat on the server, but it gets nothing, just times out. If I try running that same wscat from my local machine it times out.

Running wscat -c ws://distribu.davidmurphy.nu:9090/ws/ on the server gives me error: socket hang up which I’m sort of expecting, but running it locally it times out. So you can see why I think I’ve got two problems.

The /var/log/websocket.out.logs has

PHP Warning:  SSL: failed loading CA names from cafile in /home/user/web/domain/public_html/server.php on line 41
PHP Warning:  stream_socket_accept(): Failed to enable crypto in /home/user/web/domain/public_html/server.php on line 41
PHP Warning:  stream_socket_accept(): Accept failed: Success in /home/user/web/domain/public_html/server.php on line 41

the .out.log has nothing

Any help on this would be greatly appreciated.

I’ve been banging my head against the wall.

I’ve tried adding this to my server.php:

echo file_get_contents($certFile);

which gives me this error in my supervisor log:

PHP Warning:  file_get_contents(/usr/local/hestia/data/users/my user/ssl/mydomain.pem): Failed to open stream: Permission denied in /home/my user/web/mydomain/public_html/server.php on line 13

So I’m guessing my server can’t access these files. I’ve no idea what permissions to actually give the files though. I’ve tried setting the corresponding .key, .pem, .ca, .crt files to 777 temporarily, but that didn’t solve it either.

I’ve also opened 9090 in the firewall so now requests locally no longer time out, they just fail immediately. This is telling me the issue is purely a PHP one, but with the SSL handled by HestiaCP, I’m not sure how to provide the cert info to my PHP code.

I resolved my problem. Basically, it’s embarrassing; the supervisor just needs to run the server as root.