SSH Stopped Working

sujit · August 11, 2020, 3:09am

Hey everyone,

Woke up this morning with an error message saying SSH is not running on one of my servers (ubuntu 18.04). I checked the syslog and found:

Aug 11 14:58:20 cl sshd[18191]: /etc/ssh/sshd_config: line 127: Bad configuration option: GatewayPorts|no,admin,testuser
Aug 11 14:58:20 cl sshd[18191]: /etc/ssh/sshd_config: terminating, 1 bad configuration options
Aug 11 14:58:20 cl systemd[1]: ssh.service: Main process exited, code=exited, status=255/n/a
Aug 11 14:58:20 cl systemd[1]: ssh.service: Failed with result 'exit-code'.

I’ve commented out line GatewayPorts|no,admin,testuser in sshd config file and it started working.

My question is… will the commented out line affect hestiacp in any adverse way and if there’s any other way to not let that config affect ssh?

Thanks,

eris · August 11, 2020, 6:33am

Could you post the contents of /etc/ssh/sshd_config?

Also check /root/hst_backups/{folder-created-yesterday/today}/conf/ssh/sshd_config

It should contain the original file

sujit · August 11, 2020, 8:36pm

Hi @eris

Here you go. Contents of /etc/ssh/sshd_config

#	$OpenBSD: sshd_config,v 1.101 2017/03/14 07:19:07 djm Exp $
# This is the sshd server system-wide configuration file.  See
# sshd_config(5) for more information.

# This sshd was compiled with PATH=/usr/bin:/bin:/usr/sbin:/sbin

# The strategy used for options in the default sshd_config shipped with
# OpenSSH is to specify options with their default value where
# possible, but leave them commented.  Uncommented options override the
# default value.

#Port 22
#AddressFamily any
#ListenAddress 0.0.0.0
#ListenAddress ::

#HostKey /etc/ssh/ssh_host_rsa_key
#HostKey /etc/ssh/ssh_host_ecdsa_key
#HostKey /etc/ssh/ssh_host_ed25519_key

# Ciphers and keying
#RekeyLimit default none

# Logging
#SyslogFacility AUTH
#LogLevel INFO

# Authentication:

#LoginGraceTime 1m
#PermitRootLogin prohibit-password
#StrictModes yes
#MaxAuthTries 6
#MaxSessions 10

#PubkeyAuthentication yes

# Expect .ssh/authorized_keys2 to be disregarded by default in future.
#AuthorizedKeysFile	.ssh/authorized_keys .ssh/authorized_keys2

#AuthorizedPrincipalsFile none

#AuthorizedKeysCommand none
#AuthorizedKeysCommandUser nobody

# For this to work you will also need host keys in /etc/ssh/ssh_known_hosts
#HostbasedAuthentication no
# Change to yes if you don't trust ~/.ssh/known_hosts for
# HostbasedAuthentication
#IgnoreUserKnownHosts no
# Don't read the user's ~/.rhosts and ~/.shosts files
#IgnoreRhosts yes

# To disable tunneled clear text passwords, change to no here!
PasswordAuthentication yes
#PermitEmptyPasswords no

# Change to yes to enable challenge-response passwords (beware issues with
# some PAM modules and threads)
ChallengeResponseAuthentication no

# Kerberos options
#KerberosAuthentication no
#KerberosOrLocalPasswd yes
#KerberosTicketCleanup yes
#KerberosGetAFSToken no

# GSSAPI options
#GSSAPIAuthentication no
#GSSAPICleanupCredentials yes
#GSSAPIStrictAcceptorCheck yes
#GSSAPIKeyExchange no

# Set this to 'yes' to enable PAM authentication, account processing,
# and session processing. If this is enabled, PAM authentication will
# be allowed through the ChallengeResponseAuthentication and
# PasswordAuthentication.  Depending on your PAM configuration,
# PAM authentication via ChallengeResponseAuthentication may bypass
# the setting of "PermitRootLogin without-password".
# If you just want the PAM account and session checks to run without
# PAM authentication, then enable this but set PasswordAuthentication
# and ChallengeResponseAuthentication to 'no'.
UsePAM yes

#AllowAgentForwarding yes
#AllowTcpForwarding yes
#GatewayPorts|no,admin,core
X11Forwarding yes
#X11DisplayOffset 10
#X11UseLocalhost yes
#PermitTTY yes
PrintMotd no
#PrintLastLog yes
#TCPKeepAlive yes
#UseLogin no
#PermitUserEnvironment no
#Compression delayed
#ClientAliveInterval 0
#ClientAliveCountMax 3
#UseDNS no
#PidFile /var/run/sshd.pid
#MaxStartups 10:30:100
#PermitTunnel no
#ChrootDirectory none
#VersionAddendum none

# no default banner path
#Banner none

# Allow client to pass locale environment variables
AcceptEnv LANG LC_*

# override default of no subsystems
Subsystem sftp internal-sftp

# Example of overriding settings on a per-user basis
#Match User anoncvs
#	X11Forwarding no
#	AllowTcpForwarding no
#	PermitTTY no
#	ForceCommand cvs server

DebianBanner no
 
# Hestia SFTP Chroot
GatewayPorts|no,admin,testuser
AllowTcpForwarding yes
KeepAlive yes
Protocol 1,2
Port 222
Match User sftp_dummy99,admin,testuser
ChrootDirectory %h
    X11Forwarding no
    AllowTCPForwarding no
    ForceCommand internal-sftp

And the contents from hst_backups folder

#	$OpenBSD: sshd_config,v 1.101 2017/03/14 07:19:07 djm Exp $

# This is the sshd server system-wide configuration file.  See
# sshd_config(5) for more information.

# This sshd was compiled with PATH=/usr/bin:/bin:/usr/sbin:/sbin

# The strategy used for options in the default sshd_config shipped with
# OpenSSH is to specify options with their default value where
# possible, but leave them commented.  Uncommented options override the
# default value.

#Port 22
#AddressFamily any
#ListenAddress 0.0.0.0
#ListenAddress ::

#HostKey /etc/ssh/ssh_host_rsa_key
#HostKey /etc/ssh/ssh_host_ecdsa_key
#HostKey /etc/ssh/ssh_host_ed25519_key

# Ciphers and keying
#RekeyLimit default none

# Logging
#SyslogFacility AUTH
#LogLevel INFO

# Authentication:

#LoginGraceTime 1m
#PermitRootLogin prohibit-password
#StrictModes yes
#MaxAuthTries 6
#MaxSessions 10

#PubkeyAuthentication yes

# Expect .ssh/authorized_keys2 to be disregarded by default in future.
#AuthorizedKeysFile	.ssh/authorized_keys .ssh/authorized_keys2

#AuthorizedPrincipalsFile none

#AuthorizedKeysCommand none
#AuthorizedKeysCommandUser nobody

# For this to work you will also need host keys in /etc/ssh/ssh_known_hosts
#HostbasedAuthentication no
# Change to yes if you don't trust ~/.ssh/known_hosts for
# HostbasedAuthentication
#IgnoreUserKnownHosts no
# Don't read the user's ~/.rhosts and ~/.shosts files
#IgnoreRhosts yes

# To disable tunneled clear text passwords, change to no here!
PasswordAuthentication yes
#PermitEmptyPasswords no

# Change to yes to enable challenge-response passwords (beware issues with
# some PAM modules and threads)
ChallengeResponseAuthentication no

# Kerberos options
#KerberosAuthentication no
#KerberosOrLocalPasswd yes
#KerberosTicketCleanup yes
#KerberosGetAFSToken no

# GSSAPI options
#GSSAPIAuthentication no
#GSSAPICleanupCredentials yes
#GSSAPIStrictAcceptorCheck yes
#GSSAPIKeyExchange no

# Set this to 'yes' to enable PAM authentication, account processing,
# and session processing. If this is enabled, PAM authentication will
# be allowed through the ChallengeResponseAuthentication and
# PasswordAuthentication.  Depending on your PAM configuration,
# PAM authentication via ChallengeResponseAuthentication may bypass
# the setting of "PermitRootLogin without-password".
# If you just want the PAM account and session checks to run without
# PAM authentication, then enable this but set PasswordAuthentication
# and ChallengeResponseAuthentication to 'no'.
UsePAM yes

#AllowAgentForwarding yes
#AllowTcpForwarding yes
#GatewayPorts no
X11Forwarding yes
#X11DisplayOffset 10
#X11UseLocalhost yes
#PermitTTY yes
PrintMotd no
#PrintLastLog yes
#TCPKeepAlive yes
#UseLogin no
#PermitUserEnvironment no
#Compression delayed
#ClientAliveInterval 0
#ClientAliveCountMax 3
#UseDNS no
#PidFile /var/run/sshd.pid
#MaxStartups 10:30:100
#PermitTunnel no
#ChrootDirectory none
#VersionAddendum none

# no default banner path
#Banner none

# Allow client to pass locale environment variables
AcceptEnv LANG LC_*

# override default of no subsystems
Subsystem sftp internal-sftp

# Example of overriding settings on a per-user basis
#Match User anoncvs
#	X11Forwarding no
#	AllowTcpForwarding no
#	PermitTTY no
#	ForceCommand cvs server

DebianBanner no
 
# Hestia SFTP Chroot
GatewayPorts no
AllowTcpForwarding yes
KeepAlive yes
Protocol 1,2
Port 222
Match User sftp_dummy99,admin,testuser
ChrootDirectory %h
    X11Forwarding no
    AllowTCPForwarding no
    ForceCommand internal-sftp

I must add, the only changes I made to stock config was changing the SSH port and creating a SWAP memory. Apart from that, everything should be on default settings.

Felix · August 13, 2020, 7:54am

Since you changed the port, you need to make sure the new port is allowed through Firewall in Hestia UI.

sujit · August 13, 2020, 8:12am

Hi @Felix

The new port is in fact allowed and added through Firewall in Hestia UI. SSH was working fine, until one day it decided to stop working. Commenting out the “gateway” line in sshd_config file made it work.

However, am not sure if that would create some unwanted side effects on ssh or hestia.

eris · August 13, 2020, 8:13am

Don’t know why any sed replaced the no part with |no,users

github.com

hestiacp/hestiacp/blob/2e05ab4f56d66f94e26cec430c6adccecd35e780/bin/v-add-user-sftp-jail#L57-L58


      
          users=$(IFS=',';echo "${users[*]// /|}";IFS=$' \t\n')
          sed -i "s/$ssh_users/$users/g" /etc/ssh/sshd_config

Shouldn’t be changed. We should monitor this issue and see if it happens again.

kpv · August 16, 2020, 9:36pm

As I’ve pointed out several times, both here and on the issue tracker, inline editing of sshd_config with sed is a very bad idea.

But it’s a remnant of VestaCP which had “no security mindset in place” to quote the security analysis report of Rack911 Labs of “alternative” (to cPanel and Plesk) CPs.

eris · August 16, 2020, 9:41pm

Seems to be not a true fix if we want to allow sftp jail aditinonal accounts see https://github.com/hestiacp/hestiacp/pull/944#issuecomment-664048788

sujit · August 17, 2020, 11:30pm

For the time being I’ve changed the following in sshd_config.

GatewayPorts|no,admin,testuser
to
GatewayPorts no

I am the only one using this server, so if something goes wrong with the SSH I know what to revert back.