Troubleshooting Common ShellFTP Errors and Performance TipsShellFTP is a method of performing file transfers that combines secure shell (SSH) command-line operations with traditional FTP-like workflows. It’s widely used by system administrators and developers who need secure, scriptable, and automatable file movement between hosts. This article covers the most common ShellFTP errors you’ll encounter, how to diagnose them, and practical performance tips to improve reliability and throughput.
Table of contents
- Common ShellFTP error categories
- Connection and authentication failures
- Permission and filesystem issues
- Transfer interruptions and data corruption
- Performance bottlenecks and optimization
- Best practice checklist
- Useful commands and examples
Common ShellFTP error categories
Most problems fall into a few broad categories:
- Network and connection issues (timeouts, refusals)
- Authentication failures (bad keys, wrong passwords, agent problems)
- Permission and filesystem constraints (ownership, quotas, disk full)
- Transfer interruptions (partial files, broken pipes)
- Performance limitations (latency, bandwidth, encryption overhead)
- Client/Server configuration mismatches (SFTP vs SCP vs rsync over SSH)
Connection and authentication failures
Symptoms: “Connection refused”, “Timed out”, “Permission denied”, “Authentication failed”.
Causes and fixes:
- Host unreachable or wrong hostname/IP:
- Verify host via ping and traceroute.
- Confirm DNS resolution: dig or nslookup.
- SSH daemon not running or wrong port:
- Check SSH service on remote: sudo systemctl status sshd (or service ssh status).
- Ensure you’re connecting to the correct port (default 22). Use ssh -p PORT user@host.
- Firewall blocking traffic:
- Inspect iptables/nftables rules on server and client.
- Check cloud provider security groups (AWS/Azure/GCP).
- Wrong credentials or expired keys:
- Test SSH login manually: ssh -v user@host to see auth steps.
- For key auth, ensure public key is in ~/.ssh/authorized_keys on the server and permissions are strict (700 for ~/.ssh, 600 for authorized_keys).
- SSH agent issues:
- Run ssh-add -l to list loaded keys. Use ssh-add /path/to/key if missing.
- If using passphrase-protected keys in automated scripts, consider using ssh-agent or keyless mechanisms with careful risk assessment.
- Server-side restrictions:
- Check /etc/ssh/sshd_config for PermitRootLogin, AllowUsers, AllowGroups, and Match blocks that could restrict logins.
- Review /var/log/auth.log or /var/log/secure for clues.
Permission and filesystem issues
Symptoms: “Permission denied”, “Read-only file system”, “No space left on device”.
Causes and fixes:
- File or directory permissions:
- Use ls -la to inspect ownership and mode bits.
- Use chown and chmod carefully. Example: sudo chown deploy:deploy /var/www/uploads && sudo chmod 750 /var/www/uploads.
- Disk full or quotas:
- Check disk usage: df -h and df -i (inodes).
- Inspect user quotas: repquota -a (if quotas enabled).
- SELinux or AppArmor blocking:
- For SELinux, check audit logs or run setenforce 0 temporarily to test; use semanage/restorecon to fix contexts.
- For AppArmor, check aa-status and adjust profiles.
- Filesystem corruption or read-only remount:
- Review dmesg and syslog for I/O errors. Consider fsck in maintenance mode if corruption suspected.
- Destination path missing:
- Ensure destination directories exist; create with mkdir -p.
Transfer interruptions and data corruption
Symptoms: Partial files, “broken pipe”, checksums mismatch.
Causes and fixes:
- Network instability:
- Use rsync with –partial –partial-dir and –inplace to resume interrupted transfers.
- Consider mosh for unreliable SSH sessions (note: mosh doesn’t carry file transfer features itself).
- Keepalives:
- Configure ServerAliveInterval and ServerAliveCountMax (client) or ClientAliveInterval and ClientAliveCountMax (server) to prevent idle-time disconnects.
- Example in ~/.ssh/config:
Host example.com ServerAliveInterval 60 ServerAliveCountMax 3
- MTU or TCP fragmentation issues:
- Test with ping -M do -s SIZE host to find MTU. Adjust TCP MSS or network MTU settings as needed.
- Corruption due to incomplete writes:
- Use checksums (md5sum, sha256sum) before and after transfer.
- Use rsync -c to compare checksums (costly but thorough).
- Concurrent writes:
- Ensure processes aren’t modifying the same file during transfer; implement file locking where necessary.
Performance bottlenecks and optimization
Symptoms: Slow transfers despite available bandwidth.
Causes and fixes:
- Encryption overhead:
- Modern CPUs handle SSH encryption quickly, but on low-power devices use faster ciphers: in ssh_config set Ciphers [email protected],aes128-ctr,aes192-ctr,aes256-ctr and KexAlgorithms accordingly.
- Example:
Host * Ciphers [email protected],aes128-ctr,aes192-ctr,aes256-ctr MACs [email protected],[email protected]
- Single-stream limitations:
- SFTP/SCP typically use one TCP stream. For high latency links, use parallel transfers (GNU parallel, multiple rsync processes, or tools like lftp with pget).
- Use rsync –whole-file for LANs, and tune –bwlimit for fairness.
- TCP window scaling and congestion control:
- Ensure window scaling is enabled (sysctl net.ipv4.tcp_window_scaling=1).
- Try different congestion control algorithms: sysctl net.ipv4.tcp_congestion_control=bbr (if kernel supports it).
- Compression trade-offs:
- SSH supports compression (-C). It helps on low-bandwidth/high-CPU/low-entropy data but hurts when data is already compressed or CPU-limited.
- Disk I/O limits:
- Check iostat or atop to find disk bottlenecks. Use faster disks or RAID for high throughput.
- CPU limits:
- Monitor with top or mpstat. If CPU-bound by encryption, offload with AES-NI capable CPUs, or lower cipher cost as above.
- Parallelization example with rsync and GNU parallel:
ls large_files/* | parallel -j4 rsync -av {} user@host:/dest/path/
Client/server configuration mismatches
Symptoms: Unexpected behavior, failed transfers, protocol errors.
Causes and fixes:
- Protocol confusion:
- Confirm whether you’re using SFTP (subsystem sftp), SCP, or rsync over SSH; clients and servers must support chosen protocol.
- SFTP subsystem disabled:
- Ensure sshd_config has Subsystem sftp /usr/lib/openssh/sftp-server (path may vary).
- Different default umasks or transfer modes:
- Set proper umask in server shell profiles or configure your client to set permissions post-transfer.
- Path and shell restrictions for chrooted or restricted users:
- Chrooted SFTP users need proper directory structure and owned by root; read OpenSSH chroot documentation and test with an unprivileged user.
Best practice checklist
- Use key-based authentication with passphrases and ssh-agent for automation.
- Harden SSH: disable root login, restrict users, and use non-default port when appropriate.
- Monitor logs (/var/log/auth.log, /var/log/secure) and set up alerting for repeated failures.
- Use rsync for large or incremental transfers and enable resume support.
- Implement bandwidth controls and parallelism for faster transfers on high-latency links.
- Verify transfers with checksums or rsync’s –checksum flag when integrity is critical.
- Keep server and client OpenSSH updated to access performance and security improvements.
Useful commands and examples
- Test connection and debug authentication:
ssh -vvv user@host - Copy a file with scp (single stream):
scp -P 2222 /local/path/file user@host:/remote/path/ - Use rsync over SSH with compression and partials:
rsync -avz --partial --progress -e "ssh -p 2222" /local/dir/ user@host:/remote/dir/ - Resume an interrupted rsync:
rsync -av --partial --progress /local/dir/ user@host:/remote/dir/ - Check disk space and inodes:
df -h df -i - Measure network throughput:
iperf3 -s # on server iperf3 -c server.example.com # on client
Closing notes
Troubleshooting ShellFTP workflows is largely about isolating layers: network, authentication, filesystem, and application/protocol. Use verbose/debug modes (ssh -v, rsync -vv) and checksums to validate integrity. For performance, balance encryption choices, parallelism, and TCP tuning based on your environment. With the above diagnostics and tuning steps you should be able to resolve most common errors and achieve reliable, efficient transfers.
Leave a Reply