Skip to main content

Command documentation sourced from the linux-command project This comprehensive command reference is part of the linux-command documentation project.

syslog - System logging utilities

The syslog utility is a comprehensive system logging framework that provides centralized log management for Unix-like operating systems. It consists of the syslog daemon (syslogd), client utilities, and configuration files that work together to collect, process, store, and forward system messages from various applications, kernel events, and network services. The syslog protocol follows the RFC 5424 standard and uses facility/severity levels to categorize and prioritize log messages, making it an essential tool for system monitoring, troubleshooting, security auditing, and compliance management.

Basic Syntax​

# Syslog daemon
syslogd [OPTIONS]

# Logger command (client utility)
logger [OPTIONS] [MESSAGE]

# Traditional syslog (legacy)
syslog [OPTIONS]

Syslog Facilities and Severities​

Facilities (Message Sources)​

  • auth, authpriv - Security and authorization messages
  • cron - Cron daemon messages
  • daemon - System daemon messages
  • kern - Kernel messages
  • lpr - Line printer subsystem
  • mail - Mail subsystem
  • mark - Timestamp for log rotation
  • news - Network news subsystem
  • syslog - Internal syslog messages
  • user - User-level messages (default)
  • uucp - UUCP subsystem
  • local0 through local7 - Local use facilities

Severities (Priority Levels)​

  • emerg (0) - Emergency: system is unusable
  • alert (1) - Alert: action must be taken immediately
  • crit (2) - Critical: critical conditions
  • err (3) - Error: error conditions
  • warn (4) - Warning: warning conditions
  • notice (5) - Notice: normal but significant condition
  • info (6) - Informational: informational messages
  • debug (7) - Debug: debug-level messages

Syslogd Daemon Options​

Basic Operation​

  • -f CONFIG_FILE - Use alternative configuration file
  • -n - Don't fork, run in foreground
  • -p SOCKET_PATH - Specify alternative socket path
  • -m MARK_INTERVAL - Set mark timestamp interval (default 20 minutes)
  • -r - Accept remote messages via UDP
  • -x - Disable name resolution for remote hosts

Security Options​

  • -h - Allow forwarded remote messages
  • -l HOSTLIST - Comma-separated list of local hostnames
  • -s DOMAINLIST - Comma-separated list of domains to strip
  • -t HOSTNAME - Override hostname for messages

Modern Implementation Options​

  • -R HOST:PORT - Forward messages to remote syslog server
  • -T HOST:PORT - Forward TCP messages to remote server
  • -u - Use UNIX domain sockets only
  • -v - Increase verbosity

Logger Command Options​

Message Formatting​

  • -i - Process ID with each message
  • -t TAG - Tag every line with specified tag
  • -p PRIORITY - Specify priority (facility.severity)
  • -s - Log to stderr as well as system log

Message Source​

  • -u SOCKET - Write to socket instead of system log
  • -f FILE - Log contents of specified file
  • -n - Write to remote syslog server (skip local logging)
  • -P PORT - Use specified UDP port

Message Format​

  • -- - End options, treat remaining arguments as message
  • --journald - Send to systemd journal instead of syslog

Configuration File (/etc/syslog.conf or /etc/rsyslog.conf)​

Basic Format​

facility.severity    action

Example Configuration Rules​

# Log all kernel messages to console
kern.*                    /dev/console

# Log authentication messages to auth.log
auth,authpriv.*           /var/log/auth.log

# Log mail messages to mail.log
mail.*                    /var/log/mail.log

# Log all messages with priority info or higher
*.info;mail.none;authpriv.none;cron.none   /var/log/messages

# Log emergency messages to all users
*.emerg                   :omusrmsg:*

# Forward all messages to remote server
*.*                       @remote.example.com:514

Usage Examples​

Basic Logging Operations​

Using Logger Command​

# Log simple message with default priority
logger "System backup completed successfully"

# Log message with specific priority
logger -p auth.warning "Failed login attempt from 192.168.1.100"

# Log message with custom tag
logger -t myscript "Starting data processing"

# Log with process ID
logger -i -p daemon.info "Daemon starting with PID $$"

# Log message to remote server
logger -n logserver.example.com -p local0.info "Remote log message"

# Log file contents
logger -f /var/log/nginx/access.log

# Log with custom facility and severity
logger -p local4.err "Application error occurred"

# Log multiple lines from stdin
echo "Error line 1\nError line 2" | logger -t batchjob

# Log with timestamp
logger "Event occurred at $(date)"

# Log structured message
logger -p user.info "{ \"event\": \"user_login\", \"user\": \"john\", \"ip\": \"10.0.0.5\" }"

Advanced Logger Usage​

# Log with severity levels for testing
for level in emerg alert crit err warn notice info debug; do
    logger -p user.$level "Testing $level priority message"
done

# Log with different facilities
logger -p kern.info "Kernel driver loaded"
logger -p mail.info "Email sent successfully"
logger -p cron.info "Backup job completed"

# Log with custom tag and priority
logger -t webserver -p daemon.info "Server started on port 8080"

# Log environment information
logger -p system.info "System uptime: $(uptime)"
logger -p system.info "Memory usage: $(free -h)"

# Log script execution
logger -t script_name -p user.info "Script started at $(date)"
[commands...]
logger -t script_name -p user.info "Script completed successfully"

# Log error with exit code
if ! command; then
    logger -t script_name -p user.err "Command failed with exit code $?"
fi

Syslog Daemon Configuration​

Basic Rsyslog Configuration​

# /etc/rsyslog.conf
# Load modules
module(load="imuxsock")   # Unix socket support
module(load="imklog")     # Kernel logging
module(load="immark")     # Message marking

# Global directives
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
$RepeatedMsgReduction on
$FileOwner syslog
$FileGroup adm
$FileCreateMode 0640
$DirCreateMode 0755
$Umask 0022
$PrivDropToUser syslog
$PrivDropToGroup syslog

# Log all kernel messages to console
kern.*                                                 /dev/console

# Log anything (except mail) of level info or higher
*.info;mail.none;authpriv.none;cron.none               /var/log/messages

# The authpriv file has restricted access
authpriv.*                                            /var/log/secure

# Log all the mail messages in one place
mail.*                                                /var/log/maillog

# Log cron stuff
cron.*                                                /var/log/cron

# Everybody gets emergency messages
*.emerg                                               :omusrmsg:*

# Save news errors of level crit and higher in a special file
uucp,news.crit                                        /var/log/spooler

# Save boot messages also to boot.log
local7.*                                              /var/log/boot.log

Advanced Rsyslog Configuration​

# /etc/rsyslog.conf
# Load additional modules
module(load="imudp")          # UDP syslog receiving
module(load="imtcp")          # TCP syslog receiving
module(load="imrelp")         # RELP syslog receiving
module(load="omfile")         # File output module
module(load="omfwd")          # Forwarding module

# Enable UDP and TCP listeners
$ModLoad imudp
$UDPServerRun 514
$ModLoad imtcp
$InputTCPServerRun 514

# Template for custom log format
$template MyFormat,"%timestamp% %hostname% %syslogtag% %msg%\n"
$template DynFile,"/var/log/system-%$YEAR%-%$MONTH%-%$DAY%.log"

# Create separate logs by severity
*.warning                    /var/log/warnings
*.error                      /var/log/errors
*.crit                       /var/log/critical
*.alert                      /var/log/alerts

# Separate logs by program
httpd.*                      /var/log/httpd.log
sshd.*                       /var/log/sshd.log
mysql.*                      /var/log/mysql.log

# Log to file with custom format
*.*                          /var/log/allmessages;MyFormat

# Dynamic file creation based on date
*.*                          ?DynFile

# Forward to remote servers
*.*                          @@logserver.example.com:10514
auth.*                       @backup.example.com:514

# Conditional logging
:msg, contains, "error"     /var/log/error.log
:msg, startswith, "SUCCESS" /var/log/success.log
:msg, regex, "^[0-9]{4}-"   /var/log/structured.log

Log File Management​

Log Rotation with logrotate​

# /etc/logrotate.d/syslog
/var/log/syslog
{
    rotate 7
    daily
    missingok
    notifempty
    delaycompress
    compress
    postrotate
        systemctl reload rsyslog > /dev/null 2>&1 || true
    endscript
}

/var/log/auth.log
/var/log/mail.log
/var/log/cron.log
/var/log/messages
{
    rotate 4
    weekly
    missingok
    notifempty
    compress
    delaycompress
    sharedscripts
    postrotate
        systemctl reload rsyslog > /dev/null 2>&1 || true
    endscript
}

Manual Log Management​

# View system logs in real-time
tail -f /var/log/syslog

# Search for specific patterns
grep "error" /var/log/syslog
grep "authentication failure" /var/log/auth.log

# Filter logs by date
grep "$(date +%b\ %d)" /var/log/syslog

# Count log entries by severity
grep -c "crit\|alert\|emerg" /var/log/syslog
grep -c "error\|warning" /var/log/syslog

# Find IP addresses in auth log
grep -oE "\b([0-9]{1,3}\.){3}[0-9]{1,3}\b" /var/log/auth.log | sort -u

# Monitor specific service logs
tail -f /var/log/nginx/error.log
tail -f /var/log/mysql/error.log

# Archive old logs
tar -czf syslog-$(date +%Y%m%d).tar.gz /var/log/syslog.*

Network Logging​

Remote Logging Configuration​

# Client configuration - Forward to remote server
# /etc/rsyslog.conf
*.* @@logserver.example.com:514
auth.* @logserver.example.com:514
mail.* @@logserver.backup.com:514

# Server configuration - Receive from remote clients
# /etc/rsyslog.conf
module(load="imtcp")
$InputTCPServerRun 514

module(load="imudp")
$UDPServerRun 514

# Create separate logs for remote hosts
$template RemoteHostFile,"/var/log/remote/%HOSTNAME%.log"
*.* ?RemoteHostFile

# Separate logs by facility
$template RemoteAuthFile,"/var/log/remote/%HOSTNAME%-auth.log"
auth,authpriv.* ?RemoteAuthFile

$template RemoteKernelFile,"/var/log/remote/%HOSTNAME%-kernel.log"
kern.* ?RemoteKernelFile

TLS Encrypted Logging​

# /etc/rsyslog.conf - TLS configuration
module(load="gtls")
module(load="omgtls")
module(load="imtcp")

# Server certificate configuration
$DefaultNetstreamDriver gtls
$DefaultNetstreamDriverCAFile /path/to/ca.pem
$DefaultNetstreamDriverCertFile /path/to/server-cert.pem
$DefaultNetstreamDriverKeyFile /path/to/server-key.pem

# Input TLS listener
$InputTCPServerStreamDriverMode 1
$InputTCPServerStreamDriverAuthMode anon
$InputTCPServerRun 10514

# Output to remote TLS server
$ActionSendStreamDriverMode 1
$ActionSendStreamDriverAuthMode x509/name
$ActionSendStreamDriverPermittedPeer logserver.example.com
*.* @@logserver.example.com:10514

Practical Examples​

System Administration​

Security Monitoring​

# Monitor authentication failures
logger -p auth.info "Security scan initiated at $(date)"
tail -f /var/log/auth.log | grep --line-buffered "authentication failure"

# Alert on suspicious activity
#!/bin/bash
# Security monitoring script
ALERT_THRESHOLD=10
FAILED_LOGINS=$(grep "$(date +%b\ %d)" /var/log/auth.log | grep -c "authentication failure")

if [ $FAILED_LOGINS -gt $ALERT_THRESHOLD ]; then
    logger -p auth.alert "High number of failed logins detected: $FAILED_LOGINS"
    mail -s "Security Alert" admin@example.com <<< "Multiple failed login attempts detected"
fi

# Monitor root usage
if who | grep root; then
    logger -p auth.warning "Root login detected on $(hostname)"
fi

# Monitor sudo usage
grep "sudo:" /var/log/auth.log | tail -10

System Health Monitoring​

# Log system statistics
logger -p system.info "CPU usage: $(top -bn1 | grep "Cpu(s)" | awk '{print $2}')"
logger -p system.info "Memory usage: $(free | grep Mem | awk '{printf "%.2f%%", $3/$2 * 100.0}')"
logger -p system.info "Disk usage: $(df -h / | awk 'NR==2 {print $5}')"

# Log service status
for service in nginx mysql sshd; do
    if systemctl is-active --quiet $service; then
        logger -p daemon.info "$service is running"
    else
        logger -p daemon.err "$service is not running"
    fi
done

# Log network status
logger -p system.info "Network interface status: $(ip addr show | grep -E "state (UP|DOWN)" | wc -l) interfaces active"

Application Logging​

Application Integration​

# Shell script logging function
log_message() {
    local level=$1
    local message=$2
    local tag=${3:-$(basename $0)}

    case $level in
        "ERROR") logger -p user.err -t $tag "$message" ;;
        "WARN")  logger -p user.warning -t $tag "$message" ;;
        "INFO")  logger -p user.info -t $tag "$message" ;;
        "DEBUG") logger -p user.debug -t $tag "$message" ;;
    esac
}

# Usage in script
log_message "INFO" "Starting backup process"
if tar -czf backup.tar.gz /data; then
    log_message "INFO" "Backup completed successfully"
else
    log_message "ERROR" "Backup failed with exit code $?"
fi

# Web application logging
logger -p web.info "HTTP GET request from ${REMOTE_ADDR} for ${REQUEST_URI}"
logger -p web.err "Database connection failed for user ${DB_USER}"

Database Logging​

# MySQL operations logging
logger -p mysql.info "Starting database backup"
mysqldump --all-databases | gzip > backup.sql.gz
logger -p mysql.info "Database backup completed: $(du -h backup.sql.gz | cut -f1)"

# PostgreSQL logging
logger -p postgres.info "Starting vacuum operation"
vacuumdb --all --analyze
logger -p postgres.info "Vacuum completed successfully"

# Log database status
mysql -e "SHOW STATUS LIKE 'Connections'" | logger -p mysql.info

Troubleshooting and Debugging​

Debug Logging​

# Enable debug logging for specific service
logger -p daemon.debug "Enabling debug mode for application"
export DEBUG=1
./application

# Network debugging
logger -p kern.debug "Starting network trace"
tcpdump -i eth0 -w /tmp/network.pcap &
TCPDUMP_PID=$!
sleep 60
kill $TCPDUMP_PID
logger -p kern.debug "Network trace completed"

# System call tracing
logger -p debug.info "Starting strace on process $PID"
strace -p $PID -o /tmp/strace.log &
STRACE_PID=$!
sleep 30
kill $STRACE_PID
logger -p debug.info "Strace completed, output saved to /tmp/strace.log"

Error Analysis​

# Analyze log patterns
grep -i "error\|warning\|critical" /var/log/syslog | \
    awk '{print $1" "$2" "$3}' | sort | uniq -c | sort -nr

# Find top error sources
grep -i "error" /var/log/syslog | \
    awk '{print $5}' | sort | uniq -c | sort -nr | head -10

# Time-based error analysis
grep "$(date +%b\ %d)" /var/log/syslog | \
    grep -i "error" | awk '{print $3}' | sort | uniq -c

# Log correlation
#!/bin/bash
# Correlate errors across different log files
ERROR_PATTERNS=("segmentation fault" "connection refused" "out of memory")

for pattern in "${ERROR_PATTERNS[@]}"; do
    echo "Searching for: $pattern"
    grep -r "$pattern" /var/log/ --include="*.log" | head -5
    echo "---"
done

Advanced Configuration​

Performance Optimization​

Buffer and Queue Configuration​

# /etc/rsyslog.conf - Performance tuning
$MainMsgQueueSize 100000
$MainMsgQueueHighWatermark 80000
$MainMsgQueueLowWatermark 20000
$WorkDirectory /var/spool/rsyslog

# Async disk write
$ActionQueueType Direct
$ActionQueueFileName fwdRule1
$ActionQueueMaxDiskSpace 1g
$ActionResumeRetryCount -1
$ActionQueueSaveOnShutdown on

# Parallel processing
$ControlFlowSleepInterval 100
$RateLimit.Interval 0
$RateLimit.Burst 0

Memory and Disk Usage​

# Monitor rsyslog memory usage
ps aux | grep rsyslog

# Check disk space usage for logs
du -sh /var/log/*

# Compress old logs automatically
find /var/log -name "*.log.*" -mtime +7 -exec gzip {} \;

# Clean up old logs
find /var/log -name "*.log.*" -mtime +30 -delete

Filter and Action Configuration​

Advanced Filtering​

# Property-based filters
:msg, contains, "database" /var/log/database.log
:fromhost-ip, isequal, "192.168.1.100" /var/log/client1.log
:syslogtag, startswith, "httpd" /var/log/httpd.log

# Complex conditions
if $msg contains 'error' and $syslogseverity <= 4 then {
    /var/log/critical.log
    :omusrmsg:*
}

# RainerScript examples
if ($syslogtag contains 'sudo') and ($msg contains 'incorrect password') then {
    /var/log/sudo-failures
}

if ($fromhost-ip startswith '10.0.') then {
    /var/log/internal-network.log
}

Custom Actions​

# Execute program on specific log entry
:msg, regex, "CRITICAL" ^/usr/local/bin/alert-script.sh

# Send email on critical errors
:syslogseverity-text, isequal, "critical" :ommail:;mailtemplate
$template mailtemplate,"Subject: Critical Error on %hostname%\n\n%msg%\n"

# Database logging
module(load="ommysql")
*.* :ommysql:localhost,Syslog,logger,password

# JSON structured logging
template(name="jsonfmt" type="list") {
    constant(value="{")
    constant(value="\"timestamp\":\"")     property(name="timereported" dateFormat="rfc3339")
    constant(value="\",\"hostname\":\"")   property(name="hostname")
    constant(value="\",\"program\":\"")    property(name="programname")
    constant(value="\",\"priority\":\"")   property(name="syslogpriority-text")
    constant(value="\",\"message\":\"")    property(name="msg" format="json")
    constant(value="\"}\n")
}
*.* action(type="omfile" file="/var/log/structured.log" template="jsonfmt")

Integration and Automation​

Automation Scripts​

Log Analysis Script​

#!/bin/bash
# Comprehensive log analysis script

LOG_DIR="/var/log"
REPORT_FILE="/tmp/log-analysis-$(date +%Y%m%d).txt"
ERROR_THRESHOLD=100

echo "Log Analysis Report - $(date)" > $REPORT_FILE
echo "================================" >> $REPORT_FILE

# System errors analysis
echo -e "\n[SYSTEM ERRORS]" >> $REPORT_FILE
echo "Error count by source:" >> $REPORT_FILE
grep -i "error\|fail\|critical" $LOG_DIR/syslog | \
    awk '{print $5}' | sort | uniq -c | sort -nr >> $REPORT_FILE

# Authentication analysis
echo -e "\n[AUTHENTICATION SUMMARY]" >> $REPORT_FILE
echo "Failed login attempts:" >> $REPORT_FILE
grep "$(date +%b\ %d)" $LOG_DIR/auth.log | \
    grep -c "authentication failure" >> $REPORT_FILE

echo "Successful logins:" >> $REPORT_FILE
grep "$(date +%b\ %d)" $LOG_DIR/auth.log | \
    grep -c "session opened" >> $REPORT_FILE

# Service status
echo -e "\n[SERVICE STATUS]" >> $REPORT_FILE
for service in sshd nginx mysql httpd; do
    status=$(systemctl is-active $service)
    echo "$service: $status" >> $REPORT_FILE
done

# Disk space
echo -e "\n[DISK USAGE]" >> $REPORT_FILE
df -h | grep -vE '^tmpfs|udev' >> $REPORT_FILE

# Alert on critical issues
ERROR_COUNT=$(grep -i "error\|critical" $LOG_DIR/syslog | wc -l)
if [ $ERROR_COUNT -gt $ERROR_THRESHOLD ]; then
    logger -p system.alert "High error count detected: $ERROR_COUNT errors found"
fi

echo "Report generated: $REPORT_FILE"
logger -p system.info "Log analysis completed"

Log Backup Script​

#!/bin/bash
# Automated log backup and archiving

BACKUP_DIR="/backup/logs"
RETENTION_DAYS=30
ARCHIVE_DATE=$(date +%Y%m%d)

mkdir -p $BACKUP_DIR/$ARCHIVE_DATE

# Compress and archive logs
for logfile in /var/log/*.log; do
    if [ -f "$logfile" ]; then
        filename=$(basename "$logfile")
        cp "$logfile" $BACKUP_DIR/$ARCHIVE_DATE/
    fi
done

# Compress the backup directory
tar -czf $BACKUP_DIR/logs-$ARCHIVE_DATE.tar.gz -C $BACKUP_DIR $ARCHIVE_DATE

# Remove uncompressed backup
rm -rf $BACKUP_DIR/$ARCHIVE_DATE

# Clean old backups
find $BACKUP_DIR -name "logs-*.tar.gz" -mtime +$RETENTION_DAYS -delete

# Log the backup operation
BACKUP_SIZE=$(du -h $BACKUP_DIR/logs-$ARCHIVE_DATE.tar.gz | cut -f1)
logger -p backup.info "Log backup completed: $BACKUP_SIZE archived"

# Verify backup integrity
if tar -tzf $BACKUP_DIR/logs-$ARCHIVE_DATE.tar.gz > /dev/null 2>&1; then
    logger -p backup.info "Backup verification successful"
else
    logger -p backup.err "Backup verification failed!"
fi

Troubleshooting​

Common Issues​

Syslog Daemon Not Starting​

# Check rsyslog status
systemctl status rsyslog
journalctl -u rsyslog -f

# Check configuration syntax
rsyslogd -N1 -f /etc/rsyslog.conf

# Check for locked files
lsof /var/log/syslog

# Common problems and solutions
# Permission issues:
chown syslog:adm /var/log/syslog
chmod 640 /var/log/syslog

# Disk space issues
df -h /var/log
find /var/log -name "*.log.*" -mtime +7 -delete

# Configuration errors
grep -E "error|warning" /var/log/syslog

Log Messages Not Appearing​

# Test logger functionality
logger -p user.info "Test message"
tail -f /var/log/syslog

# Check syslog socket
ls -la /dev/log
nc -U /dev/log

# Check filters and rules
rsyslogd -d -n 2>&1 | grep -i "error"

# Verify configuration reload
systemctl reload rsyslog
kill -HUP $(cat /var/run/rsyslogd.pid)

Performance Issues​

# Monitor rsyslog performance
top -p $(pgrep rsyslog)
iotop -p $(pgrep rsyslog)

# Check queue sizes
grep -i queue /var/log/rsyslog/stats

# Optimize configuration
# Reduce imjournal polling interval
$IMJournalPollInterval 1

# Enable async disk writes
$ActionQueueType Direct
$ActionQueueFileName async

Debug Techniques​

Enable Debug Logging​

# Enable rsyslog debug mode
rsyslogd -dn > /tmp/rsyslog-debug.log 2>&1 &

# Test message flow
logger -p user.debug "Debug test message"
tail -f /tmp/rsyslog-debug.log

# Monitor file descriptors
lsof -p $(pgrep rsyslog)

# Track network connections
netstat -an | grep 514
ss -an | grep 514
  • journalctl - systemd journal query tool
  • logger - Shell command interface to syslog
  • logrotate - Log rotation and management
  • tail - Display end of files
  • grep - Search text patterns
  • awk - Text processing and pattern scanning
  • systemctl - System service manager
  • dmesg - Kernel ring buffer messages
  • tailf - Follow file growth
  • wc - Word, line, and character count

Best Practices​

  1. Use structured logging with JSON format for better parsing and analysis
  2. Implement log rotation to prevent disk space exhaustion
  3. Monitor critical services with dedicated log files and alerts
  4. Secure remote logging using TLS encryption for sensitive environments
  5. Configure proper permissions on log files to maintain security
  6. Use appropriate severity levels to enable effective filtering
  7. Implement centralized logging for distributed systems
  8. Regular log analysis to detect security incidents and performance issues
  9. Backup important logs for compliance and forensic purposes
  10. Test configuration changes in non-production environments first

Performance Tips​

  1. Use async disk writes for high-volume logging environments
  2. Optimize queue sizes based on message volume and system resources
  3. Implement message filtering early to reduce processing overhead
  4. Use dedicated storage for high-traffic log files
  5. Compress old logs to save disk space while maintaining accessibility
  6. Monitor memory usage and tune buffer sizes accordingly
  7. Avoid excessive logging in production environments
  8. Use indexing for large log files to improve search performance
  9. Implement log sampling for high-frequency debug messages
  10. Regular maintenance including cleanup of old logs and rotation management

The syslog system is an essential component of Linux system administration, providing centralized logging capabilities that are crucial for monitoring, troubleshooting, security auditing, and compliance management. Proper configuration and management of syslog facilities enable administrators to maintain visibility into system operations and quickly identify and resolve issues as they arise.

Sponsors:DigitalOceanVultrSetappCloudflare