Hardening your Linux Debian 7 Wheezy – Part 1

I’ve been blogging about things related to me moving my blog not a new server and hosting in Amazon Web Services (AWS). So far I’ve liked the environment a lot, although I’ve had some issues with AWS terminology their documentation is extensive and usually very clear, which helps a lot.

One of my main concerns has been securing my server. I’ve always been a bit paranoid in this regard, but I think this is more of a feature than a bug in my personality so today I making a long post on the basics steps one can take to Harden/Secure your Linux (Debian) server.

NOTE: Most things apply to other Linux distributions but you may have to check the commands to match your distribution.

Server Security 101

Some points to consider

  • Identify your attack surface: This step may seem trivial or obvious but properly identifying the attack surface can help you decide what you need to secure. Make a list of the services, daemons, etc that need network access (SSH, HTTP, SFTP, NFS, etc) In my case I’m running a blogging Web Server and I use
    1. SSH
    2. HTTP(S)
    3. SMTP
  • Research for guides to secure your distribution: This step is crucial since distributions may have different ways of working and having a guide tailored for this can help. For Debian you can find the guides in here
  • Before rolling changes make a backup: Making changes to your current set up will need testing before using in production, be sure to have backups ready.

Lets get this done!

Partitioning Scheme

An intelligent partition scheme depends on how the machine is used. (From Debian Guide)

  • Any directory tree which a user has write permissions to, such as e.g. /home, /tmp and /var/tmp/, should be on a separate partition. This reduces the risk of a user DoS by filling up your “/” mount point and rendering the system unusable (Note: this is not strictly true, since there is always some space reserved for root which a normal user cannot fill), and it also prevents hardlink attacks. 1
  • Any partition which can fluctuate, e.g. /var (especially /var/log) should also be on a separate partition. On a Debian system, you should create /var a little bit bigger than on other systems, because downloaded packages (the apt cache) are stored in /var/cache/apt/archives.
    • From a security point of view, it makes sense to try to move static data to its own partition, and then mount that partition read-only.

But how?

In Linux you can force the file system to mount a partition with some restrictions. Those restrictions can be used on a mount command as option or on the /etc/fstab file.

Note that some mount options are filesystem specific. The most common options are:

I’ll enumerate the ones we will focus on. (Click here for a more complete list)

  • noexec – Disallow execution of binaries on the filesystem.
  • ro – Mount the filesystem read-only.
  • rw – Mount the filesystem read-write.
  • nouser – Allow only root to mount the filesystem.
  • nodev – Don’t interpret block special devices on the filesystem.
  • nosuid – Block the operation of suid, and sgid bits.
  • defaults – the default mount options for the filesystem to be used. The default options for ext4 are: rw, suid, dev, exec, auto, nouser, async.

For example I have a partition I use whose only purpose is to store backups, hence it has no need for most options. Here it’s how it looks

/dev/xvdf /serverBackups ext3 rw,relatime,errors=continue,nosuid,noexec 0 0

This tells the system to mount:

  • Device: /dev/xvdf/
  • Mount Point: /serverBackups
  • FileSystem: ext3
  • Options: rw,relatime,errors=continue,nosuid,noexec
  • The last 0 zeros tell the system not to use dump for backups and not to check them with fsck.

Your set up depends on how you use or like your system.

Sample Partion Scheme

  • /tmp – Set nodev,nosuid,noexec
  • /home – Set nosuid, and nodev with disk quota option
  • /usr – Set option nodev
  • /var/log – Set nodev,nosuid,noexec

For more information on /etc/fstab check this.

Secure SSH

This days almost all flavors of Linux have SSH included and on by default. SSH allows you to connect and administer your server remotely in a secure way. Still you can further increase the security of your server by tweaking the configuration.

To make the changes just open the configuration file and reload/restart the service(when your done)

I recommend you open the file and check if the directive already exists before attempting the change.

Configuration Files and Port

On most systems you can find SSH files in here

  • /etc/ssh/sshd_config – OpenSSH server configuration file.
  • /etc/ssh/ssh_config – OpenSSH client configuration file.

The default port is

  • SSH default port : TCP 22

Configuration Changes (Server file /etc/ssh/sshd_config )

Use SSH 2 Only

SSH protocol version 1 (SSH-1) has man-in-the-middle attacks problems and security vulnerabilities. SSH-1 is obsolete and should be avoided at all cost. To force ssh to use protocol 2 add this.

Protocol 2

This is the default in Debian 7 😉

Limit Users Access

By default all users have SSH access. This may pose a security risk for accounts that don’t need this.

You can black list (DenyUsers) or white list (AllowUsers) users using this directives:

AllowUsers bob thomas jorge

or

DenyUsers mark daniel alan

You can do the same for groups

AllowGroups admins techSupport

or

DenyGroups mortalUsers ftpOnly

Disable root Login via SSH

To be honest this point is debatable I don’t see any reason to login directly as root. As a matter of fact I don’t even remember my root password! is a BIG password, so I disallow root login and I log in with my own account. Then I do sudo to gain root level access. This also make sure you get full auditing information about who ran privileged commands on the system via sudo.

PermitRootLogin no

Use Strong SSH Passwords

The importance of a good password should be obvious to any IT professional, if not go read about brute force, rainbow tables and all that stuff.

Use PAM to enforce good quality passwords

From wikipedia

A pluggable authentication module (PAM) is a mechanism to integrate multiple low-level authentication schemes into a high-level application programming interface (API). It allows programs that rely on authentication to be written independently of the underlying authentication scheme.

PAM is standard this days, so lets use it to force good quality passwords (this applies to all users but root)

First lets install Cracklib

admin@var-log-it:~$ sudo apt-get install libpam-cracklib

On Debian you do

admin@var-log-it:~$ sudo vim /etc/pam.d/common-password

RedHat, CentOS and others use

admin@var-log-it:~$ sudo vim /etc/pam.d/system-auth

Here is my default configuration, but you can tweak it
It allows 3 password retrys, minimum password lenght is 12, can reuse 6 characters of old password, forces at least 2 digits and 2 other characters (symbols)

password   requisite   pam_cracklib.so retry=3 minlen=12 difok=6 dcredit=2 ocredit=2

Where

  • retry=# : Le the user retry password input # times before returning error
  • minlen=#: minimum length allowed for an account password is set to 10 characters. This is the minimum simplicity count for a good password. And you are allowed only 2 times using retry option.
  • difok=#: How many characters can be the same in the new password relative to the old. User will see error – BAD PASSWORD: is too similar to the old one

The following options allow you to control the ‘unsimplicity’ of the password.

  • dcredit=N : Digits characters
  • ucredit=N : Upper characters
  • lcredit=N : Lower characters
  • ocredit=N : Other characters

On Debian you have to enable it.

 admin@var-log-it:~$sudo pam-auth-update

Lastly dont forget to enable PAM for ssh

 admin@var-log-it:~$ sudo vim /etc/ssh/sshd_config

Add this to the file

 UsePAM yes

After the changes are doen dont forget to reload the SSH daemon

 admin@var-log-it:~$ sudo /etc/init.d/sshd reload

Stop Using Passwords…wait what? Use Keys instead

We already enforce good quality passwords, but typing long complex passwords becomes a burden rapidly. Instead why dont you use

Public Key Based Authentication

I wont go into details about this one but theres a great article in here

Then go to ssh config file

 admin@var-log-it:~$ sudo vim /etc/ssh/sshd_config

And set this

PasswordAuthentication no
RSAAuthentication yes
PubkeyAuthentication yes

Those are just a few of the steps to take. I will cover more over the upcoming days.

See ya and have fun hacking around! 😉

Advertisements

How to upgrade fedora 17, 18

The latest fedora version have a new recommended way to upgrade the system

Enter FedUp

From the Fedora Project Wiki

FedUp (FEDora UPgrader) is the name of a new system for upgrading Fedora installs in Fedora 18 and above releases. It replaces all of the currently recommended upgrade methods (PreUpgrade and DVD) that have been used in previous Fedora releases. Anaconda, the Fedora installer, has no built-in upgrade functionality in Fedora 18 or above releases. It has been completely delegated to Fedup.

How do I Upgrade Fedora then?

Buried in the documentation you can find the actual commands needed.

First update/install FedUp

sudo yum --enablerepo=updates-testing install fedup

After that you need actually use FedUp passing the method of update and version, in this case Network update to Fedora 19

sudo fedup-cli --network 19

Is that it?

Actually no…

  1. Reboot the system if fedup has completed without error.
  2. Once the system reboots, there should be a new entry in the GRUB menu titled System Upgrade.
    • If you add rd.upgrade.debugshell boot argument, you will get a login shell on VT2, allowing you to tinker with the system in case something goes wrong
  3. Select the System Upgrade option from the GRUB menu
    • Remark: If the System Upgrade item is not shown in the grublist at boot, it is most often caused by having a different grub, most often installed by another Linux distribution you may have in multiboot. To correct this quickly: reinstall grub:
      1. grub2-mkconfig -o /boot/grub2/grub.cfg
      2. grub2-install /dev/sda (replace /dev/sda by any other device you prefer to boot from)
  4. The system should boot into the upgrade process and a plymouth boot screen should be displayed
    • If you press ‘esc’, a more detailed log of progress will be desplayed but if you switch back to the graphical progress indicator, it will remain at 0% for the remainder of the upgrade but that does not mean the upgrade has stopped. See Need section reference here once it’s written
  5. Once the upgrade process has completed, the system will reboot and an option to boot Fedora 19 will be on the grub menu

Now everything should be good to go enjoy 😉

Limit CPU for a process

CPU Limit is a tool that allows you Limit the amount of CPU Time your Linux gives to a certain process. It is also very easy to use.

Last week I installed cpulimit to help me throttle nginx php-cgi process. When running a web server with php this can be a CPU hog so I decided to try and limit the CPU time php-cgi gets.

First install it

sudo apt-get install cpulimit

To Use run this

sudo cpulimit --path=/usr/bin/someBinary -l 70 -z -b

That will limit that binary to 70% CPU time and by running cpulimit in the background.

Continue reading “Limit CPU for a process”

Change Rsyslog log format ( and time zone )

So theses days I’m still working on closing holes in my setup (security wise). I’ve been getting hit by a lot of bots on SSH as its normal on public facing servers. So today I was checking at my logs and I spotted potential problems.

Tracking the issues

I went first to get a list of accepted logins

me@var-log-it:~$ sudo grep 'Accepted publickey for' /var/log/auth.log | less

The log was showing my user connecting to the server at time that I wasn’t there. So I did a whois to the IP and so…

me@var-log-it:~$ whois xx.xx.xx.xx

It turns out I get my ISP in the owner of the domain.

This logs correspond to me connection from home to the server, still the time did not matched, so I went on to see of Rsyslog was logging with UTC and google helped.

Problem Found

It turns out that Rsyslog default log format does not takes timezone into logging message format it logs int UTC by default.

To make Rsyslog match your server date with time zone included, you need to change Rsyslog’s default log format.

I tried using some of the options available like… (not all listed!)

  • RSYSLOG_FileFormat – a modern-style logfile format with high-precision timestamps and timezone information

  • RSYSLOG_TraditionalForwardFormat – the traditional forwarding format with low-precision timestamps.

  • RSYSLOG_ForwardFormat – a new high-precision forwarding format with high-precision timestamps and timezone information.

  • RSYSLOG_SyslogProtocol23Format – the format specified in IETF’s internet-draft ietf-syslog-protocol-23, which is assumed to be come the new syslog standard RFC.

But they didn’t quite fit my taste.

Using some of those options made my logs look ugly, for example while using RSYSLOG_FileFormat I got this

2013-10-03T12:28:00.429271-06:00 var-log-it me: test

Look at that date is awful! no spaces and showing 6 figures after the second! I had no need for that

How to fix it and pimp it your way!

To output the time the way I wanted it, I had to change the template for a custom one.

sudo vim /etc/rsyslog.conf

Locate the section

$ActionFileDefaultTemplate

Replace it with this

$template CustomFormat,"%timegenerated% %HOSTNAME% %syslogtag%%msg:::sp-if-no-1st-sp%%msg:::drop-last-lf%n"
$ActionFileDefaultTemplate CustomFormat

Taken from Rsyslog format documentation
This will change the default template to the custom one specified there, and the final result looks like this:

Oct  3 12:29:33 var-log-it me: test

NOTE: Remember to reload rsyslog after the change!

you@sever:~$ sudo /etc/init.d/rsyslog force-reload

Hope it helps 😉

Make NTP match your time zone

So today I installed ntp on the server so ti would have accurate time reading.

Install its super easy as always on Debian (or derivatives)


admin@var-log-it:~$ sudo apt-get install ntp
....
All done!

With that out of the way I ran date


admin@var-log-it:~$ date
Mon Sep 23 10:58:04 MDT 2013

But the time is off by several hours. To fix we just need to reconfigure a package so it knows our time zone.

Now run


admin@var-log-it:~$ sudo dpkg-reconfigure tzdata

Select your time zone and your done!

Fix Postfix database /etc/aliases.db is older than source file /etc/aliases

So today I receive this message in my daily logwatch summary


warning: database /etc/aliases.db is older than source file /etc/aliases

Since I’ve been messing with Postfix I’m not surprised but even though it was working I don’t like warning messages in my logs if I can help it.

Turns out its very easy, we need to tell Postfix that we have new aliases so it can update its internal database. To do this run the following command (as root)


admin@var-log-it ~ # newaliases

Now reload your Postfix instance


admin@var-log-it ~ # postfix reload
postfix/postfix-script: refreshing the Postfix mail system

And your done! check your logs if your curious (like me) ;P see ya!

How to set PATH variables Linux, Bash

Hey there, this is a classic question in Linux systems.

It shows whenever you wanna run certain command and you get


admin@yourserver:~$ whatever
bash: whatever: command not found

You need to add ‘whatever’ (or the program you want) path into your current path

To see your current path do this


admin@yourserver:~$ echo $PATH
/usr/local/bin:/bin:/usr/local/games:/usr/games

You can see my current path, lets say I want to add /usr/bin I will do this

PATH=$PATH:/usr/bin

note: You don’t need export if the variable is already exported: any change of the value of the variable is reflected in the environment. (This wasn’t true in older shells, but you’re highly unlikely to encounter such old shells these days.)
Check this post for more info

Now you can check again


admin@yourserver:~$ echo $PATH
/usr/local/bin:/bin:/usr/local/games:/usr/games:/usr/bin

Check how /usr/bin was added to the end.

And that’s it very easy and useful, next time just call command by name instead of full path 😉