Recursively copy from a FTP server
Linux, Windows November 18th, 2011
In Linux (and also in the World of Windows), there is a small great tool called wget (you probably already know it, it’s pretty popular). I have been using it for some years know as it is very convenient for downloading files (or even entire HTML websites) from the command line (specially when you manage a Linux server through SSH). What I didn’t know about it was that it also allows you to recursively copy files from a FTP server (a task that is not usually pleasant).
This is how:
wget -r ftp://username:password@ftp.server.com/*
If the FTP username is something like this: user@somedomain.com, then, you should replace the @ by a + simbol, like this: user+somedomain.com.
Denyhost: Permanently whitelist a host
Linux September 3rd, 2011
Thanks to this site, I found an easy way to whitelist a host when using denyhosts. Just need to create a file called allowed-hosts with the IP addressses you want to whitelist inside /var/lib/denyhosts (or the work directory in your box).
In my case, I found very useful to whitelist the localhost address (127.0.0.1) to avoid some problems.
Debian/Ubuntu’s apt behind a proxy
Linux, Ubuntu January 27th, 2011
To allow the Advanced Packaging Tool, or just APT for friends to do its job when the box is behind a proxy, we can use one of the following options.
From the command line (this will only work for the session we are working on):
export http_proxy=http://username:password@server:port
Or if the proxy does not require a username:
export http_proxy=http://server:port
Another option is to add the following line into the file /etc/apt/apt.conf (note the semicolon at the end):
Acquire::http::Proxy "http://username:password@server:port";
Nginx and Munin
Linux November 6th, 2010
To be allowed to show Nginx graphs on the Munin report, it is required to enable nginx status and to listen on:
http://127.0.0.1/nginx_status
To do this, we just need to add the following code on /etc/nginx/sites-enabled/default or into any other site file (like create a new one /etc/nginx/sites-available/status and then make symbolic link to site-enabled directory):
server {
listen 127.0.0.1;
server_name localhost;
location /nginx_status {
stub_status on;
access_log off;
allow 127.0.0.1;
deny all;
}
}
Source: Server Fault
No DEFAULT or UI configuration directive found
Linux November 3rd, 2010
When I tried to run the VMWare ESXi installer from a USB stick (after creating the installer on a USB from the ISO), I got the following error:
SYSLINUX 3.85 2010-02-20 CBIOS Copyright (c) 1994-2010 H. Peter Anvin et al No DEFAULT or UI configuration directive found! boot:
The solution was pretty simple thanks to this post. I just needed to rename a couple files on the USB as follows:
- isolinux.bin to syslinux.bin
- isolinux.cfg to syslinux.cfg
It worked flawless. But if that does not work for you, you may try formatting the USB as FAT instead of FAT32.
Disable cron log on syslog and enable cron.log
Linux, Ubuntu October 17th, 2010
By default, Ubuntu logs all cron activity directly to syslog (/var/log/syslog), instead to a dedicated file like /var/log/cron.log.
In my opinion, it is best to keep them separate if you have several cron jobs running on your server.
To enable this behavior, what we must do is edit the syslog configuration file /etc/syslog.conf, and modify the following lines:
- Change
*.*;auth,authpriv.nonewith*.*;auth,authpriv,cron.none, this will avoid the messages to be recorded on the syslog file. - Uncomment (removing the # character) the line starting
cron.*, this will record the messages to the specified file on that line.
Then, we need to restart the syslog service, doing the following:
$ sudo /etc/init.d/sysklogd restart
And maybe, restarting the cron service also:
$ sudo /etc/init.d/cron restart
Resolving troubleshooting with Munin
Linux April 14th, 2010
This are some problems that you may encounter after installing Munin:
Apache modules are not tracking, so they are not showing statistics, you may want to check Munin logs (specifically /var/log/munin/munin-node.log) to see what is going on.
If you got something like Can’t locate object method “new” via package “LWP::UserAgent”, you need to install the package libwww-perl which contains the LWP::UserAgent required. Thanks to Crowdway.
If you already have installed the libwww-perl package, but Munin is still not showing data, you may need to enable and configure the status module on Apache. Make sure you allow localhost (and the name of the server as well) to access the location server-status on apache2.conf or status.conf. Also, this plugins need the ExtendedStatus flag to be On.
You should have something like:
<IfModule mod_status.c>
#
# Allow server status reports generated by mod_status,
# with the URL of http://servername/server-status
# Uncomment and change the ".example.com" to allow
# access from other hosts.
#
ExtendedStatus On
<Location /server-status>
SetHandler server-status
Order deny,allow
Deny from all
Allow from localhost ip6-localhost
Allow from vps02.graphium.net
#Allow from .example.com
</Location>
<IfModule>
Sometimes you can get permission errors on your logs, so the best way to get rid of them is to ensure that all involved files in Munin process are own by munin user and group. We can run as root:
$ chown -R munin.munin /var/lib/munin /var/log/munin /var/www/munin
There are some plugins that require a parameter to work. For example, those that refer to networking (if_, if_err_, ip_) needs the network interface we want to track. In my case (VPS), this is venet0, but on a usual box it can be eth0.
We can list the available interfaces by running the command:
$ ifconfigThen, to correctly enable this modules, we need to append the name of the interface:
$ ln -s /usr/share/munin/plugins/if_ /etc/munin/plugins/if_venet0
A convenient way to see the result of each module, we can run them independently:
$ munin-run apache_processes busy80.value 7 idle80.value 1 $ munin-run if_venet0 down.value 7721322 up.value 37550800
By this command, we can ensure that the plugin is working fine and getting results.
Tags: Apache, Monitoring, Munin, Ubuntu, VPS
Install Munin on Ubuntu Server
Linux, Ubuntu April 6th, 2010
Munin is a small tool for monitoring resources on servers. I think it is very useful, specially on small VPS, that needs to save resources. Reports are written as HTML files, so we will need a Web Server like Apache to see this reports.
First, we install it and add some extra plugins:
$ sudo apt-get install munin munin-plugins-extra
Now, we can make some changes to the default configuration, located at /etc/munin/munin.conf. For example, we can change any of the paths where Munin works:
dbdir /var/lib/munin htmldir /var/www/munin logdir /var/log/munin rundir /var/run/munin
Specially, the htmldir path, where all the reports are written to see through Apache or the one you are using. Remember to move the directory /var/www/munin to where you wanted if you change that configuration line. We can protect this directory with an htaccess file to only give access to some users.
We can configure email notifications if a change occur (like from a OK situation to a WARNING). To do this, we just need to uncomment or add the following line:
contact.someuser.command mail -s "Munin notification" your@email.com
By default, Munin will monitor localhost, but we can add other boxes (clients), these machines will only need to install munin-node package.
Then, we can enable some plugins (more plugins can be found here and here). To do this, we need to create a symbolic link per each plugin we want to activate. I’m going to enable apache and mysql modules, but you are free to enable the modules you need:
$ cd /etc/munin/plugins $ sudo ln -s /usr/share/munin/plugins/apache_* . $ sudo ln -s /usr/share/munin/plugins/mysql_* .
Each time a module is enable or disable, we need to restart the service, so we can do the following:
$ sudo /etc/init.d/munin-node restart
Also, it is recommended to reassign all files on the htmldir to munin user and group by doing:
$ sudo chown munin.munin -R /var/www/munin
And then, to avoid waiting 5 minutes until munin cron runs again, we force it by:
$ sudo /usr/bin/munin-cron --force-root
If we are not completely satisfied with the default template, we can modify it, they are HTML files (with some minor special template tags). Anyone with some knowledge of HTML and CSS can do that. We can even download other already created templates (I have found some errors on that template’s JavaScript, I hope I’ll get some time to post the modified template, in the meanwhile, if anyone need it, please drop me a line to send you the files).
Finally, as this post is not as complete as I would like, I leave some links that may help:
- Munin on Ubuntu Documentation
- Monitor Servers and Clients using Munin in Ubuntu
- Munin on Debian Help
Install VirtualBox Guest Additions on Ubuntu Server
Linux, Ubuntu April 5th, 2010
On my work, I have a box with WinXP, running Virtual Box as a host, and a Ubuntu Server 9.10 box as a guest. My problem is that the firewall on the corporate network does not allow Ubuntu to update the date and time against any NTP server (like pool.ntp.org or ntp.ubuntu.com). So, I need a way to keep the hour updated on the guest. Fortunately, VirtualBox has the ability to synchronize it from the host, the only thing I need to do was to install the Guest Additions package.
To do this, first, we need to click on Devices/Install Guest Additions (from the VirtualBox menu).
Then, on the Ubuntu Server, we install some pre-requisites:
$ aptitude install build-essential linux-headers-$(uname -r) -y
Now, we will mount the virtual CD-ROM (where the Guest Additions are):
$ mount /dev/cdrom /mnt/
And then, run the installer script (there is a 32-bit and 64-bit versions). For 32-bit (which is the most probable, as VirtualBox Open Source only supports 32-bit guests):
$ /mnt/VBoxLinuxAdditions-x86.run
Or for 64-bit guest:
$ /mnt/VBoxLinuxAdditions-amd64.run
It should install the available modules (like timesync), and drop a fail message saying that X server
was not found, which is OK as we are working with a server without GUI.
Finally, we umount the CD-ROM:
$ umount /mnt/
Now, the guest box time should be sync with the host, so we have one less thing to worry about.
Tags: NTP, Ubuntu 9.10, VirtualBox, Windows XP
Configuring a Linux Server – Part 1: Sudoer user and SSH
Linux, Ubuntu March 29th, 2010
This will be a series of posts about configuring a Linux server (I’m working with Ubuntu, but should work with minor changes with other distributions), mainly because I have done this several times now by reviewing different websites and blogs, so I wanted to do a guide that works for me (and maybe for others too).
First, we create a user to avoid the use of root (in case you are not using the default Ubuntu installation, that asks you to create a user):
$ adduser myuser Adding user 'myuser' ... Adding new group 'myuser' (1000) ... Adding new user 'myuser' (1000) with group 'myuser' ... Creating home directory '/home/myuser' ... Copying files from '/etc/skel' ... Enter new UNIX password: Retype new UNIX password: passwd: password updated successfully Changing the user information for myuser Enter the new value, or press ENTER for the default Full Name []: Room Number []: Work Phone []: Home Phone []: Other []: Is the information correct? [Y/n] y
It is also possible to use the command useradd (that is a linux command itself, not that user-friendly) instead of useradd (this one is a perl script that makes a little more easy user creation).
We need to assign add this user to the sudoers (users that are able to use the sudo command to make changes to the system).
$ visudo
And add the following line:
myuser ALL=(ALL) ALL
Then, we need to hardening SSH server (be very careful about this modifications because if SSH is the only way you have to configure your server, any error may left you lock out of your box).
Before making any change, we better make a backup of our original config file:
$ cp /etc/ssh/sshd_config /etc/ssh/sshd_config.bk
Then, we need to change the following parameters on /etc/ssh/sshd_config:
- Port Number: Change it to any random port number, try to use one that any known service uses, see the list of services in here.
- Listen Address: Instead of listen to every interface on your server, just listen to the main IP address.
- Protocol: Make sure that this parameter is set to 2 (instead of 2, 1).
- If possible, avoid root login by disabling PermitRootLogin. All root operations can be done using sudo command (at least on Debian/Ubuntu based distributions). Make sure you have created other users on your box, and add them to the allow list of users to log through SSH (AllowUsers [username1] [username2] parameter).
- Disable X11Forwarding.
- Replace password authentication with Public/Private Keys. To do this, make sure your users have their Private Keys and the Public Keys are configured on the server. Then, disable PasswordAuthentication and UsePAM parameters. Make sure that RSAAuthentication and Pubkeyauthentication parameters are enabled.
- If you have noticed that OpenSSH server may take a while from the moment you enter your username and the password prompt, it is because it tries to make some DNS resolves. You can avoid this by disabling UseDNS parameter.
You should end with a file like this:
Port XXXX ListenAddress XXX.XXX.XXX.XXX [...] PermitRootLogin no [...] RSAAuthentication yes PubkeyAuthentication yes [...] PasswordAuthentication no [...] X11Forwarding no [...] UsePAM no UseDNS no AllowUsers myuser user2
Then, we need to restart the SSH server by doing:
$ /etc/init.d/ssh restart
Do not close your current session, just open another to your box using the new port, this way we make sure we are not locked out of it.
On the next post I will be configuring IPTables as a firewall for our box.
UPDATE 17/Apr/2010: Parameter PasswordAuthentication included for disabling keyboard-interactive authentication.
In this link you can find a very good explanation about the main options on SSH config file.
About
