Sorry for being away from the blog for this long period but things were overcrowded here with all this new sign ups and upgrades we were doing so we were on a tight time schedule but anyhow I got some free time to write back here to all the followers of our blog .

This Howto will help you virus can all uploaded via FTP to make sure all files are virus free:

First lets install clamAV , it has been now been included in Cpanel/WHM you can do so from your WHM interface as root by clicking on : Cpanel —> Manage Plugins —> on clamavconnector
Check the “Install and Keep Updated” and then click at the bottom of the page save.

After install finishes edit the file /etc/pure-ftpd.conf and change this line to look like this :

CallUploadScript yes

Next create file /etc/pure-ftpd/clamav_check.sh with this content :

#!/bin/bash
#Maximum file size to scan in bytes that's set to 10MB
MAXSIZE=10485760
if [ "$UPLOAD_SIZE" -le "$MAXSIZE" ]; then
    /usr/bin/clamdscan  --remove --quiet --no-summary "$1"
fi

Change its permissions so its executable :

chmod 755 /etc/pure-ftpd/clamav_check.sh

Now we should start the pure-uploadscript daemon to call our script every time a file is uploaded

/usr/sbin/pure-uploadscript -B -r /etc/pure-ftpd/clamav_check.sh

And to let it start with your server/vps reboots just run this :

echo "/usr/sbin/pure-uploadscript -B -r  /etc/pure-ftpd/clamav_check.sh" >> /etc/rc.d/rc.local

Now restart pure-ftpd :

service pure-ftpd restart

Now all your uploaded files which are less than 10MB in size will be scanned , adjust this size limit per your needs

Last year EXT4 was announced as stable release and it was merged in kernel 2.6.28 as part of the coder afterwards, it brings many great features am stating some here before getting on how to move an existing partition to ext4 without formatting

Large file system
The ext4 filesystem can support volumes with sizes up to 1 Exbibyte (1,152,921,504,606,846,976 bytes) and files with sizes up to 16 tebibytes (1,099,511,627,776 bytes).

Extents
Extents are introduced to replace the traditional block mapping scheme used by ext2/3 filesystems. An extent is a range of contiguous physical blocks, improving large file performance and reducing fragmentation. A single extent in ext4 can map up to 128MB of contiguous space with a 4KB block size. There can be 4 extents stored in the inode. When there are more than 4 extents to a file, the rest of the extents are indexed in an Htree.

Persistent pre-allocation
The ext4 filesystem allows for pre-allocation of on-disk space for a file. The current methodology for this on most file systems is to write the file full of 0s to reserve the space when the file is created. This method would no longer be required for ext4; instead, a new fallocate() system call was added to the Linux kernel for use by filesystems, including ext4 and XFS, that have this capability. The space allocated for files such as these would be guaranteed and would likely be contiguous. This has applications for media streaming and databases.

my favorite ;)
Journal checksumming
Ext4 uses checksums in the journal to improve reliability, since the journal is one of the most used files of the disk. This feature has a side benefit; it can safely avoid a disk I/O wait during the journaling process, improving performance slightly.

NOTE
Filesystem changes/hacks contains certain risk ,though we tested this on our system we don’t guarantee this will work perfectly on your systems and we shall not be responsible for any data loss or hard disk crash .
p.s. backup your system before proceeding .

Step 1 :
Assuming we are changing sda2 edit its line in /etc/fstab to look like this

/dev/sda2     /                ext4    defaults      1  1

Step 2 :
Make sure needed packages are up-to-date then update your initrd image

yum upgrade mkinitrd e2fsprogs
mv /boot/initrd-`uname -r`.img /boot/initrd-`uname -r`.img.old
mkinitrd -v --with=ext4 /boot/initrd-`uname -r`.img `uname -r`

Step 3 :
Boot using your OS CD-ROM into rescue mode and run this commands to apply the new FS extensions , make sure you’re working on the correct partition and you’re using

/mnt/sysimage/sbin/tune2fs -I 256 -O has_journal,extents,huge_file,flex_bg,uninit_bg,dir_nlink,extra_isize /dev/sda2

The last command should tell your to run fsck and reboot if it didn’t it has failed , and you should remove the tune2fs option it asks to be removed

Last step :

fsck -pf /dev/sda2

Port Knocking is an approach that helps protecting your services ports from attacks the most famous brute forced would be SSH , the port knocking method depends that the client trying to connect will first attempt to connect to a predefined ports which will enable connection to your secured service port for 5 seconds , here is a simple script to set this kind of protection

#!/bin/sh
#
# Netfilter/IPtables - example of multiple-port knocking
# Note: Knock ports 3456,2345,1234 to open SSH and MySQL ports for 5 seconds.
# Nice thing to knock TCP with is `nc' program:
# $> nc -w 1 <ip> 3456 ; nc -w 1 <ip> 2345 ; nc -w 1 <ip> 1234 ,ssh <ip>
#
# Change this to the name of the interface that provides your "uplink"
# (connection to the Internet) or connection you want to protect.
UPLINK="eth0"
#
# Comma seperated list of ports to protect with no spaces.
SERVICES="22,3306"
#
# Location of iptables command
IPTABLES='/sbin/iptables'
#
${IPTABLES} -N stage1
${IPTABLES} -A stage1 -m recent --remove --name knock
${IPTABLES} -A stage1 -p tcp --dport 3456 -m recent --set --name knock2

${IPTABLES} -N stage2
${IPTABLES} -A stage2 -m recent --remove --name knock2
${IPTABLES} -A stage2 -p tcp --dport 2345 -m recent --set --name heaven

${IPTABLES} -N door
${IPTABLES} -A door -m recent --rcheck --seconds 5 --name knock2 -j stage2
${IPTABLES} -A door -m recent --rcheck --seconds 5 --name knock -j stage1
${IPTABLES} -A door -p tcp --dport 1234 -m recent --set --name knock

${IPTABLES} -A INPUT -m --state ESTABLISHED,RELATED -j ACCEPT
${IPTABLES} -A INPUT -p tcp --match multiport --dport ${SERVICES}  -i ${UPLINK} -m recent --rcheck --seconds 5 --name heaven -j ACCEPT
${IPTABLES} -A INPUT -p tcp --syn -j door

Some known pitfalls of port knocking are :

1- using consecutive port numbers like 100,200,300 which would cause the secured service port to be opened while using a port scanner like Nmap.

2- port knocking is not effective against replay attacks

3- using port knocking as your sole line of defense, it has to be one of many .

Some other famous scripts used for port knocking are fwknop and knockd

A new exploit has been discovered in WordPress the famous blogging web application , the exploit resets the admin password without the need of the “Password Reset” email the detailed explanation is here , here is the exploited code :

function reset_password($key) {
    global $wpdb;

    $key = preg_replace('/[^a-z0-9]/i', '', $key);

    if ( empty( $key ) )
        return new WP_Error('invalid_key', __('Invalid key'));

    $user = $wpdb->get_row($wpdb->prepare("SELECT * FROM $wpdb->users WHERE user_activation_key = %s", $key));
    if ( empty( $user ) )
        return new WP_Error('invalid_key', __('Invalid key'));

Since the key is only checked for being an empty string then if you pass an empty array (which is a different type of variables) we will bypass this step and the password will be reset right away and sent to the admin listed e-mail .

In order to fix this problem edit wp-login.php with your favorite editor and change this lines :

Line 190 in WordPress 2.8.3 or line 169 in earlier 2.8 versions

if ( empty( $key ) )

TO

if ( empty( $key ) || is_array( $key ) )

This will add a check to invalidate passing an array to the $key variable .

Enjoy!

This are a set of commands that a sys admin could use to perform various tasks on a server , varying from listing server Ips to optimizing MySQL tables , hope it helps you managing different tasks fast :

!!:gs/foo/bar

Runs previous command replacing foo by bar every time that foo appears
Very useful for rerunning a long command changing some arguments globally.

ping -i 60 -a IP

Set audible alarm when an IP address comes online
Waiting for your server to finish rebooting? Issue the command above and you will hear a beep when it comes online. The -i 60 flag tells ping to wait for 60 seconds between ping, putting less strain on your system. Vary it according to your need. The -a flag tells ping to include an audible bell in the output when a package is received (that is, when your server comes online).

du -b --max-depth 1 | sort -nr | perl -pe 's{([0-9]+)}{sprintf "%.1f%s", $1>=2**30? ($1/2**30, "G"): $1>=2**20? ($1/2**20, "M"): $1>=2**10? ($1/2**10, "K"): ($1, "")}e'

Sort the size usage of current directory tree by gigabytes, kilobytes, megabytes, then bytes.

tr -dc A-Za-z0-9_ < /dev/urandom | head -c 16;echo

useful for generating passwords,
Find random strings within /dev/urandom. Using tr to use only Alphanumeric characters, and then print the first 16.

mkdir -p a/long/directory/path

This will create the intermediate directories that do not exist.

ctrl-t

Switch 2 characters on a command line.
If you typed ‘sl’, put the cursor on the ‘l’ and hit ctrl-t to get ‘ls’.

grep -i --color=auto

Highlights the search pattern in red.

!!

Repeat last executed command

DD=`cat /etc/my.cnf | sed "s/#.*//g;" | grep datadir | tr '=' ' ' | gawk '{print $2;}'` && ( cd $DD ; find . -mindepth 2 | grep -v db\.opt | sed 's/\.\///g; s/\....$//g; s/\//./;' | sort | uniq | tr '/' '.' | gawk '{print "CHECK TABLE","`"$1"`",";";}' )

Generate CHECK TABLE statements for all MySQL database tables on a server

export dbname=DB;for i in `mysql --batch --column-names=false -e "show tables" $dbname`;do  mysql -e "ALTER TABLE $i DEFAULT CHARACTER SET utf8 COLLATE utf8_unicode_ci" $dbname;done

Change all tables inside a database to UTF8 character set.

ifconfig | grep "inet [[:alpha:]]\+" | cut -d: -f2 | cut -d' ' -f1

Get the IP address of all your network cards.

Usually the kernel handles memory utilization pretty well it caches memory for dentry cache, page cache and inodes which improves IO speed and performance generally. But in some cases user applications needs lots of memory and we need to clear what’s called dirty memory which could be inodes already written to the disk, so now the kernel given us the option to manage this manually.

First of all Run sync command to flush all file system buffers

Then run one of this commands :

echo 1 > /proc/sys/vm/drop_caches

This will free only page caches

echo 2 > /proc/sys/vm/drop_caches

This will free inodes and dentry caches

echo 3 > /proc/sys/vm/drop_caches

This will free ALL

Normally when you build a custom kernel you use some config files available online which have everything enabled to assure everything works, but this elongates compilation time which also could make your kernel bigger than needed, finally Steven Rostedt has come up with a awesome solution for this problem .

He developed a perl script “”streamline_config” that you can download here it will make a custom config file that will still boot your box, but bring down the compile time of the kernel can be really long.

The script will perform “lsmod” to find all the modules loaded on the current running system. It will read all the Makefiles to map which CONFIG enables a module. It will read the Kconfig files to find the dependencies and selects that may be needed to support a CONFIG. Finally, it reads the .config file and removes any module “=m” that is not needed to enable the currently loaded modules. The output goes to standard out.

Here is Steven’s thread about the script

Steps to run the script :

1- Boot up the kernel that you want to streamline the config on.
2- Change directory to the directory holding the source of the kernel that you just booted.
Then run this commands :

./streamline_config.pl arch/x86/Kconfig > new_config
mv .config config_old
mv new_config .config
make oldconfig

if you wanna modify anything you can run make menuconfig instead of make oldconfig

I know this has been considered impossible for quiet a long time but its now possible – actually for quiet awhile now- thanks to foremost software, it can recover files based on their headers, footers, and internal data structures. This process is known as data carving.

Foremost can work on image files, such as those generated by dd, Safeback, Encase, etc, or directly on a drive. The headers and footers can be specified by a configuration file or you can use command line switches to specify built-in file types. These built-in types look at the data structures of a given file format allowing for a more reliable and faster recovery.

Foremost can recover files with the following extensions:
jpg – Support for the JFIF and Exif formats including implementations used in modern digital cameras.
gif
png
bmp – Support for windows bmp format.
avi
exe – Support for Windows PE binaries, will extract DLL and EXE files along with their compile times.
mpg – Support for most MPEG files (must begin with 0x000001BA)
wav
riff – This will extract AVI and RIFF since they use the same file format (RIFF). note faster than running each separately.
wmv – Note may also extract -wma files as they have similar format.
mov
pdf
ole – This will grab any file using the OLE file structure. This includes PowerPoint, Word, Excel, Access, and StarWriter
doc – Note it is more efficient to run OLE as you get more bang for your buck. If you wish to ignore all other ole files then use this.
zip – Note is will extract .jar files as well because they use a similar format. Open Office docs are just zipped XML files so they are extracted as well. These include SXW, SXC, SXI, and SX? for undetermined OpenOffice files.
rar
htm
cpp – C source code detection, note this is primitive and may generate documents other than C code.
You can tweak /etc/foremost.conf to add support for more file types.

Please note that there’s no guarantee that foremost will succeed in recovering your files, but at least there’s a chance.

On Debian and Ubuntu, foremost can be installed simply by issuing this command :

apt-get install foremost

or download and extract the source from the tar.gz and run this inside the extracted dir :

make;make install

So easy , now you’re ready to start using foremost here are some example usage:

foremost -t jpeg -i /dev/sda1

in this example we are looking for all deleted jpg files on /dev/sda1 , foremost always output the results in directory called output, its created in the directory where you ran foremost so be sure to run it from a directory not located in /dev/sda1 to avoid overwriting one of your deleted files ;)

Inside the output directory you will find a file named audit.txt which the log of this restoration process and also a directory named jpg cuz we were searching for jpegs in our last example

foremost -t pdf -T -i /dev/sda1

In this example we are searching for pdf files and appending results to the output directory (because foremost will not start if there’s already an output directory)

foremost -s 100 -t jpg -i image.dd

Here we are searching for jpeg files skipping the first 100 blocks inside this dd image

foremost -t all -i /dev/sda1

Searching for all predefined types

We do appreciate and respect all our customers, this post is just for the sake of laughing :)

Lie. Seriously, people will do this, even when you can easily prove it, and when they know that you can. I have a lot more respect for someone that will say I screwed things up than for someone who pretends he has done nothing wrong at all.

Rant. Seriously, I don’t give a heck how many other people you’ve talked to who don’t have this problem, how much am paying for the service, or what kind of problems you’ve had in the past. That does NOTHING WHATSOEVER to help me solve your immediate problem.

Do things to your computer, while they’re on the phone with you, that they didn’t tell you to. They’re usually going through a mental checklist of things that could be wrong. If you could’ve done this by yourself, then why did you call in the first place?!

Ask them very open-ended questions.

Call and ask ‘Is the server down?’ They love that.

When something doesn’t work, randomly change configuration options until it does.

Claim to be knowledgable due to some bloated certification. Claim that you could fix the problem faster.

Use Windows ICS. Call your webhosting provider about problems that are obviously related to your internal network configuration. Tell them they should help you.

Email support and complain that you can’t send mail.

Get annoyed and huffy when they assume you’re a moron.

Relate to them how your company would handle a problem of this nature with one of it’s clients.

Tell them exactly how your day is going Include amusing anecdotes.

Be as nonspecific as possible.

Its always annoying to move accounts around between servers because you have to coordinate lots of changes for example :

1- dns changes
2- minimizing downtime so you wont get users screaming at your face
3- notifying your users early enough (beware you will never be early enough)

In this tutorial we will guide you in step by step in how to perform a flawless migration.

Please note this tutorial is for migrating some accounts on the server not all the server, it will work well if you use the steps we mention on all accounts as well .

Now lets rock and roll.

First step has to be done 48-72 hours prior to migration (this used to be 24-48 hours back in the good old days when root nameservers acted correctly ) :

First we create a file named accounts which have the domains we need to transfer one domain in each line like this

domain1.com
domain2.com

Then run this command

for account in ` cat accounts `;do perl -pi.bak -e "s/14400/300/g if /^\s*\$TTL/" /var/named/$account.db;done
/etc/init.d/named restart

Here we have edited all  DNS zones listed in our accounts file and created a .bak of each (remember always backup). Changing the TTL (time To Live) from 14400 (4 hours) to 300 (5 minutes) means that as soon as we make a change to a DNS zone, it will take affect everywhere within 5 minutes as opposed to 4 hours, personally I find that this doesn’t work perfectly anymore but I will tell you how to overcome it ;) 

Now in the day designated for the migration

First we stop all services on the old server other than apache and MySQL ( to avoid loosing any emails or other stuff)

/etc/init.d/cpanel stop
/etc/init.d/exim stop
/etc/init.d/pure-ftpd stop;/scripts/ckillall -9 pure-ftpd;/scripts/ckillall -9 pure-authd
/etc/init.d/proftpd stop

After that login to WHM  on the new server as root then :

1- Click on copy multiple accounts
2- Fill out the old server data
3- Select accounts to transfer
4- Initiate the transfer process

Once the transfer has finished ssh to the old server and run this commands :

for account in ` cat accounts `;do perl -pi -e "s/[oldip]/[newip]/g" /var/named/$account.db;done
/etc/init.d/named restart

This will point all the sites to the newip. Hopefully most of your sites are on a shared ip. For all the accounts that are NOT on a shared ip we will need to edit that domain dns zone manually to reflect the new IP.

Transfer completed. Sites are functioning on new server, now transfer the nameservers! Here we go:

Be sure to set up your NEW nameservers on the NEW server. (WHM will only add an A record for the oldips) until you transfer the nameservers with the registrar) so you need to edit the DNS zones to reflect the new ips.

Submit the ip changes to your registrar and give it 48 hours to update. Within 48 hours your DNS will be served off your new server, thus you can cancel that old server with that host you couldn’t wait to leave.

If you have resellers, we have to copy all the files in /var/cpanel that reference resellers and anything ending in .accts to the new server.

Still one problem persists, some nameservers doesn’t respect TTL changes and work on their own schedule (usually 48 hours)

What can we do to overcome this , if we use iptables to route all traffic to the new ip we will face a problem cuz of the shared ip on the old server , we may have other domains still using it.

The solution to this problem is using Apache mod_proxy , here is how to do so:

Edit /usr/local/apache/conf/httpd.conf and inside each transferred site virtualhost directive and add this :

ProxyRequests Off
ProxyPass / http://domain.com/
ProxyPassReverse / http://domain.com/

If one of the domains has SSL certificate use this :

SSLProxyEngine On
ProxyRequests Off
ProxyPass / https://domain.com/
ProxyPassReverse / https://domain.com/

To assure that this domains resolves to the new ips you can edit /etc/hosts and add this domains with their new ips
Then mod_proxy will direct traffic to the new server.

Once the lagged nameservers have updated their cache – usually in 48 hours as I mentioned before – you will be good to go.

Migrating a server as a whole is a different story which strangely is easier but that’s a different article , hope this helps anyone out there .