System Administrator Appreciation Day is on the Last Friday of July

Most people read “system” to mean an individual computer, and think that all a sysadmin does is clean viruses off your computer and replace your monitor. That’s not wrong — but it is only one page of the whole story.

A real computing system is larger. Very few computers work just on their own anymore; when you use the web, play a game online, share files with a friend, or send email, you’re using a complex and intricate collection of computers, networks and software that come together to do the job you’re asking.

A sysadmin manages these systems — they figure out how to bring storage from one server, processing from another, backups from a third and networking from a fourth computer all together, working seamlessly. For you

So when you think of a sysadmin, think of the people who run the servers that help you clean it off, the people who run your backups to make sure your data is safe, the people who bring you the network, the people who monitor it for security — and yes, the person who cleans the virus off your computer and replaces your monitor.

So again here are some ideas on how to properly use your System Administrator’s valuable time

• Make sure to save all your MP3 files on your network drive. No sense in wasting valuable space on your local drive! Plus, your sysadmin loves browsing through 100+ GB of music files while he backs up the servers.
• Play with all the wires you can find. If you can’t find enough, open something up to expose them. After you have finished, and nothing works anymore, put it all back together and call your sysadmin. Deny that you touched anything and that it was working perfectly only five minutes ago. your sysadmin just loves a good mystery. For added effect you can keep looking over his shoulder and ask what each wire is for.
• Never write down error messages. Just click OK, or restart your computer. your sysadmin likes to guess what the error message was.
• When talking about your computer, use terms like “Thingy” and “Big Connector.”
• If you get an EXE file in an email attachment, open it immediately. your sysadmin likes to make sure the anti-virus software is working properly.
• When your sysadmin says he coming right over, log out and go for coffee. It’s no problem for him to remember your password.
• When your sysadmin sends you an email marked as “Highly Important” or “Action Required”, delete it at once. He’s probably just testing some new-fangled email software.
• Send urgent email ALL IN UPPERCASE. The mail server picks it up and flags it as a rush delivery.
• Don’t use online help or FAQs or Knowldgebases . They are for wimps.
• When you receive a 130 MB movie file, send it to everyone as a high-priority mail attachment. your sysadmin’s provided plenty of disk space and processor capacity on the new mail server for just those kinds of important things.
• Don’t ever thank your sysadmin. He loves fixing everything AND getting paid for it!

Once again this was just for the sake of laughter don’t feel offended we love you all.

Sorry for being away from the blog for this long period but things were overcrowded here with all this new sign ups and upgrades we were doing so we were on a tight time schedule but anyhow I got some free time to write back here to all the followers of our blog .

This Howto will help you virus can all uploaded via FTP to make sure all files are virus free:

First lets install clamAV , it has been now been included in Cpanel/WHM you can do so from your WHM interface as root by clicking on : Cpanel —> Manage Plugins —> on clamavconnector
Check the “Install and Keep Updated” and then click at the bottom of the page save.

After install finishes edit the file /etc/pure-ftpd.conf and change this line to look like this :

CallUploadScript yes

Next create file /etc/pure-ftpd/clamav_check.sh with this content :

#!/bin/bash
#Maximum file size to scan in bytes that's set to 10MB
MAXSIZE=10485760
if [ "$UPLOAD_SIZE" -le "$MAXSIZE" ]; then
    /usr/bin/clamdscan  --remove --quiet --no-summary "$1"
fi

Change its permissions so its executable :

chmod 755 /etc/pure-ftpd/clamav_check.sh

Now we should start the pure-uploadscript daemon to call our script every time a file is uploaded

/usr/sbin/pure-uploadscript -B -r /etc/pure-ftpd/clamav_check.sh

And to let it start with your server/vps reboots just run this :

echo "/usr/sbin/pure-uploadscript -B -r  /etc/pure-ftpd/clamav_check.sh" >> /etc/rc.d/rc.local

Now restart pure-ftpd :

service pure-ftpd restart

Now all your uploaded files which are less than 10MB in size will be scanned , adjust this size limit per your needs

Last year EXT4 was announced as stable release and it was merged in kernel 2.6.28 as part of the coder afterwards, it brings many great features am stating some here before getting on how to move an existing partition to ext4 without formatting

Large file system
The ext4 filesystem can support volumes with sizes up to 1 Exbibyte (1,152,921,504,606,846,976 bytes) and files with sizes up to 16 tebibytes (1,099,511,627,776 bytes).

Extents
Extents are introduced to replace the traditional block mapping scheme used by ext2/3 filesystems. An extent is a range of contiguous physical blocks, improving large file performance and reducing fragmentation. A single extent in ext4 can map up to 128MB of contiguous space with a 4KB block size. There can be 4 extents stored in the inode. When there are more than 4 extents to a file, the rest of the extents are indexed in an Htree.

Persistent pre-allocation
The ext4 filesystem allows for pre-allocation of on-disk space for a file. The current methodology for this on most file systems is to write the file full of 0s to reserve the space when the file is created. This method would no longer be required for ext4; instead, a new fallocate() system call was added to the Linux kernel for use by filesystems, including ext4 and XFS, that have this capability. The space allocated for files such as these would be guaranteed and would likely be contiguous. This has applications for media streaming and databases.

my favorite ;)
Journal checksumming
Ext4 uses checksums in the journal to improve reliability, since the journal is one of the most used files of the disk. This feature has a side benefit; it can safely avoid a disk I/O wait during the journaling process, improving performance slightly.

NOTE
Filesystem changes/hacks contains certain risk ,though we tested this on our system we don’t guarantee this will work perfectly on your systems and we shall not be responsible for any data loss or hard disk crash .
p.s. backup your system before proceeding .

Step 1 :
Assuming we are changing sda2 edit its line in /etc/fstab to look like this

/dev/sda2     /                ext4    defaults      1  1

Step 2 :
Make sure needed packages are up-to-date then update your initrd image

yum upgrade mkinitrd e2fsprogs
mv /boot/initrd-`uname -r`.img /boot/initrd-`uname -r`.img.old
mkinitrd -v --with=ext4 /boot/initrd-`uname -r`.img `uname -r`

Step 3 :
Boot using your OS CD-ROM into rescue mode and run this commands to apply the new FS extensions , make sure you’re working on the correct partition and you’re using

/mnt/sysimage/sbin/tune2fs -I 256 -O has_journal,extents,huge_file,flex_bg,uninit_bg,dir_nlink,extra_isize /dev/sda2

The last command should tell your to run fsck and reboot if it didn’t it has failed , and you should remove the tune2fs option it asks to be removed

Last step :

fsck -pf /dev/sda2

Port Knocking is an approach that helps protecting your services ports from attacks the most famous brute forced would be SSH , the port knocking method depends that the client trying to connect will first attempt to connect to a predefined ports which will enable connection to your secured service port for 5 seconds , here is a simple script to set this kind of protection

#!/bin/sh
#
# Netfilter/IPtables - example of multiple-port knocking
# Note: Knock ports 3456,2345,1234 to open SSH and MySQL ports for 5 seconds.
# Nice thing to knock TCP with is `nc' program:
# $> nc -w 1 <ip> 3456 ; nc -w 1 <ip> 2345 ; nc -w 1 <ip> 1234 ,ssh <ip>
#
# Change this to the name of the interface that provides your "uplink"
# (connection to the Internet) or connection you want to protect.
UPLINK="eth0"
#
# Comma seperated list of ports to protect with no spaces.
SERVICES="22,3306"
#
# Location of iptables command
IPTABLES='/sbin/iptables'
#
${IPTABLES} -N stage1
${IPTABLES} -A stage1 -m recent --remove --name knock
${IPTABLES} -A stage1 -p tcp --dport 3456 -m recent --set --name knock2

${IPTABLES} -N stage2
${IPTABLES} -A stage2 -m recent --remove --name knock2
${IPTABLES} -A stage2 -p tcp --dport 2345 -m recent --set --name heaven

${IPTABLES} -N door
${IPTABLES} -A door -m recent --rcheck --seconds 5 --name knock2 -j stage2
${IPTABLES} -A door -m recent --rcheck --seconds 5 --name knock -j stage1
${IPTABLES} -A door -p tcp --dport 1234 -m recent --set --name knock

${IPTABLES} -A INPUT -m --state ESTABLISHED,RELATED -j ACCEPT
${IPTABLES} -A INPUT -p tcp --match multiport --dport ${SERVICES}  -i ${UPLINK} -m recent --rcheck --seconds 5 --name heaven -j ACCEPT
${IPTABLES} -A INPUT -p tcp --syn -j door

Some known pitfalls of port knocking are :

1- using consecutive port numbers like 100,200,300 which would cause the secured service port to be opened while using a port scanner like Nmap.

2- port knocking is not effective against replay attacks

3- using port knocking as your sole line of defense, it has to be one of many .

Some other famous scripts used for port knocking are fwknop and knockd

A new exploit has been discovered in WordPress the famous blogging web application , the exploit resets the admin password without the need of the “Password Reset” email the detailed explanation is here , here is the exploited code :

function reset_password($key) {
    global $wpdb;

    $key = preg_replace('/[^a-z0-9]/i', '', $key);

    if ( empty( $key ) )
        return new WP_Error('invalid_key', __('Invalid key'));

    $user = $wpdb->get_row($wpdb->prepare("SELECT * FROM $wpdb->users WHERE user_activation_key = %s", $key));
    if ( empty( $user ) )
        return new WP_Error('invalid_key', __('Invalid key'));

Since the key is only checked for being an empty string then if you pass an empty array (which is a different type of variables) we will bypass this step and the password will be reset right away and sent to the admin listed e-mail .

In order to fix this problem edit wp-login.php with your favorite editor and change this lines :

Line 190 in WordPress 2.8.3 or line 169 in earlier 2.8 versions

if ( empty( $key ) )

TO

if ( empty( $key ) || is_array( $key ) )

This will add a check to invalidate passing an array to the $key variable .

Enjoy!

This are a set of commands that a sys admin could use to perform various tasks on a server , varying from listing server Ips to optimizing MySQL tables , hope it helps you managing different tasks fast :

!!:gs/foo/bar

Runs previous command replacing foo by bar every time that foo appears
Very useful for rerunning a long command changing some arguments globally.

ping -i 60 -a IP

Set audible alarm when an IP address comes online
Waiting for your server to finish rebooting? Issue the command above and you will hear a beep when it comes online. The -i 60 flag tells ping to wait for 60 seconds between ping, putting less strain on your system. Vary it according to your need. The -a flag tells ping to include an audible bell in the output when a package is received (that is, when your server comes online).

du -b --max-depth 1 | sort -nr | perl -pe 's{([0-9]+)}{sprintf "%.1f%s", $1>=2**30? ($1/2**30, "G"): $1>=2**20? ($1/2**20, "M"): $1>=2**10? ($1/2**10, "K"): ($1, "")}e'

Sort the size usage of current directory tree by gigabytes, kilobytes, megabytes, then bytes.

tr -dc A-Za-z0-9_ < /dev/urandom | head -c 16;echo

useful for generating passwords,
Find random strings within /dev/urandom. Using tr to use only Alphanumeric characters, and then print the first 16.

mkdir -p a/long/directory/path

This will create the intermediate directories that do not exist.

ctrl-t

Switch 2 characters on a command line.
If you typed ‘sl’, put the cursor on the ‘l’ and hit ctrl-t to get ‘ls’.

grep -i --color=auto

Highlights the search pattern in red.

!!

Repeat last executed command

DD=`cat /etc/my.cnf | sed "s/#.*//g;" | grep datadir | tr '=' ' ' | gawk '{print $2;}'` && ( cd $DD ; find . -mindepth 2 | grep -v db\.opt | sed 's/\.\///g; s/\....$//g; s/\//./;' | sort | uniq | tr '/' '.' | gawk '{print "CHECK TABLE","`"$1"`",";";}' )

Generate CHECK TABLE statements for all MySQL database tables on a server

export dbname=DB;for i in `mysql --batch --column-names=false -e "show tables" $dbname`;do  mysql -e "ALTER TABLE $i DEFAULT CHARACTER SET utf8 COLLATE utf8_unicode_ci" $dbname;done

Change all tables inside a database to UTF8 character set.

ifconfig | grep "inet [[:alpha:]]\+" | cut -d: -f2 | cut -d' ' -f1

Get the IP address of all your network cards.

Usually the kernel handles memory utilization pretty well it caches memory for dentry cache, page cache and inodes which improves IO speed and performance generally. But in some cases user applications needs lots of memory and we need to clear what’s called dirty memory which could be inodes already written to the disk, so now the kernel given us the option to manage this manually.

First of all Run sync command to flush all file system buffers

Then run one of this commands :

echo 1 > /proc/sys/vm/drop_caches

This will free only page caches

echo 2 > /proc/sys/vm/drop_caches

This will free inodes and dentry caches

echo 3 > /proc/sys/vm/drop_caches

This will free ALL

Normally when you build a custom kernel you use some config files available online which have everything enabled to assure everything works, but this elongates compilation time which also could make your kernel bigger than needed, finally Steven Rostedt has come up with a awesome solution for this problem .

He developed a perl script “”streamline_config” that you can download here it will make a custom config file that will still boot your box, but bring down the compile time of the kernel can be really long.

The script will perform “lsmod” to find all the modules loaded on the current running system. It will read all the Makefiles to map which CONFIG enables a module. It will read the Kconfig files to find the dependencies and selects that may be needed to support a CONFIG. Finally, it reads the .config file and removes any module “=m” that is not needed to enable the currently loaded modules. The output goes to standard out.

Here is Steven’s thread about the script

Steps to run the script :

1- Boot up the kernel that you want to streamline the config on.
2- Change directory to the directory holding the source of the kernel that you just booted.
Then run this commands :

./streamline_config.pl arch/x86/Kconfig > new_config
mv .config config_old
mv new_config .config
make oldconfig

if you wanna modify anything you can run make menuconfig instead of make oldconfig

I know this has been considered impossible for quiet a long time but its now possible – actually for quiet awhile now- thanks to foremost software, it can recover files based on their headers, footers, and internal data structures. This process is known as data carving.

Foremost can work on image files, such as those generated by dd, Safeback, Encase, etc, or directly on a drive. The headers and footers can be specified by a configuration file or you can use command line switches to specify built-in file types. These built-in types look at the data structures of a given file format allowing for a more reliable and faster recovery.

Foremost can recover files with the following extensions:
jpg – Support for the JFIF and Exif formats including implementations used in modern digital cameras.
gif
png
bmp – Support for windows bmp format.
avi
exe – Support for Windows PE binaries, will extract DLL and EXE files along with their compile times.
mpg – Support for most MPEG files (must begin with 0x000001BA)
wav
riff – This will extract AVI and RIFF since they use the same file format (RIFF). note faster than running each separately.
wmv – Note may also extract -wma files as they have similar format.
mov
pdf
ole – This will grab any file using the OLE file structure. This includes PowerPoint, Word, Excel, Access, and StarWriter
doc – Note it is more efficient to run OLE as you get more bang for your buck. If you wish to ignore all other ole files then use this.
zip – Note is will extract .jar files as well because they use a similar format. Open Office docs are just zipped XML files so they are extracted as well. These include SXW, SXC, SXI, and SX? for undetermined OpenOffice files.
rar
htm
cpp – C source code detection, note this is primitive and may generate documents other than C code.
You can tweak /etc/foremost.conf to add support for more file types.

Please note that there’s no guarantee that foremost will succeed in recovering your files, but at least there’s a chance.

On Debian and Ubuntu, foremost can be installed simply by issuing this command :

apt-get install foremost

or download and extract the source from the tar.gz and run this inside the extracted dir :

make;make install

So easy , now you’re ready to start using foremost here are some example usage:

foremost -t jpeg -i /dev/sda1

in this example we are looking for all deleted jpg files on /dev/sda1 , foremost always output the results in directory called output, its created in the directory where you ran foremost so be sure to run it from a directory not located in /dev/sda1 to avoid overwriting one of your deleted files ;)

Inside the output directory you will find a file named audit.txt which the log of this restoration process and also a directory named jpg cuz we were searching for jpegs in our last example

foremost -t pdf -T -i /dev/sda1

In this example we are searching for pdf files and appending results to the output directory (because foremost will not start if there’s already an output directory)

foremost -s 100 -t jpg -i image.dd

Here we are searching for jpeg files skipping the first 100 blocks inside this dd image

foremost -t all -i /dev/sda1

Searching for all predefined types

We do appreciate and respect all our customers, this post is just for the sake of laughing :)

Lie. Seriously, people will do this, even when you can easily prove it, and when they know that you can. I have a lot more respect for someone that will say I screwed things up than for someone who pretends he has done nothing wrong at all.

Rant. Seriously, I don’t give a heck how many other people you’ve talked to who don’t have this problem, how much am paying for the service, or what kind of problems you’ve had in the past. That does NOTHING WHATSOEVER to help me solve your immediate problem.

Do things to your computer, while they’re on the phone with you, that they didn’t tell you to. They’re usually going through a mental checklist of things that could be wrong. If you could’ve done this by yourself, then why did you call in the first place?!

Ask them very open-ended questions.

Call and ask ‘Is the server down?’ They love that.

When something doesn’t work, randomly change configuration options until it does.

Claim to be knowledgable due to some bloated certification. Claim that you could fix the problem faster.

Use Windows ICS. Call your webhosting provider about problems that are obviously related to your internal network configuration. Tell them they should help you.

Email support and complain that you can’t send mail.

Get annoyed and huffy when they assume you’re a moron.

Relate to them how your company would handle a problem of this nature with one of it’s clients.

Tell them exactly how your day is going Include amusing anecdotes.

Be as nonspecific as possible.