Archive for the ‘Tools’ Category

Pseudothreading with BASH

January 20, 2010

It’s more like a trick, but it’s better than repeat the same operation linearly. Some explanations:

TH_NUM=`ps aux | grep Python | grep -v "grep" | wc -l`

  • TH_MAX is the maximum number of “threads” that can be executed at the same time.
  • The first grep selects the threads that make use of python (you can change this, it depends on your script)
  • The second grep excludes the command you issued above 😉
  • wc counts the number of lines. The first time the result of the pipe is empty, so wc gives “0” as result.

#!/bin/bash
TH_MAX=10
for sample in `ls ./data`
do
while [ TRUE ]; do
TH_NUM=`ps aux | grep Python | grep -v "grep" | wc -l`
if [ "$TH_NUM" -le "$TH_MAX" ]
then
echo $( ./analyze_sample.py -s ${sample} ) > /dev/null &
echo -en " ${sample} "
break
else
echo -en "."
sleep 1
fi
done
done

sed one-liners

March 24, 2009

Handy one-liners for UNIX stream editor sed.

Undelete with grep

July 3, 2008

Earlier this afternoon, I did something really really really really really^(infinity) careless and stupid.
I was working on a bash script containing various commands to automate some things that had taken me days and days to learn and get my head round. The script was outputting its results in a directory, which i was continuously deleting, as I repeatedly (re)ran the script to debug and improve stuff.

Cue moment of complete madness:
instead of typing rm -rf [output_dir]
i typed rm -rf [script_filename]

…and then it was gone.

Quick… backups…
Oh… I forgot to back it up or take a copy before I started editing in improvements didn’t I

big sigh…. this is unix… no easy undelete available (at least not that I know of)
now what… ? This would take me days and days to reproduce.

some frantic googling later… this linuxforums thread came to the rescue!

Now I didn’t actually unmount the partition because I had an active session with lots of other stuff i needed not the best idea… always unmount while doing this

– STOP everything and DO NOT edit anything more (to prevent the diskspace being overwritten)
– Unmount the partition
– run the following command:
grep --binary-files=text -500 «search_string» /dev/sda4 > output.txt

where:
– 500 is how many lines before and after the seatch_string to display
– search_string is a string contained in the file to be recovered
– /dev/sda4 is the partition from which the file was deleted

This reads through the whole partition, prints the number of lines before/after the string pattern into the file output.txt

Then you can search through the output.txt file and hopefully find a block of text that corresponds to the content of the file…. ready for copying/pasting/saving into a new file and immediate backing up onto every partition on the machine…. onto every USB memory stick you can find lying around your desk and in your pockets…. via ssh to every remote machine you own or have an account on… … … etc.

There are also various tools available like magicrescue (on Ubuntu sudo apt-get install magicrescue), but in this instance it didnt work for me – probably coz I didn’t create the recipe file correctly or didn’t know how to run it properly.

But good old grep was enough.

Open-Source Anti-Virus

May 12, 2008

http://www.clamwin.com/

Hidden default config settings in Firefox

May 1, 2008

You can change various default settings within firefox through the config page by typing the following URL into the Location field:

about:config

Do you want to put a black/white/both frame around your picture?

April 7, 2008

I was tired of looking at complicated (ehm, not free) applications to do this, so I wrote my own 🙂

Enjoy


#!/usr/bin/env python

from PIL import Image
import sys

if len(sys.argv)!=3 and len(sys.argv)!=4:
    print "USAGE: <command> <filename> <black border percentage> <white border percentage>"
    sys.exit(0)
    
filename=sys.argv[1]
percBlack=float(sys.argv[2])

if len(sys.argv)==3:
    percWhite=0
else:
    percWhite=float(sys.argv[3])
    

im = Image.open(filename)
blackWidth=int(im.size[0]*percBlack)
whiteWidth=int(im.size[0]*percWhite)
white = Image.new('RGB',(im.size[0]+whiteWidth,im.size[1]+whiteWidth),'white')
black = Image.new('RGB',(white.size[0]+blackWidth,white.size[1]+blackWidth),'black')
white.paste(im,(int(whiteWidth/2),int(whiteWidth/2)))
black.paste(white,(int(blackWidth/2),int(blackWidth/2)))
black.save('framed-'+filename)

Command-line construction of auguments list with xargs

January 23, 2008

xargs is a really useful unix command line tool that can be used to construct an argument list for piping into other unix commands.
For example, lets say we want to find and delete all files fitting a certain criteria, such as those with a suffix of “.java~”.
We can use the find utility to easily list these:


$ find . -name *.java~
./path/to/file/file1.java~
./path/to/file/file2.java~
./path/to/file/file3.java~
./path/to/file/file4.java~

However, the output from this is in the form of a list, which cannot be piped directly into a command like rm. If the list is very long, it can be difficult to write it out manually into a singe argument list.

Instead, we can pass the output from the find command into xargs to convert this list into a single argument list for us:


$ find . -name *.java~ | xargs
./path/to/file/file1.java~ ./path/to/file/file2.java~ ./path/to/file/file3.java~ ./path/to/file/file4.java~

This can then be piped into rm or any other command as needed.

This is just one simple example of the use of xargs. It’s a much more powerful tool that is able to handle much larger argument lists with great flexibility.

How to use GAIM with G-Talk

December 16, 2007

This quick tutorial is copied from www.tech-recipes.com.

1. Open GAIM
2. From the buddy list screen, click Tools
3. Then click Accounts
4. Click the Add button
5. Select Jabber as the protocol
6. Your screen name is your gmail login. (For example, login@gmail.com)
7. Server is gmail.com
8. Resource is Gaim
9. Your password is your gmail password.
10. Select as alias what you want to be called on your IM screen. If you leave this blank, it’ll use your whole gmail address which is way too big for your little IM screen. If nothing else, just put in Me in the alias box.
11. Click Show More Options
12. Select the Use TLS if available checkbox
13. As you connect server type: talk.google.com
14. Click Save

MD5 SHA1 Reverse Lookup

November 29, 2007

If you have an MD5 or SHA-1 hash and you need the value that created the hash, try to look it up here. They also provide a hashing function and will store in their database, whatever you hash.