<SourceBench/> Your place to share code stuff » Bash/shell

Simple parallel processing with Bash using “wait” and “jobs”

The SourceBench — Mon, 01 Feb 2010 13:53:14 +0000

#!/bin/bash
# this is a simple helper script that can be used when you are in need of running multiple processes at once
# simply run your command with an ampersand at the end and after this
# execute forky with the number of parallel processes as argument.

function forky() {
	local num_par_procs
	if [[ -z $1 ]] ; then
		num_par_procs=3
	else
		num_par_procs=$1
	fi

	while [[ $(jobs | wc -l) -ge $num_par_procs ]] ; do
		sleep 1
	done
}

# below is an example - make sure to change this to your needs
for fname in `ls -1`; do
	echo "`jobs | wc -l` jobs in spool"
	echo -e "processing $fname\n"
	# below should be the command you like to execute. in this example it's a php script that would do something with a file.
	# important is the & at the end of the command which sends the command to the background
	php ~/a_script_that_does_something_with_the_file --file=$fname &
	# after this call the forky function with the amount of parallel processes as parameter
	forky 5
done

wait

Test if a nameserver change is done right

The SourceBench — Tue, 19 Jan 2010 23:52:28 +0000

# imagine you just updated your whois data so it will point your domain to an other nameserver and you want to know if that worked out
# this little command lets you know on *nix based machines
for i in a b c d e f g h i j k l; do dig +short ns domainname.com @$i.GTLD-SERVERS.NET; done

command line real-time view of what files Time Machine is copying

The SourceBench — Wed, 09 Dec 2009 09:15:51 +0000

sudo fs_usage -w -f filesys backupd | grep open

Print the number of established and listening connections to your server on port 80 per each IP address

The SourceBench — Sun, 06 Dec 2009 23:49:27 +0000

netstat -plan|grep :80|grep -E "(EST|LIST)"|awk {'print $5'}|cut -d: -f 1|sort|uniq -c|sort -nk 1

Get a list of urls/domains from a text file

The SourceBench — Fri, 04 Dec 2009 23:05:54 +0000

Start an application from the command line in OSX

The SourceBench — Thu, 03 Dec 2009 22:03:57 +0000

# use open -a followed by the program name and optionally a file name for an output log file
open -a /Applications/TextEdit.app logfile.txt

Simple Socks5 Proxy connection with SSH

The SourceBench — Thu, 03 Dec 2009 21:49:10 +0000

# Sometimes you want to get a simple proxy to fake your IP or encrypt your connection
# in case you are surfing in an insecure space. All you need is a server which you can access via ssh.
# then use the following command to open a ssh proxy connection with this command
ssh -ND 9999 your_user@your_ssh_remote_host
# after this you will have SOCKS5 server on localhost port 9999 which you can enter as your browser proxy

Download (mirror) an entire website with wget

The SourceBench — Fri, 27 Nov 2009 23:30:10 +0000

wget \
     --recursive \
     --no-clobber \
     --page-requisites \
     --html-extension \
     --convert-links \
     --restrict-file-names=windows \
     --domains website.org \
     --no-parent \
         http://www.website.org/tutorials/html/
# found at http://www.linuxjournal.com/content/downloading-entire-web-site-wget

Search for a function definition in a bunch of php files

The SourceBench — Fri, 27 Nov 2009 13:40:08 +0000

find . -name *.php -exec grep -HE "function FUNCTION_NAME" {} \;

Create a self signed SSL Certificate (server.key/pem/csr/crt file)

The SourceBench — Fri, 27 Nov 2009 03:36:45 +0000

#The following commands will generate you server.key/pem/csr/crt files, needed for working with apache/ssl on port 443. You can replace -days #365 with any value you want.
openssl genrsa -des3 -rand file1:file2:file3:file4:file5 -out server.key 1024
openssl rsa -in server.key -out server.pem
openssl req -new -key server.key -out server.csr
openssl x509 -req -days 365 -in server.csr -signkey server.key -out server.crt
#If you plan using valid SSL certificate, signed by thawte for example, then when entering data for creating .csr file, you have to enter valid data, #and then send this file to them. They will return the .crt file.