sample mongo shell update call

db.item.update({cluster:"nieuws"},{$set:{last_crawl_date:new Date(2011,6,24)}}, false, true);

xmlstartlet

example for printing

xmlstarlet sel -t -m "//category" -v "text()" -n filename.xml | sort | uniq

subversion ignore porperty

svn propedit svn:ignore ./some_path

^M and vim

To replace the DOS newline character with a unix/linux newline:

:%s/^M/\r/g

Remember, the ^M is [ctrl-v + ctrl-m] and not [shift-6].

(source: http://bigredradio.com/index.php?option=com_content&view=article&id=49:newline&Itemid=51)

cross host diff

use ndiff

'find' and 'rm'

remove vim .swp files with find, xargs and rm

find ./ -name ".*.swp" -print0 | xargs -0 rm -f

vim reminders

folding:set foldmethod=indent (marker works as well)

tail like functionality: setlocal autoread

two mysql reminders

full text searches
http://dev.mysql.com/doc/refman/5.0/en/fulltext-search.html

REPLACE syntax
http://dev.mysql.com/doc/refman/5.0/en/replace.html

convert tweets into speech bash one-liner

This works on linux but the festival speech output is not very clear. You will need curl, xmlstarlet and festival binaries for this to work.

$curl "http://api.twitter.com/1/statuses/user_timeline.xml?count=1&screen_name=barackobama" | xmlstarlet sel -t -m "//status" -v "text" | festival --pipe --tts


Should work on OS X as well with the "say" command and some grep and sed magic:
$curl "http://api.twitter.com/1/statuses/user_timeline.xml?count=1&screen_name=barackobama" | egrep -w "(.*)" | sed -E "s/<\/?text>//g" | say

tar directly to email

tar cvjf - folder/ | mailx email@address.com

tar to another host over ssh

tar zcvf - /home | ssh rhost "cat > /dev/tape"

occurences of a line in a file, counting words

uniq -c is especially handy in combination with cut to get parts of csv files

nice conkyrc

background yes
use_xft yes
xftfont Sans:size=8
xftalpha 1
update_interval 1.0
total_run_times 0
own_window yes
own_window_transparent yes
own_window_type desktop
own_window_hints undecorated,below,sticky,skip_taskbar,skip_pager
double_buffer yes
minimum_size 200 200
maximum_width 200
draw_shades yes
draw_outline no
draw_borders no
draw_graph_borders yes
default_color white
default_shade_color black
default_outline_color white
double_buffer yes
alignment top_right
gap_x 12
gap_y 12
no_buffers yes
uppercase no
cpu_avg_samples 2
override_utf8_locale no

TEXT
${font sans-serif:bold:size=8}SYSTEM ${hr 2}
${font sans-serif:normal:size=8}$sysname $kernel $alignr $machine
Host:$alignr$nodename
Uptime:$alignr$uptime
File System: $alignr${fs_type}

${font sans-serif:bold:size=8}PROCESSORS ${hr 2}
${font sans-serif:normal:size=8}${cpugraph cpu1}
CPU1: ${cpu cpu1}% ${cpubar cpu1}

${font sans-serif:normal:size=8}${cpugraph cpu2}
CPU1: ${cpu cpu2}% ${cpubar cpu2}


${font sans-serif:bold:size=8}MEMORY ${hr 2}
${font sans-serif:normal:size=8}RAM $alignc $mem / $memmax $alignr $memperc%
$membar

${font sans-serif:bold:size=8}DISKS ${hr 2}
${font sans-serif:normal:size=8}/ $alignc ${fs_used /} / ${fs_size /} $alignr ${fs_used_perc /}%
${fs_bar /}
SWAP $alignc ${swap} / ${swapmax} $alignr ${swapperc}%
${swapbar}

${font sans-serif:bold:size=8}TOP PROCESSES ${hr 2}
${font sans-serif:normal:size=8}${top_mem name 1}${alignr}${top mem 1} %
${top_mem name 2}${alignr}${top mem 2} %
$font${top_mem name 3}${alignr}${top mem 3} %
$font${top_mem name 4}${alignr}${top mem 4} %
$font${top_mem name 5}${alignr}${top mem 5} %

${font sans-serif:bold:size=8}NETWORK ${hr 2}
${font sans-serif:normal:size=8}IP address: $alignr ${addr eth0}
${downspeedgraph eth0}
DLS:${downspeed eth0} kb/s $alignr total: ${totaldown eth0}
${upspeedgraph eth0}
ULS:${upspeed eth0} kb/s $alignr total: ${totalup eth0}

bulk replace with sed

just one of the many ways:

grep -l fromstring `find . -name "*.ext"` | while read i; do sed -i 's/fromstring/tostring/g' $i; done;

wordcount for a site / nice for translators

after wget -r http://yourdomain.com

find | egrep -v "^.*(\.swf|\.css|\.js|\.gif|\.jpeg|\.jpg|\.png|\.ico)$" | while read i; do echo "$i |" `lynx -dump -nolist $i | wc -w`; done > /tmp/wordcount.csv

when yum hangs try this

rm /var/lib/rpm/__db.00*
rpm --rebuilddb

default vimrc settings part III

syntax on
filetype on
set ignorecase
set smartcase
set tabstop=4
set softtabstop=4
set shiftwidth=4
set expandtab
set autoindent
set smartindent
set nowrap
set ttyfast
set number
set hlsearch
set showcmd
set incsearch


colorscheme desert

autocmd FileType python set omnifunc=pythoncomplete#Complete
autocmd FileType javascript set omnifunc=javascriptcomplete#CompleteJS
autocmd FileType html set omnifunc=htmlcomplete#CompleteTags
autocmd FileType css set omnifunc=csscomplete#CompleteCSS
autocmd FileType xml set omnifunc=xmlcomplete#CompleteTags
autocmd FileType php set omnifunc=phpcomplete#CompletePHP
autocmd FileType c set omnifunc=ccomplete#CompleteCpp

autocmd FileType php let php_sql_query=1
autocmd FileType php let php_htmlInStrings=1
autocmd FileType php let php_noShortTags=1


inoremap =InsertTabWrapper()

function! InsertTabWrapper()
let col = col('.') - 1
if !col || getline('.')[col - 1] !~ '\k'
return "\"
else
return "\"
endif
endfunction

fav vimrc settings part ||

syntax on
filetype on
set ignorecase
set smartcase
set tabstop=4
set autoindent
set smartindent
set nowrap
set ttyfast
set number
set hlsearch
set showcmd
set incsearch


colorscheme desert

autocmd FileType python set omnifunc=pythoncomplete#Complete
autocmd FileType javascript set omnifunc=javascriptcomplete#CompleteJS
autocmd FileType html set omnifunc=htmlcomplete#CompleteTags
autocmd FileType css set omnifunc=csscomplete#CompleteCSS
autocmd FileType xml set omnifunc=xmlcomplete#CompleteTags
autocmd FileType php set omnifunc=phpcomplete#CompletePHP
autocmd FileType c set omnifunc=ccomplete#CompleteCpp

autocmd FileType php let php_sql_query=1
autocmd FileType php let php_htmlInStrings=1
autocmd FileType php let php_noShortTags=1


inoremap =InsertTabWrapper()

function! InsertTabWrapper()
let col = col('.') - 1
if !col || getline('.')[col - 1] !~ '\k'
return "\"
else
return "\"
endif
endfunction

linux hardware info

use dmidecode

lshw is nice as well

starting redmine 'stand alone'

ruby script/server webrick -e production


also see: http://www.redmine.org/wiki/redmine/RedmineInstall

mysql C API note

when using mysql_use_result () mysql_num_rows () does not return the correct value until all rows have been fetched. To do this, use mysql_store_result instead.

classic (My?)SQL mistake

Dunno about other databases.

SELECT * FROM my_table WHERE date > '2009-11-17'



But what you intend to do is actually
SELECT * FROM my_table WHERE `date` > '2009-11-17'

Of course. If you have an editor with SQL highlighting this would probably not happen.

sample config for apache mod_ldap

This works:

global stuff

LDAPTrustedGlobalCert CA_BASE64 /etc/ssl/certs/ca.pem
LDAPVerifyServerCert Off



Directory things




AuthType Basic
AuthBasicAuthoritative Off
AuthUserFile /dev/null
AuthBasicProvider ldap

AuthName "my authfolder"

Options Indexes FollowSymLinks

IndexIgnore .*
AllowOverride None
order allow,deny
allow from all


AuthzLDAPAuthoritative On

AuthLDAPURL ldaps://ldap.server.com:636/ou=my-unit,dc=domain,dc=com?uid?sub?(ObjectClass=*)

require valid-user

DAV On

python working dir fix for cronjobs

I fyou have to run a python script from a cronjob and imports aren't found add this to the top of the script




import os, sys

os.chdir(os.path.dirname(sys.argv[0]))

rsync multiple directories

/usr/bin/rsync -avu /srv /var /etc /home /usr/local /root /media/backup

nice shells

bash
scponly
true
false

favorite apps for mac

macVim (nice gvim like editor)
Transmission (torrents)
Skype (chat and call)
VLC (good replacement for quicktime)
XQuartz (xorg server)
NeoOffice (openoffice in mac style)
MacPorts
Cyberduck (ftp client)
Xmeeting (SIP, make calls etc.)
Thunderbird and Firefox
Netbeans

svn mv html to php rename bash

for i in $(ls *.html); do svn mv $i `echo $i | sed s/html/php/`; done;

rsync over another ssh port

if you want to rsync over port 7777 instead of 22 use

rsync -vnze 'ssh -p 7777' user@myserver.fake:/src/dir/* /dest/dir

-n is a dry run!

bulk rename bash

ls *old|sed 's/\(.*\)\.old/mv \1.old \1.new/'|sh

from: http://www.moxleystratton.com/articles/rename-files-bash

one line loop bash

check php files for errors

bash$ for f in $( ls *.php ); do php -l $f; done

pecl / pear gives error: phpize not found

solution: apt-get install php5-dev

RSSH scp sftp rsync restrictive shell

rssh is a restricted shell, used as a login shell, that allows users to
perform only scp, sftp, cvs, rdist, and/or rsync operations. It can
also optionally chroot user logins into a restricted jail.

ssh port forwarding

ssh port forwarding example (port 8000)

ssh -C -N -L 8000:local_ip:8000 remote_ip


with compression.

javascript nice urls

javascript nice urls

replaces spaces punctuation and some other crap.

str.replace ( /(_|\t| )+/g, "-" ).replace(/[^a-z0-9\-]|(^-)|(-$)/gi,"").toLowerCase ().replace ( /-+/g,"-");

php curl twitter api

If you receive a 417 http status code when you try to post something using php and curl. Try to send an empty "Expect:" header with your request.

example:

$curl_conn = curl_init ( "http://twitter.com/statuses/update.xml" );
curl_setopt($curl_conn, CURLOPT_HTTPAUTH, CURLAUTH_BASIC );
curl_setopt ($curl_conn, CURLOPT_HTTPHEADER, array ( "Expect:" ) );
curl_setopt($curl_conn, CURLOPT_USERPWD, $user . ":" . $password );
curl_setopt($curl_conn, CURLOPT_POST, true );
curl_setopt($curl_conn, CURLOPT_RETURNTRANSFER, 1 );
curl_setopt ( $curl_conn, CURLOPT_POSTFIELDS, array ( "status" => $msg ) );
$response = curl_exec ( $curl_conn );

mysql dumps

from:
http://mrothouse.wordpress.com/2007/01/15/mysql-restore-from-compressed-backup/

Copied a compressed database backup from the Production server to the Development server. The database backup file was compressed using bzip2 while piping from mysqldump.

mysqldump -u root -p –databases perfparse | bzip2 -z1 > perfparse.bkp.bz2

bzip2 Options

z = compression
1 = fastest compression (compared to 9 = best compression)

Used bunzip2 to uncompress while piping to mysql due to limited disk space.

bunzip2 < perfparse.bkp.bz2 | mysql -uroot -p

Very basic information, but I do use this blog for referring myself and others back to my notes.

bulk rename bash

for f in oldprefix*; do mv "$f" "${f/oldprefix/newprefix}"; done

simple python mail script

got it from here: http://www.eskimo.com/~jet/python/examples/mail/smtp_email.html

import smtplib
import email.Message

def mail(serverURL=None, sender='', to='', subject='', text=''):
message = email.Message.Message()
message["To"] = to
message["From"] = sender
message["Subject"] = subject
message.set_payload(text)
mailServer = smtplib.SMTP(serverURL)
mailServer.sendmail(sender, to, message.as_string())
mailServer.quit()

Leopard samba client

Some Mac leopard clients ignore the force create mode settings from your smb.conf which work with most other clients.

If you want stuff to work with those clients try this (user+group can read, write all, others cannot do anything):


create mask = 0660
force create mode = 0660
security mask = 0000
force security mode = 0660

directory mask = 0770
force directory mode = 0770
directory security mask = 0000
force directory security mode = 0770


http://discussions.apple.com/message.jspa?messageID=5904901

php4-cgi with php5 apache2 module

Ran in a situation where I had to run php4 and 5 on the same box (but different folders).

On debian you'll have to install php4-cgi and enable apache mods (suexec, ssl, actions, mime) following VritualHost config worked for me:

<VirtualHost *:80 >

ServerAdmin bart@dlma.nl
DocumentRoot /var/www/tutos/HTML/
ServerName tutos.dlma.nl


ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/
<Directory /var/www/tutos/HTML>
AllowOverride All
Options -Indexes

AddHandler php-script .php
Action php-script /cgi-bin/php4


</Directory>

</VirtualHost>

switch user svn+ssh

If you want to change the username of your svn+ssh working copy (and the username is in the url i.e. svn+ssh://username@host.nl/repos ) the '--username' will not work. Instead you'll have to change the URL with the 'switch --relocate' command as follows:

$svn switch --relocate svn+ssh://username@host.nl/repos svn+ssh://newusername@host.nl/repos

list scsi devices linux

The 'lsscsi' command lists the scsi drives attached to your system.

ubuntu (aptitude install lsscsi)

Also checkout the file
/proc/scsi/scsi


and the command hwinfo --scsi

trac Integrety Error

If your trac starts giving you Intgrety Errors try to do a:

trac-admin /your/trac/path resync

loop throught files with bash

For-in loops don't work well when you have filenames with spaces in them

#!/bin/bash
ls *.pdf | while read i; do
j=`echo $i | sed s/\.pdf/\.swf/`
pdf2swf "$i" -o "$j"
done

ssh X forwarding on your server

If you have an ubuntu/debian server running and want to run X apps through ssh -X you should "apt-get install xauth" for this to work.

tortoise SVN

In some cases tortoise svn (windows svn client) can't do checkouts of your repository and the following message is displayed:

"Not authorized to open root of edit operation"

Adding the following line to your svnserve.conf solved the problem for me:

"anon-access=none"

Hollandse hitmachines

Voor al uw nederlandse artiesten en hollandse hitmachines

http://www.watzitjenoutekijken.nl

convert int to a separate bytes

def intToBytes(self,n, bytecount):
""" returns the binary of integer n, using count number of digits """
str = ""

i=bytecount-1
#8 bit mask
mask = ~(sys.maxint^(pow(2,8)-1))
while i >= 0:
str = str + chr((n>>(i*8))&mask)
i-=1

return str

convert bytes to an integer in python

ran into a situation where i had to convert some bytes to an integer value in python. This func does the trick. The mask is only used to have a counter of the same length counting backwards (hence the negation).
def bytesToInt ( self, bytes ):
""" convert certain amount of bytes to an integer """
size=0
i=0

mask = sys.maxint&(len(bytes)-1)
while i < len(bytes):
size = size|ord(bytes[i])<<((~i)&mask)*8
i+=1

return size

rolling back with svn

Say you want to rollback (parts of) your svn repository to a certain revision. This can be done with svn merge.

it works like this:

$ svn merge -r 1481:1373 svn+ssh://your.host.com/srv/repos/subdir

where the first number is the current revision and the second number is the revision you want to 'rollback' to.

A list with changes should appear (also use svn status for this). If you think you're done do a commit and the changes will be done on the repository.

tcpdump

for ftp traffic
tcpdump -A -i eth0 tcp port 21

comes in handy when you want to see what passwords are used.

WEP cracking with aircrack-ng

Open three terminals

terminal 1:
$airodump-ng -c 4 --bssid FA:KE:FA:KE:FA:KE -i -w jah eth1

This is an exmaple that dumps only weak iv's (channel 4) interface eth1 from the AP with this particular BSSID

You can also do a scan for WEP AP's first by running "airodump-ng eth1" without any further args.

If you're seeing some traffic between the AP and the client (use bssid for reference). use aireplay like this (in the second terminal):

$aireplay-ng -3 -b FA:KE:FA:KE:FA:KE -h FA:KE:FA:KE:FA:KE eth1

the three means you're doing ARP. The -b is the AP. The -h is the client.

If you leave this running for some time you can use the third terminal to run aircrack on the .ivs file airodump-ng created. It'll try to crack the key every now and then so you can leave it running.

filenames beginning with a - (dash)

If you want to (for example) cd to a file starting with a dash (some mac users tend to use weird filenames like "-folder" ). do

$cd ./-folder

removing files matching a certain pattern

from the xargs man:

delete all .svn folder

$find /somedir -name .svn -print0 | xargs -0 /bin/rm -rf

xargs comes in handy when handling great amounts of files.

replacing text in a bunch of files matching a certain pattern

Replace comes with mysql. It's a handy tool for sed/awk n00bs. This one I used to change some static paths while excluding all .svn folders and logfiles.

~$ replace '/var/www/mydir' '/var/www/newdir/' -- `grep --exclude-dir=.svn --exclude=*.log -clr "/var/www/mydir" ./*`

Of course there are other/better ways to accomplish this, look at this
http://www.brunolinux.com/02-The_Terminal/Find_and%20Replace_with_Sed.html

ejecting a cd from a mac (running OS X)

Before rebooting try in terminal:

$drutil eject

page up / page down bash

add these lines to your /etc/inputrc to search backwards through history. It searches on the string already typef

"\e[5~": history-search-backward
"\e[6~": history-search-forward

example post commit hook

Sample script for a conditional post-commit hook in bash. This post-commit hook updates a local copy (www directory) to the latest revision when files are changed within a certain directory.

#!/bin/bash
umask 0002

REPOS=$1
REV=$2
MATCH=`/usr/bin/svnlook -r $REV dirs-changed $REPOS | /bin/grep -c "just a string"`

if [ $MATCH -gt 0 ];then
/usr/bin/svn update /var/www/projects/mysite
fi

Never create a PagesController in Cake

While fiddling with cake. I got some weird error in my PagesController:

You are seeing this error because the action display is not defined in controller PagesController"

Whatever I did, it kept asking for a default view. After some searching it seems the PagesController is reserved for cake itself.


Reference: http://logs.cakephp.nu/cakephp/chat.log.2007-06-26