Removing leading whitespace from file

Sometimes clients like to annoy me and send IPs in a "fancy" format with tabs and space and crap..ugh. I'm a one entry per line kind of guy. So I paste the entries into vim and I want to get rid of leading whitespace. This is the command in vim:

:%s/^\s*//g

EXPLAIN THIS BULLSHIT:
vim has sed-like regex recognition. This command tells vim to search (%s) for lines begining (^) with whitespace (\s), and include the rest of the whitespaces directly after the first one (*). Next, delete them (//). Now do this globally (g). By default, it will only match the first instance of the regex, g means "do this globally".

Add Timestamps to bash history

Found the solution here:
http://linux.byexamples.com/archives/467/list-command-line-history-with-timestamp/

TL;DR
add:
export HISTTIMEFORMAT="%F %T "
to your .bash_profile

Ghetto-ized IP address range generator

Once i got my head out of my ass, I figured out an uber easy way to generate and sort a long list of IP addresses:


echo -e "\n"192.168.{1..255}.{1..255} #generates the actual IPs using Bash brace expansion
sort -t . -k 1,1n -k 2,2n -k 3,3n -k 4,4n #sorts the IPs into the proper order (optional)
>> hosts # output to file!

end result:
echo -e "\n"192.168.{1..255}.{1..255} | sort -t . -k 1,1n -k 2,2n -k 3,3n -k 4,4n >> hosts


The only thing you would need to edit would be the original echo line to match the range you are trying to generate.

SSH and GatewayPorts configuration

I love SSH, like alot, but I was having a problem with lately. Specifically the problem was with the security of remote forwarded ports (-R). I was told that by default SSH only allows remote ports to be bound to the local interface for security reasons. I understand that, completely. The problem is that wanted to be able to use my VPS's SSH connections like a bouncer. I wanted to be able to type:

ssh account@vpssserver.net -p2222

and have the connection go through vpsserver into the server behind it. In effect, making vpsserver a type of central hub for reverse ssh connections. You can think of it almost like a ghetto botnet.

I found out that all i had to do was change GatewayPorts to "yes" in /etc/ssh/sshd_config and it would work if I issued the following command on the BACK server.

ssh account@vpsserver.net -R 2222:localhost:22

All was well with the world for a while. Then my paranoia was sinking in. I didnt want someone to portscan my vps and see that i have 20 different ports open from reverse ssh connections. What was I to do? Well it turns out that GatewayPorts has 3 different settings; yes, no, and clientspecified.

no(default) = force remote port forwardings to only be accessible to localhost
yes = Force remote port forwardings to public interface (technically no, but in essence thats what it does)
clientspecified = the client decides which to choose

So i changed GatewayPorts to clientspecified and experimented. If you typed the remote forward command we typed in earlier:
ssh account@vpsserver.net -R 2222:localhost:22
we would get a port remotely bound to the vpsserver's localhost address. This would force you to first log into the VPS and then log into the 2222 on localhost.

BUT, if you want the port to be bound publicly on vpsserver, it only takes 1 more character. pay attention closely:
ssh account@vpsserver.net -R :2222:localhost:22
notice that ":" in from of the 2222? that essentially tells ssh to bind it to the public interface*.

Now i have two very similar commands to do two importantly different things. I am a happy camper.


*technically it tells SSH to bind it to all interfaces, which consequently includes the external facing one :)

vi & vim acting retarded

Ever just fire up a fresh VM or log into a box and try to edit a file with vi and the backspace/arrow keys are acting retarded? It happened it me, I noticed it was because there was no .vimrc file.

Simply copying the default one over to my dir fixed my problem:

cp /etc/vim/vimrc ~/.vimrc


Now vi & vim aren't so retarded...

Use Pandora ? you'll want Pianobar

I love Pandora.com, it provides me the majority of my new music selections/artists. The problem is that pandora has a few usability issues (with good reason). Pianobar is a command line tool that will login to pandora, grab your playlists, and play any one you want. And it has a feature that makes it all worth it- infinite skips.

Here is the install site:
https://github.com/PromyLOPh/pianobar

Install it, I have it on every machine that I listen on, its awesome.

The following is the output of the menu command that outlines a lot of what the prog does;

+ love song
- ban song
a add music to station
c create new station
d delete station
e explain why this song is played
g add genre station
h song history
i print information about song/station
j add shared station
m move song to different station
n next song
p pause/continue
q quit
r rename station
s change station
t tired (ban song for 1 month)
u upcoming songs
x select quickmix stations
b bookmark song/artist
( decrease volume
) increase volume
= delete seeds/feedback


as you can see, it is versatile.

Google Ruby Script

My friend coded this for our team. It will take a google search query, and parse out the returned URLs one on a line. It comes in handy for scripting recon on a client. Obviously you need the nokogiri gem installed.

#!/usr/bin/ruby
require 'rubygems'
require 'nokogiri'
require 'open-uri'

# Perform a google search
# Add numbers in increments of 10 to the array to search more pages
[0].each do |num|
doc = Nokogiri::HTML(open('http://google.com/search?q=' + ARGV[0].to_s + '&start='+num.to_s))
puts "Scrubbing Google for good-ness"
doc.xpath('//h3/a[@class="l"]').each do |link|
temp = link.attribute("href")
puts temp
end

end

I hope you find it useful...

Organizing my /gif

I have accumulated MANY gif files over the year. The problem is a gif can both be static or animated. I wanted all my animated gifs in one folder. After a little research on wikipedia I discovered that the original gif specification didnt allow for animation. Gifs that animate have a header containing the value "NETSCAPE2.0". Knowing this, it was trivial to sort animated vs static gifs. As shown in the following bash one liner:

mkdir animooted_gifs; for i in *; do if head $i | grep 'NETSCAPE2.0'; then mv $i animooted_gifs/$i; fi;done

Lol, i was creating this nice long script that would do checks and verifications and all that crap...meh...one liner ftw...

A little explanation:
A search through wikipedia taught me that animated GIFs should contain a certain header value: "NETSCAPE2.0". The bash line, searches for all files in a directory, greps the file for the header NETSCAPE2.0 and moves the file to animooted_gifs directory. If it doesnt detect NETSCAPE2.0, it moves on to the next file.

Running the unix util "file" on the gif doesnt yield much. As you can see below, a static gif and an animated gif report the same properties (obviously the size will be different since they are two different gifs):

gifs/1266641860129.gif: GIF image data, version 89a, 1003 x 1217
gifs/1245733506951.gif: GIF image data, version 89a, 375 x 375


BUT, if you run strings through the files you will see something different:

$ strings 1245733506951.gif
GIF89aw
33 84...

$ strings 1245733506951.gif
GIF89aw
33 84
...
NETSCAPE2.0
...


If you have been paying attention, you will know which file is the animated one (hint, its the 2nd one).

NOTE: Some files are constructed in a retarded way for some reason and don't have the NETSCAPE2.0 header even though they are animated. I have found this to be a rare exception to the rule. In any case, my one-liner should sort the vast majority of the animated GIFs into the correct location.

SSH Escape Sequence

If i'm SSH'd into a box and lose my internet connection, the ssh session will most likely just hang there like a retard. SSH has an escape sequence similar to telnet's ctrl+]

hit enter
then type ~.

That will close the ssh session and return you to the shell. ~? will return a help menu with different escape options that SSH accepts. the list is below:
Supported escape sequences:
~. - terminate connection (and any multiplexed sessions)
~B - send a BREAK to the remote system
~C - open a command line
~R - Request rekey (SSH protocol 2 only)
~^Z - suspend ssh
~# - list forwarded connections
~& - background ssh (when waiting for connections to terminate)
~? - this message
~~ - send the escape character by typing it twice
(Note that escapes are only recognized immediately after newline.)

Now you dont have to lose your terminal window! yay!

Major Experiment

I decided to delete all of my RSS feeds from google reader. I had over 60 subscriptions that I felt I needed to go through on a daily basis. Well recently, I discovered myself spending too much time on feeds. So, in response, I decided to delete them all to see how my productivity increases.

I have enough friends/chat channels so that I dont miss anything major that is happening in the field. I will edit this blog post with my thoughts/findings regarding this experiment :D

EDIT: So its been less than 24h and I have already been discovering new blogs/reading on things I have been meaning to learn for a while now. I did decide to actually have 3 subscriptions on my reader though(ITS NOT CHEATING I SWEAR). The subscriptions are xkcd, questionable content comic, and woot.com. I dont consider it cheating because those take all of 5 seconds to check and i check them regularly anyway.