Node.js Hajime !

So I begun with node.js yesterday after watching some insane and cool apps on www.nodeknockout.com . The first thing to do was obviously to watch Ryan Dahl’s videos explaining the philosophy behind node.js.   The major portion of it is about avoiding blocking I/O by using callback functions, together with an event loop based system. Not too novel, several similar frameworks already exist for this – think Twisted for python.

However, what it does do is allow JS developers earlier restricted to the front end/browser/desktop, to do I/O in an efficient way by integrating with Google’s V8 JS engine, and combining the power of POSIX libraries like libev (for select/poll/epoll) and libeio ( for threadpools which it uses to implement callbacks ).

So I began with node.js, and tried to implement an MTV platform ( the next thing to do after the Hello World tutorial is over ) a-la Django.

The source code for this project is at http://github.com/addictiv/nodjango

The entire thing comprises of 4 files :

• urls.js: You write the mappings from urls to view functions here, like you do in urls.py ( in Django).
• views.js : You define the view functions that you have mapped to ( in urls.js ) here, as you would do in views.py ( Django).
• controller.js : You don’t need to worry about this file as a user. This takes care of the HTTP request handling and URL dispatching, based on urls.js .
• runserver.js: You don’t need to worry about this file as a user. This takes care of starting the server and initializing it with a
  request handler.

 You run your webserver as :

> node runserver.js

Once you have run your webserver using the above command, go to your browser and request http://127.0.0.1:8000/url_path

If you have defined a mapping for ‘url_path’, in urls.js, the mapping’s view function in view.js is called and the appropriate content returned. If not, you get a 404 not found error.

As an example for what and how to write view functions, you can simply take a look at the existing view functions in views.js !

The code for this project is at http://github.com/addictiv/nodjango

So there you have it, your first trivial node.js app, up and running in less than an hour.

Next up, hopefully something nontrivial. :D

Wah!

The above one was by Dwarka Prasad Maheshwari. Inspirational and beautiful words :)  Of course, when I actually read it ( class Xth) for the first time, I couldn’t possibly fathom the depth and relevance of these words. To me, back then they were just some lines that I had to vaguely grok so that I could answer related questions about what the poet intends to say. I sometimes look back and realize what a dumbass I’d been for the first decade and a half in my life. Ah well, youth is wasted on the young. And inspirational poetry wasted on a generation obsessed with scoring marks and pipping their peers more than original thought, something that this poem so beautifully promotes.

Markov chain sentence generator in erlang

Long ago I’d implemented an python IRC bot (first post on the blog) that used Markov chains to speak semi-intelligible drivel based on a corpus(brain/soul) that is filled with other drivel that gets talked about in chatrooms.

Here is the logic behind using Markov chains to generate text that sounds intelligible :

- Feed the program a big input text ( from a text file or from a chatroom), and have it parse it into a ‘bag’ ( a dictionary with multiple values for a key, which are arranged in a list ).

- The key can be a number of words, which is the chain length. e.g: {‘the’, ‘dream’, ‘of’} is one key of length 3. {‘what’, ‘can’} is a key of length 2. The value for these keys is the word that follows them in the input body of text.

e.g: “the dreams of men never die.” yields  { {‘the’, ‘dreams’, ‘of’} -> ‘men’, {‘dreams’, ‘of’, ‘men’} -> ‘never’ ……}

as the bag, for a chain length of 3 words. Let’s assume we have a chain length of 4.

1. Now pick at random any 4 consecutive words from the input text. This is our initial state S0. If S0 = {W0,W1,W2,W3}, then our output string is W0 W1 W2 W3 right now.

2. Based on current state S0, and the value of this key in the bag ( say V), we move to a next state S1. i.e: If S0 = {W0,W1,W2,W3}, and Bag[S0] = V, then S1 = {W1,W2,W3,V}. Our output string is W0 W1 W2 W3 V. Our state becomes S1 now.

3. Repeat 2. with state S0 = S1 (the output state of step 2) until we have the output string of the desired length or property.

Here is a small implementation in erlang. http://pastebin.com/bGaW7CkK

- Save the above file as markov.erl

- Compile with erlc markov.erl

- Run with erl -noshell -s markov start “/path/to/corpus/file” -s init stop

Here is another version ( a distributed version, dishing out chores to small processes ) - http://pastebin.com/d1RWmZ3Z . Be warned though – the distributed version will easily stall your dual core laptop if you feed it with a monstrosity comprising of 10K words. It spawns one process for each word.

On running either of the above versions, a random line that sounds intelligible shall be displayed. The program uses Markov chains of length 3. You can change it by changing the constant ARITY defined at the top.

Running it on a txt file (about hackers and painters) as its corpus yielded the following output – with a chain length of 3 - http://pastebin.com/aWZeZKmw

A genetic algorithm example in Erlang.

I’ve never cared too much about genetic algorithms. It sounded way too far fetched and impractical, so I’d absolutely no interest or knowledge about it. That was, until today  - and this awesome article explaining genetic algorithms. There’s nothing like a well written article, and this was surely one of that kind. At the very least, it inspired me to write my own version of the idea, although without the fancy graphs, etc. Since the time I read the article, I’ve already had a few places where I think such an idea could easily be put to work :).

The idea behind this program is to begin with a base population of X people, and try to breed/mutate into Y ( where Y is specified on the command line ), through general selection and random mutation. Basically, what this means is that we can select the population fit to breed, and can randomly mutate the progenies formed out of the mating. Varying the parameters (like the policy with which we select the population fit to mate, or the probabiliy with which mutation occurs, leads to interesting results!)

So here it is in erlang, in all its glory http://pastebin.com/hk8yNEi3

1. Download and save as genetic.erl

2. Compile as erlc genetic.erl

3. Run from the command line as erl genetic.erl -s genetic test dinosaur -s init stop     (If you want to evolve into a ‘dinosaur’).

The output will display a series of cross-breeding, followed by random mutations until a ‘dinosaur’ is formed ( from random 8 lettered words ).

To reiterate – please read the post linked to, it’s awesome.

Thanks,

Abhinav

Erlang websocket server ( websocket protocol 76 )

*Note* – As of Oct 24, 2011, This version of websocket server will only work properly with Google Chrome <= 13.X. The new and last call ( hopefully stable) version of the websocket draft has been released, and soon someone will implement the handshake, which changes a bit ( the headers change a little, and so do the framing/encoding bits ).

In this post earlier today, I’d written a simple websocket server implementing the 76thwebsocket protocol ietf draft in Python. Erlang felt bad and whined about it so I redid it in Erlang. :)

http://pastebin.com/fmacSxGA

It’s mostly based on Joe Armstrong’s original implementation of draft 75 of the  same, but written from scratch to implement the 76th draft, which differs from the 75th, as mentioned here.

Thanks,

Abhinav

Python websocket server ( websocket protocol 76 )

*Note* – As of Oct 24, 2011, This version of websocket server will only work properly with Google Chrome <= 13.X. The new and last call ( hopefully stable) version of the websocket draft has been released, and soon someone will implement the handshake, which changes a bit ( the headers change a little, and so do the framing/encoding bits ).

 

 

http://pastebin.com/zBjN02jQ

A simple python server that handshakes an HTML5 enabled browser connecting to it using websockets. Also includes basic message framing. pywebsocket was way too overkill for what I needed, and there were no other python implementations implementing revision 76 (most implement 75 which is slightly different) so I decided to implement a quick handshake myself. The server’s end of the handshake has 13 steps. If you think that’s too much, you should know that the client’s side has 43 steps !

TODO : disconnect handshake ( doesn’t interrupt functionality though ).

Websockets mark the death of workarounds like ajax, orbited/comed, polling, since a browser can now simply open a socket and connect to any application with an open socket that is willing to handshake it according to the w3c’s recommendations.

Enjoy !

A very basic erlang Crawler

1.Save the code below as spyder.erl

2.Run as : erl spyder.erl  -s spyder start http://site_to_crawl.com -s init stop

http://pastebin.com/Qw9f11bi

This is a very crude crawler – will crawl all the links in a page, and then further. It doesn’t protect you from black holes, and will crawl away without concern for robots.txt. Just something to brush up my rusty(probably visible in the code ?) erlang. But what the hell – it works ! Feel free to improve on it :)

[OwlKun] Integrating OAuth with Twitter’s API in python

I believe everyone’s heard that  Twitter will be doing away with basic authentication come June ’10. They’re switching to OAuth. It’s a mechanism via which a consumer (C) and access the resources of a site (S), on behalf of a user (U), without a username or a password being provided to C by U.  ( C can be either an application or a website, U is a user registered on S .)

In our case, C is our application ( which here is OwlKun). U is you – your twitter account. S is Twitter.

Now, every python twitter API/library will eventually have to switch to OAuth, and many have already done so, but there is absolutely no comprehensive documentation on how to go about integrating your app with twitter, using any given language apart from the existing source code.

Lines 75-104 contain the OAuth specific material, others contain integration stuff so that you can actually run these commands from inside vim. :)

Here’s the link to OwlKun’s source :

http://owlkun.googlecode.com/files/owlkun.py

Etymology : OwlKun = Owl ( Stays awake at nights, it was night time when I wrote this ) + Kun = Friend ( Since I have no friends since it’s a nice and friendly application that’s well documented. )

Thanks,

Abhinav

Setting your currently playing rhythmbox song as your pidgin status message.

This script will set your pidgin status message as the song that’s currently being played by your Rhythmbox.

It’s a simple process – save http://pastebin.com/jeHPqKWT to a file called status_changer.py, and run it. i.e:

1. Save it to a file – say status_changer.py
2. Run it : python status_changer.py

3. Stare in awe :)

 

:)

 

Why Love ?

This is in attempt to answer one of the questions that baffle me, and expectedly a few others – why love ? That is, why do we fall in love ? The kind of love I’m talking about is  love at first sight. Those who think such love doesn’t exist are merely lucky or unlucky enough not to have experienced it. However, it does exist. But why ? First, a little about evolution.

Like it or not, we have all evolved. From stuff that came from some star. In the words of Carl Sagan, “We’re star stuff, harvesting star light.”. There’s absolutely nothing special or divine about us, except that we might be able to prolong our survival through our developed brains.

Has it ever occurred to you why we find certain things repulsive or scary while certain other things attractive ? We’re instinctively disgusted by the smell of rotting flesh, scared by the sight of a snake, but charmed by flowers. Why ? The answer lies in evolution.

Throughout evolutionary stages, our ancestors roamed the African plains, hunting and gathering. In the course of their lives, they encountered various stuff. Like rotting flesh. To the first human tribe, rotting flesh probably did not smell ‘bad’. It probably didn’t smell at all.  It was like all flesh, except that it had harmful bacteria that made the person who ate it, sick . Unaware, they ate it. And fell sick. Some died. The ones that survived, made a note of this in their brains. Over generations, this piece of information about rotting flesh being bad, got converted into a series of chemical reactions in the brain that developed a ‘smell’, which is essentially a flow of hormones to your body that made the smeller feel nauseated. Thus, the smeller became averse to the rotting flesh, through the ‘bad’ smell. Chances are that the rotting flesh does not smell bad to a dung beetle, since they thrive in Hydrogen Sulfide.

Now how does all this add up to love at first sight ?

When you think about it, a person’s face is merely a collection of contours that might or might not appeal to you in the same way it does to another.  Why this difference in taste ? If your ancestors, ( not just your parents, but 10s or 100s or 1000s of people above you in your hierarchy ) found a great degree of success in producing kids with a female having a given kind of face, their brain stored this information. Passed on over generations, if each of the people along that path found great success of producing healthy offspring in the same kind of face, the feeling that the bearer of that given kind of face will help you produce an offspring better than the bearer of any other kind of face, intensified greatly.

This is where the intensity of love that people talk about comes in. How can love at first sight be so intense if you don’t even know the person ? Modern social dynamics requires that people get to know each other over a period of time and then, your love could be called intense. Most people would rubbish Love at first sight as mere lechery or weakness of character, without even looking into what causes it. So now, we have two kinds of love -

1. The cultivated love, ( the one that grows with companionship, trust, etc. )

2. Love at first sight.

Let’s dissect both of them with the perspective that we’ve held up until now – evolution.

1. The cultivated love between Male M and Female F

Well, throughout the generations, no particular kind of trait stood out, and hence no particular face stood out greatly. They all averaged out, and good still remained good, but no kind of face was electric.  However, throughout the generations, one thing was common – the chances of bearing healthy young would increase if the couple formed a bond of trust and affection with one another, preferably for life.  Hence the ‘Cultivated Love’.

2. Love at first sight between Male M and Female F.

From an evolutionary perspective, this kind of love would occur when the  males slept around with females that had a lot of similarity in their facial/physical features.

For example, if the first generation male picked a female that had narrower eyes, a pug nose, large shoulders, a small chin, etc. and if he was able to produce healthy offsprings with her – this kind of a face would get persisted into his memory. But this attraction  was still not written in stone. The next generation ( his offspring ),  found a female that had similar features, and was once again able to produce healthy offsprings with her. The genetic persistence into his limbic brain got a little stronger. Some 50 generations of similar females later, the rule of attraction was that much closer to being  written in stone. How ? The limbic portion of the brain of the 51st generation kid was imprinted with brain code telling it – “This kind of face/physique is right for you. If you copulate with her, your kids will be strong. Go for it ! “.  When the kid actually sees a girl with a similar face or physique,  his brain tells him – “Hey, that girl is so pretty ! Go for it !“.  How ? Once again, through a rush of hormones, that give the kid a high. The kid of course, interprets this as love, and goes for it. The female agrees, and they have healthy offspring. Now, this feeling is intensified over generations. We’re probably into 10000s of generations, maybe more, since humans developed the brain. Imagine the intenstity of this love at first sight, if most of our ancestors reproduced with similar girls.

This is why Love at first sight is not balderdash. And is probably more intense than Cultivated Love, atleast at first.

Now, what happens when the female you have fallen in love with at the first sight, refuses your advances ? Is it doom ? Does it mean your genes will end with you ?

Hardly. Even ignoring the wrong  assumption that there’s only a single female with the approximate facial/physical characteristics that your brain has been told is good for you, there’s the option of using that which makes us human – the frontal cortex. I.e: Cultivated love, through conditioning. Without the frontal cortex, and our control executive, we would have been left at the mercy of our ancestors’ experiences. However, with the inclusion of this newly evolved brain, we get to cope with constantly changing surroundings. It’s like the inclusion of a RAM chip and a hard drive  over a ROM chip – the latter is read only, while the former are configurable.

While making a choice, the executive area of the brain is consulted first. The limbic portion usually comes into play during emergencies, or when the executive area is too swamped, or resting (like when we’re daydreaming ), or when we’re intoxicated ( alcohol, lack of sleep ) .So, for the most part, you can configure your executive brain to trigger the release of the same hormones that get triggered when love at first sight occurs. This means that cultivated love can be just as fun as love at first sight, since the same hormones get released with both of them.

However, since your limbic portion still has some other, possibly conflicting, information about the person you’re supposed to be with, you will have to lead a life of slight  subconscious conflict ( which you’ll probably never realize anyway.) All it will mean is that your offspring will not feel AS attracted to the girl that you fell in love at first sight with, since you were not able to produce offspring with that girl. If this continues, the rule written in stone might get erased and 1000 generations later, your descendant might find another kind of girl attractive.

Conclusion  (and some fundae :P ):

There is nothing divine or pure about either us or love. No Cupid, nothing.  The ability to feel loved is but in your own hands. And the best way is to love everyone, like the sun – the object all gods are based on. The sun gives warmth to one and all without distinguishing between people, and without asking for anything in return. If we, are able to achieve such a level of giving love, it must mean we have love. For, in the words of John Marks Templeton, “How can you give love if you don’t have love ?” .

So, if you’re feeling piqued because some girl dumped you, don’t ! Mainly because you’ve got other options, and secondly, because you’ve got company !! :D

Cognos | The cognos #prompt()# macro

Here’s another bitch. The Cognos #prompt()# macro.

In case you’re wondering about giving a default value to the prompt, here’s how you do it :

#prompt(‘prompt_name’, ‘prompt_type’, ‘default_value’)#

However, there is a catch. The prompt won’t accept just anything as its default value – the third argument can only be a column name in case of strings. However for numbers, it accepts numeric constants. For example :

#prompt(‘id’, ‘INTEGER’,'-1′)# is correct,

but

#prompt(‘name’, ‘STRING’, ‘HELLO’)# is wrong. It will have to be :

#prompt(‘name’, ‘STRING’, column_name)#

Weird. I’m not very sure about this, but After a lot of hit and trials on it, where there is no documentation available on this particular topic, I think this is correct.

Please let me know if this is correct/wrong, if you, like me are fedup with the lack of documentation too.

PERL | DBI | Batch Upload/Insert – Row wise vs Column Wise binding.

Today’s topic is Bitch uploading. Oops – Batch Uploading. Using Perl DBI. Into SQL Server. It’s hard to believe that anything perl is poorly documented. Well, atleast non-intuitively documented. How else am I supposed to find out how to bind row-wise instead of column-wise to a prepared statement ? You would think that `perldoc DBI` would help, but it has but one nondescript line about row-wise binding.

Anyway, since I am documenting anything that people might find useful in their lives, here’s what I have found about row-wise binding in DBI as opposed to column-wise binding ( which is easy, well – better documented ). I show you examples of both.

To bind column-wise, use this :

$sth = $dbh->prepare(“insert into database..table values (?, ?, ?)”);
@array1 = ( 1, 2, 3 ) # array that contains the values of column1 across all the rows.
@array2 = ( 1, 2, 3 ) # array that contains the values of column2 across all the rows.
@array3 = ( 1, 2, 3 ) # array that contains the values of column3 across all the rows.

$sth->execute_array({}, \@array1, \@array2, \@array3); # {} is on purpose, not a typo. Read the perldoc for more.

—————————————–

To bind row-wise, use this :

$sth = $dbh->prepare(“insert into database..table values ( ?, ?, ?)”);
$sth->execute_array({ArrayTupleFetch => sub { return_a_referecnce_to_an_array(a row)_with_three_columns }});

what the second statement does is that it calls the subroutine repeatedly over and over ( by itself, without you specifying a loop ) till it returns an undef.

e.g:

$sth->execute_array({ArrayTupleFetch => sub { return shift @array_of_row_references }} );

so, when called repeatedly, the code will keep shifting an array ( thereby returning its first element ) till it runs out, in which case it will return undef.

Have fun with bitc..err..batch uploading !

—-

All this info is courtesy the Author/Architect of the perl DBI – > Tim Bunce ( or something to that effect, apologies to him – a great guy, ubiquitous with his help ).

Sex | The Irish Elks and the Gibraltar Macaques

Once upon a time, there lived an Elk in Ireland, we shall refer to it as the Elk. Now, Ireland being a male place, there were all of  6 female elks left. These female elks were not very attractive elks, but had the female reproductive organ nevertheless, which sanctioned unto them the right to being called female. Now our Irish Elk was good and all, but was faced with the challenge of trumping the remaining 659 of his elk brethren in the battle for one of the 6 female reproductive organs. This was not just his plight, mind you, for scroll back through the annals of Irish elk history, and you’d see a similar pattern emerge across several pedigrees.

But first, a few things about Elks.

The Elk. This animal,

1. Has terrible eyesight.

2. Will dive more than 2 feet underwater if it sights food there.

3. Is herbivorous.

Ok, enough random Elk trivia. If you had to relate them to another animal, you’d probably name them the nordic cousins of the moose, or a reindeer. They are nice animals overall, but in all their nobleness, they are, still, animals. Now elks, as all other animals, have this read-only chip branded into their limbic brain, which renders them helpless when it comes to reproduction. They get bent out of shape, desperate to impregnate their way  into the future.

So, with 6 females and 659 other males, our elk was in trouble. He was, after all, an average elk, with average-sized antlers. And antlers, as the cruel times would have it,  were the currency for copulation with the female elks. He tried every way in the book of increasing the size of his antlers, but the elk books weren’t so anatomically advanced yet. He did finally copulate though, but it was with a female who several elks had already copulated with, and the chances that his sperm would make it through now, were one in a googol. Frustrated, our elk passed away bearing the same complex that had plagued his lineage and several more in different lineages – small antlers.

Despite all our efforts however, randomness prevails, and our elk’s sperm fight their way,  through zillions of other elk sperm cells – cheesy inspiring background music playing all the while – and impregnate the female elk. A healthy young elk is born, and has substantially bigger antlers than that of its biological father.  Now I know what you’re thinking, “Ah everything’s fine,  the story has a relatively happy ending – the young avenges the deceased. ” .

But no, that’s not what happens.

The newborn  looks around, only to find  tens of other fledgling elks, with antlers bigger than its own. Another generation of inferiority complex sets in. The story repeats, and fortunately, this time around too, our second-generation elk manages to impregnate a female elk before dying a sad death after a life filled with want of bigger antlers.

Fast forward to 2008 antler-obsessed generations later, a young elk is born – for the 2008th time,  in our original Elk’s hierarchy. Punch drunk from the bloody, roller-coaster transport from the elk uterus into a soft tuft of grass on the ground, the elk opens its eyes and is instantly in awe. It looks at the vast blue canvas littered with cumulo nimbus like bales of cotton in disarray. It looks around, awestruck at the vastness of the meadow and the enormity of the horizons. It’s this elk’s time to shine, to leap into these horizons that seem to challenge its gallant leaps.  Carpe Diem time !

It rolls its back sideways , two tiny paws touch the grass, the stomach and the back get off the wet meadow grass and the two other paws follow suit.

“Wow, that’s hard! “, it thinks.

CRACK .

The last thing the elk sees before the lights go out on him is the other side of the meadow, littered with baby elks, mostly dead, the alive ones trying to get their giant antler heads off the ground, and its mother – smeared in her own blood and in her death throes, still bleeding from the lacerations caused by the young elk’s huge antlers. And then, all goes black.

Meanwhile, two random macaques in Gibraltar have sex.

Not a single Irish Elk has been sighted since mid-2007. The Gibraltar Macaques continue to flourish.

Go ahead, get huge antlers.

*This post was inspired by a documentary on eovlution. And how evolution and survival are more often than not,  trade-offs.

Adding Robotic/Vocoder effect to your song using Audacity.

Have you heard songs where it appears as if a robot is singing a line instead of a person ? Want that effect for your songs ? Now you can. Yes we can !

You will need Audacity ( comes for windows/Linux/Mac – I use the linux one, pretty much the same as the others ) , the vocoder plugin ( which will automatically be there if you install the latest audactiy version. ) Here’s what to do :

1. Load up your song.

2. If your song is stereo ( has two tracks, Left and right, ), split the track into left and right, by clicking on the small arrow that is there in the control box to the left of the track.

Once the track has been split into left and right tracks, select  the entire right track, and then replace it with a sound that has a high amplitude to it, like a synth lead or a rave lead . Or, for experimenting, just with noise. To generate white noise, Click Generate -> Noise -> White.

3. Then recombine the left and right tracks to form a stereo track. Do this by again clickin on the small arrow to the left of the lef track, and selecting ‘Make stereo track’.

4. Now, in Effects->Plugins, select Vocoder. You might want to play with the settings a bit, but a higher number of vocoder bands mean a higher quality of robotic sound, and a lower value would mean more noise.

Apply the effect and enjoy a slightly robotic voice ! To get a more robotic voice, you would like to have a sound with more energy in it. If I had to exemplify, it would be something like a lot of synth organs playing. Like a cathedral noise with lots of strings ( cello/violins/hurdy-gurdy ) but synchronized. Luckily, I had such a song, ( The Undertaker’s WWE entrance theme played using a synth organ kinda instrument ), so my effect sounds realistic. With white noise, it won’t sound very robotish, so you need a proper sound which is in tune, unlike noise, as your carrier ( more on carriers below ).

Basically, how is works is that to vocode, the plugin needs two signals – one is the carrier signal ( the synth lead sound or the noise – in the right track ), and the other is the main sound that we want to vocode ( your song – in the left track ).  The vocoder  does the rest.

It was good for playing around, so I thought I’d let everyone know – there’s a tonne of cool plugins for audacity and it’s fun to play around with them, so by all means experiment !!

Next time, I’m trying the Pitch Snap effect ( commonly known as the T-Pain effect ? )

BTW, if you have windows and FL Studio ( Fruity Loops), you might want to visit www.warbeats.com , an awesome site for people interested in composing/mixing/merging music. Some really nice music stuff out there !