Archive for the 'General' category

Books and Blogs

Aug 04 2010 Published by Niyaz PK under General

Rant follows.

Reading is taking up much of my free time these days. Of course I am not talking about the reading online. Spending time online has been a huge time sink for me (and as I can imagine, for millions of other people like me) over the years. I have turned to reading books with more enthusiasm ever since I realized that the stuff that I read online does not stay with me for a long while, while the books I read have a profound way of changing the way I think.

Now don’t get me wrong; I am not blindly implying that all the information out there in the web is a waste of time. I am just saying that for me personally, books give more satisfaction when I finish one compared to hundreds of blog entries I used to devour on a constant basis. Even though I have cut short on the number of blogs I visit and the number of articles I actually read, there are communities like Hacker News where I tend to hang out. Hacker News is a great source of news, information and inspiration for the curious type, but it may also have the effect of wasting a whole lot of your time if you start believing that all of the time you spend on Hacker News is productive.

Reading itself is not a productive activity in the absolute sense of the word (or in any other sense of the word I can imagine), but it is a really fulfilling and satisfying experience (not to mention that it strokes your ego when the height of the books you stacked up in your cupboard grows more than that in your close friends’ room). Even though reading books involves seemingly similar activities as to reading a blog, there are some real differences that make them worlds apart in actual utility.

The first is that your mouse is a constant stream of distraction at your fingertips. Every time I read an article, I try (in vain) not to click on the other tabs to check my email or the traffic stats of my blog; not to mention the advertisements and other outbounds links in the same page itself. Even the hyper-links in the article you read are very distracting, according to some [1]. Now that you have noticed it, that is the reason I am not using any hyper-links in the middle of this article. Many of the links in a typical blog post point to the definition of terms in the Wikipedia anyway, and they are not very useful; but I digress.

The point is that just the act of sitting in front of the computer with the goal of consuming the internet translates into thousand of interesting (or irrelevant) articles, news, stories, pictures and videos fighting for your attention. The problem is that you have only a finite amount of attention. You have only a finite amount of stuff you can process per day.Yes, there is a limit to your cognitive ability. This means that there is a limit to what you can read and assimilate in a given time. And this in turn means that all those distractions are getting the way of you learning something useful. Now I am not advocating that you abandon your fair share of lolcats, but rather wondering whether you are spending a lot more time on them than what can be deemed excusable.

The second difference between books and blogs is that a book is a lot more harder to write than a blog post. You can be sloppy while writing a blog post, knowing that you can later change your words as necessary. Books are very difficult to get published compared to blog posts. Getting a decent publisher to publish your book is not going to be easy for you.  This means that only very few books which are written are actually published and whatever is published goes through a good deal of scrutiny and editing which enhances the quality of the material you get to consume. This does not mean that there are no well-written and well-edited articles out there in the web. I am talking about the typical blog post (By the way, referring to the typical stuff is a great way to say anything you want in an argument). Even given these facts, you can find great articles worth reading in the web by following news aggregators like Reddit or Hacker News, but then again there comes the problem of loads of wasted time with these websites. Even though news aggregators are good at finding the best articles in the web, there is one other fundamental difference between books and essays that makes me go back to good books.

The third difference between a book and essays is that a book has got all the time in the world to convince you of the idea that it is trying to sell. Books are usually hundreds of times longer than your typical blog post and this means that a book author can take his time to establish the basics and build on them and the take the reader through the innards of the subject he wants to write about. The fact that a book takes hundreds of pages to present its plot means that the idea the book tries to sell will stay in the back of my mind for many years, compared to an essay whose central thesis will be forgotten in a matter of minutes when other distractions occupy my brain.

So my request to the dear reader will be to complement what you read in the web with some very good offline reading. Good books can change you life, while blog posts usually are not that powerful. And while we are at it, just keep in mind that producing something useful and making something people wantTM is way more better than reading books 24 hours a day.

That’s it. We are done here. Go away.

[1] Experiments in delinkification

[2] Photo credit: somegeekintn

4 responses so far

Chess programming and such

Jan 16 2010 Published by Niyaz PK under General

Another story from my college days. Be warned, long article.

Engineering colleges in Kerala require students to submit a mini project in the third year of their computer science and engineering courses. This is in addition to the main project which is to be submitted in the last year of the course.

Two of my classmates – Praveen Kumar (who in our circles is regarded equal to Jon Skeet), Philip and I formed our team for this mini project. We were very much excited about this project that we started discussions about it a lot earlier than the official project start time. After a lot of late night debates we decided to develop a chess playing program.

Three of us were sharing a rented house near college and we used to develop software utilities for various purposes#1. The main point to note here is that we developed these programs in – wait for it – Visual Basic 6.

We were very much comfortable in Visual Basic, and since it is very easy to develop a good chess UI in Visual Basic, we started building a prototype of the application in VB6. We thought that once the basic logic is done and working, we would port it to a better and faster language. Unfortunately, we worked on the code for very long and hard that the code base grew larger and longer.

Testing the program was somewhat tricky. The alpha-beta based algorithm was in a working condition and the program was generating and making some basic moves, but we were facing two problems:

Problem 1: How do we know if the computer has played correctly? There is no way to really make sure that the computer has played the correct move (for any given depth) because we ourselves don’t know the correct move! Of course if you are a good chess player you can find out a good move for a particular board position, but that does not solve the problem. First of all you cannot be sure that your move is the best move. What if there were better moves that you just did not see? Secondly, chess players do not think exactly like a computer. They do not calculate the moves strictly using an algorithm. Some of their decisions are based on intuition.

The very best chess players can just look at a chess position and see the 2-3 lines of play that they should analyze instead of analyzing all the 30+ available moves. Computers cannot do that. Computers should look at all the moves to decide whether to analyze that line of play to more depths or not. Humans are good in chess strategy while computers are good in the tactics. This means that a human cannot tell whether a computer is playing the perfect game (for any given depth) or not.

Problem 2: Since our implementation was not complete and no optimizations were made initially, the computer would take very long time to think and make its move. We had to wait something like 4 minutes or so to get each move from the computer and it was taking up much of our development time. Verifying the correctness of the program was going to take a lot of time.

One good way to test a Chess program is to make it play against itself#2. So we would code till late nights and in the mornings when we go to college we would start the program and make it play against itself. When we come back we would look at the logs and see what happened. As much as excited we were, the results were often depressing. Most of the time the game will go into a loop where each player played (back and forth) the same moves. The reason was that once the chess AI#3 found the best move for a particular board position, it just plays the move disregarding any other facts like the history. The best way to fix this is to tell the program to consider the move history too in making its decision about which piece to move.

Another reason for frequent loops was that both the players in the game were of the same strength (they were both thinking x moves ahead). To fix this and to see some real results we made them of different strengths. One of the players will think x moves ahead while the other would think x-2 or something like that. This improvement helped us see some real results. When we came back to check the results at the end of the day, we could see one of the players checkmated!

If you are into computers, chess programming is one of the best ways to have fun. There are a lot of intricacies in the implementation so that you can tweak the program again and again to get performance improvements. The best chess programming tutorial (for beginners) I have come across is the one from GameDev.net.

I remember one other small bug we faced. Here is how a computer chess program finds out the best possible move:

Find out all the valid moves available to the player. Assume that you played the move and find out the new board position. Now look at the new board position from the perspective of the opponent and try to figure out what move he is most likely to make. If he makes that move, what will the new board position look like? What move will you make to counter his move?

You can go on and on iteratively deeper and deeper into this search tree#4 . The deeper you search, the better your decision (move) will be. For each board position the algorithm will assign a score. The score determines how good the board position is. If one of your pieces is captured, you lose that much score. For example if you lose a pawn, you lose 1 point.

Now look at the following game tree. Blue represents a move by the computer and red, a move by the opponent. The numbers indicate the points gained by the computer in each step.

For example in the first line of play, the computer gains 5 points in the first move (probably by capturing a rook) and in the counter-move the opponent fights back by gaining 3 points (probably by capturing a bishop/knight). Now as you may remember, the algorithms just looks at the total points of the player at the end of each line of play. In both of the above cases, the total gain by the computer is 2 points.

If the computer chooses between these two lines of play randomly, you are in for trouble. The problem is that the computer does not know what happens after the last move in the line of play. This means that if you follow the first line of play you are almost certain to have gained 2 points after the next two moves while in the second line of play you will lose 3 points after the next two moves. May be you can gain 5 points later in the game, but what if after two more moves when you can see further in the tree you realize that the move you saw earlier would be disastrous? Then you will have to change your game and this means that you cannot get those points.

There are two ways to solve this issue. The first is to prefer early points to later ones. In our example, it is better to choose the first line of play over the second one.

The second way is to use quiescence search:

The horizon effect can be mitigated by extending the search algorithm with a quiescence search. This gives the search algorithm ability to look beyond its horizon for a certain class of moves of major importance to the game state, such as captures.

We did a lot of modifications to the code. We used to take the printed versions of the source code (about a 100 pages) to our college and tried to find optimizations that can be applied to the code. Chess programming is one of the areas where over-optimization is not frowned upon.

So after all the hard work, the program was working fine and it could play a decent game of chess (given enough time of course).

Translating the source code

We were just about being happy at how the project was going on when it hit us – the college put a restriction that all projects must be done using Java. We had to port the application to Java soon.

Rewriting the whole application from scratch seemed time consuming and demotivating. We needed a faster way. What about converting the source VB6 source code to Java code automatically? Of course for this to work we would have to write a VB to Java language translator. It seemed too difficult considering the fast approaching deadline for project submission. We downloaded some code translation software from the internet and tried them out, but none of them seemed to work perfectly. Converting a complex VB6 UI having a fair amount of custom animations to Java is almost impossible even for an advanced translator.

Then we had an idea. Regex!

Of course Regular Expressions are not ideal for parsing source code of any kind. We had no time, so we decided to try anyway.

In the case of any decent chess playing program, there are two basic parts: a chess engine and the UI module. The chess engine does all the complex calculations – recording the user moves, finding out whether a move is valid or not, thinking about the best move for the computer etc. The UI module displays the chess board to the user and allows the user to make a move using the mouse. Now as you may have already inferred, the UI part is not very translatable between VB6 and Java. We decided to develop the UI from scratch in Java.

The interesting thing about the chess engine is that it contains lots and lots of calculations, conditions and loops which help it to make an intelligent move. I should probably note that a chess engine cannot be considered intelligent as such. A chess engine crunches millions of number very fast and finds out the best move the computer can make against a human opponent. The moves may look intelligent to a human player, but a chess engine cannot be regarded as an example of an AI engine. It is just a number cruncher beneath its layers. Anyway the point is that we had hundreds of pages of code that consisted purely of algorithms which aided the computer in playing a better game of chess.

As we found out, converting these algorithms from VB6 code to Java was not that hard as it seemed. We employed the powers of Regular Expressions to do that. Here is a glimpse of what we did.

1. Replaced:

If with if(
For with for(
Then with {
Else with } else {
End if, Next, End Function etc with }
True with true
False with false
And with &&
Or with ||
Mod with %
Exit For with break;
= in conditional statements with ==

etc.

2. Added ; (semicolon) at the end of lines which did not start with any of the above keywords.

3. Changed the array referencing code. Something like Board(x,y) in vb6 code became board[x][y] in Java code.

4. There were some complex conditional statements in the VB code that we thought would be impossible to convert to Java using Regex. We had to convert these manually.

5. Converted looping statements of the form

For i = 0 To NumMoves - 1 to

for(int i = 0; i < numMoves-1; i++){

6. Variable declarations of the form

Dim index As Integer to

int index = 0;

There was more of this kind.

Of course it was not all Regex stuff. It is impossible to get it right with Regex alone. The small program we wrote to translate the code read each line of VB code separately and converted it to Java, mostly using Regex. The whole translation (Regex replacement) was done in several passes. We never implemented a full-blown translator or language parser.

Even after this entire Regex circus the translation was not complete. We had to read through most of the code and change a lot of things to get it working correctly. There are many differences between the languages (like array indexes started 0 in Java and 1 in VB6) and those needed to be taken care of. It did take us a bit of effort to get the details correct, but finally we completed converting the code to Java with much less effort than completely rewriting it.

Please don’t let this article make you think that I advocate using the Regex for this type of work. I don’t.

#1More on that later. Remind me if I forget. I promise that this will change the life of many undergraduates. ;)

#2Another good way is to pit it against better chess engines.

#3I included the word “AI” there just so that I could write this note. Chess playing algorithm cannot be considered Artificial Intelligence. Computer chess is a number crunching problem. Implementing the algorithms in the most efficient manner is the best way to get good results. There is no intelligence involved (compared to real AI techniques such as Neural Networks).

#4It is limited by the computing power you have. On an average the number of valid moves you have for each chess move is about 30. If you are going to search 8 moves ahead, you will have to search through 30^8 nodes, which is a very large number.

10 responses so far

Elegant logic puzzles

Dec 25 2009 Published by Niyaz PK under General

Nick Yee on elegant logic puzzles:

…an elegant logic puzzle is one that can be told to anyone age 10 and up and doesn’t rely on gimmicks, but always seems impossible to anyone when first told. In other words, the problem is tough but the solution is satisfactorily simple once explained. The solution doesn’t involve a person or tool that wasn’t explicitly stated in the problem itself…

Then he goes on and asks you two elegant logic puzzles. For a curious mind, both of them are extremely rewarding to muse over and the solutions are clean and elegant too. Go solve them if you haven’t already.

No responses yet

The General Pirate License

Nov 15 2009 Published by Niyaz PK under General

I have come across many situations where I wanted to share an idea but never wanted the idea to be attributed to me, mainly because the original idea never came from me and also because I did not know who the original idea came from.

There are some other instances where I want to share some source code which is almost working, but may contain bugs and untested edge conditions that can produce undocumented results (if at all there is any documentation). I want people to use the source code for solving their problems, but in most of the cases they may have to modify the code to fit their particular problem. I don’t want to take the blame if anything goes wrong, and I don’t ever want to support the source code, other than in the cases where I really want to.

I hereby propose a new license to help you in cases like these to promote sharing of your content in the best possible way – by allowing others to pirate it. This license should be used if you want your ideas or work to be shared and modified freely, but you don’t want to maintain or support the original ideas unless you really wish to do so.

(Edited to remove unwanted clauses. Thanks Scott)

The General Pirate license (GPiL)

1. This work may be copied as many times as you wish, modified in any way you want and published in any medium you like, provided you adhere to all the seven rules in this license.

2. You will not attribute the modified version of the work/product to the original author.

3. Once you modify the work, you should clearly mention the work as yours and you will be responsible for supporting and maintaining the work (if required).

4. You will not disclose the identity of the original author (i.e. your source) without the written permission from that person.

5. You will publish your modified version of this work under this same license (i.e. the General Pirate License).

6. All uses of this work will be at your own risk. The original author is liable to give you support for this work unless (a) You modified any part of this work/product (b) They do not wish to do so.

7. The source where you obtained this work from may or may not be the real original author of this work, but that is not the point here. The real point is the work itself, not the author.

There is a reason why all works under GPiL (General Pirate License) should continue to be under GPiL (clause 5). Any project under GPiL is supposed to be free to be modified and copied at will. According to the above rules, it is perfectly legal even to sell any intellectual property guarded by the GPiL. In essence GPiL allows you to do whatever you want with the IP – copy, modify, share, sell etc. The only restrictions that apply are the seven rules of GPiL. Clause 5 ensures that even if somebody is making profit from an IP, they have to share their version under GPiL. This is how the basic spirit of GPiL is carried forward.

That’s all. Let me know if any amendments can be made to this license to make it more piratical.

Now spread the word mate, and get sailin’. Arrrr!

9 responses so far

Geocities won’t be missed

Nov 11 2009 Published by Niyaz PK under General

Yahoo!’s shutting down of geocities made news for the past couple of months. Even thought the sweet memories of starting our very first pages in geocities will be lurking around for a long time, I think we are not going to miss geocities much. Yahoo! may have their own reasons# to close geocities, but I think all in all it was good that geocities got shut down.

If you think about it, geocities did not matter any more.

Yahoo!_Geocities

The traffic to geocities was declining very rapidly in the last few years. Very rarely did the old geocities pages get featured in the search engine results. I don’t remember getting a geocities page as a result for any of my google searches (May be my queries are too much specialized and are biased).

I would argue that geocites did not have much quality content. Most of the pages in geocities were personal pages which were “under construction” for eternity. Newer users never signed up for geocities. Social networking was in and creating personal pages was out and users flocked to Facebook and the like. If anyone wanted to create pages so badly, they usually started a blog in Blogger or WordPress. After Yahoo!’s announcement of a probable closure of geocities, much of the quality data was moved by the users to other sites. All this meant that the pages in geocities no longer mattered. It was just the junk of the internet that ought to be cleaned out.

The biggest impact the closure of geocities will have on the web is on the search engine results. Even though the pages from geocities were not prominently featured in the search results, they always polluted the long tail results. (38 million pages do carry a very long tail with it). Most biggies in the search engine business have removed geocities from their index. There is another big aftereffect to this. The search engine rankings of other websites will be affected. You see, this 38 million web pages of geocities had lots and lots of outbound links. Remember that these links are old  and do carry significant weight. If these links are removed from the PageRank calculations, the search results will not be the same. I hope that the search results will improve at least a  little bit.

By the way if you were sleeping for the past few months and missed the party, and if you really want to get some of your pages back from geocities, you can try to get the data back from the Reocities project or from the Internet Archive.

#Every reason is economic. Isn’t it?

2 responses so far

Bangalore!

Jul 16 2009 Published by Niyaz PK under General

I moved to Bangalore last week.

I will no longer be working with Keane. Instead, I will be working on my own startup. (Will keep you posted on it)

My new phone number is: +91 90364 98589. If you are in/around Bangalore, give me a call and we can meet.

In other news my good friend Binny is trying to release one wordpress plugin per day for a week. That is what you do when you want to shake up things a bit – you sprint, and that can make a big difference.

5 responses so far

Sorry. I screwed up.

Jun 23 2009 Published by Niyaz PK under General

Yesterday was not the best day in the history of this blog. Probably those of you who subscribed to the feeds have noticed. I tried to combine the feeds from here and the voidy blog to one single feed and it resulted in a multitude of duplicate feeds to be sent to your inbox.

Really sorry for that. I will not try that again!

For the technically inclined, here is what I guess happened:

I used yahoo pipes to combine both the feeds:

pipe  = feed(diovo) + feed(voidy)

Then I burned this pipe as the original Feedburner feed.

feed = feed(diovo) + feed(voidy)

I am using the Feedsmith plugin for this blog. Feedsmith plugin redirects all the feed references in this blog to the original Feedburner feed.

feed(diovo) = feed

Therefore now the new feed (using yahoo pipes) becomes:

feed = feed + feed(voidy)

Yeah. I see.

6 responses so far

Older posts »