Good to know that there are people who care.
A big thanks to Silky and all others who expressed concern over the blog.
Archive for: October, 2008
On scalability and reliability
I saw the following comparison chart in a web hosting site:
This is just nonsense. Scalability and Reliability are two things which you cannot enable on the flick of a switch. Scaling web applications is a very difficult job. For instance, twitter is still fighting scaling issues. Reliability is hard, very hard, too.
Quick tips for database programmers
Since you are smart, you know these things anyway. Just tell your friends too.
Avoid using select * from in your queries.
Why? Selecting every column in the table reduces the performance of the database, the network and the application in general.
Alternative: Use something like select required_column1, required_column2,… from table_name instead.
Avoid using code like a = resultSet.fields(1); in your application
Why? When somebody changes the query, 1 may refer to another column.
Alternative: Use code like a = resultSet.fileds(“column_name”);
Never assume same order of records when a query is fired multiple times.
Why? Each database engine retrieves records using different optimization techniques. The order of the records fetched may vary each time due to difference in different system conditions.
Alternative: Try to use order by clause in your queries wherever applicable. (Remember that there is a performance penalty associated with the order by clause)
Paying it forward
Poverty is one of the the biggest problems human kind is facing. Even though one part of the world is enjoying the overwhelming economic growth, another part of the world is dying of the lack of food and water. We all must do something to help eradicate poverty once and for ever from the face of earth.
This month we have seen a lot of websites trying to help eradicate poverty by many means. Here is what I am going to do:
I will provide up to 10000 MB of hosting space for any non-profit organization that works towards eradicating poverty. There is no limit in the number of organizations I will support. I will give away the hosting space till the 10000 MB quota is reached. There are some conditions though, but most of you will not be having problems with them. Contact me for more on this.
If you know about any organization in need for free hosting space, let me know by mail (niyazpk@gmail.com) or by a comment below.
Even if you do not know about any organization/website, you can still help this cause. Just spread the news by mail, blog or word of mouth.
Thanks in advance.
Towards artificial intelligence
The last week was a significant milestone in the field of Artificial Intelligence. A robot named Elbot came very close to be the first real intelligent piece of software in human history. Elbot won the first prize in the Loebner Artificial Intelligence competition. The Loebner prize is given for the bot that can make the most number of people think that it is a human.
Turing test says that if a robot can fool a human judge to believe that it is human, the robot is intelligent. The robot must successfully do this in at least 30% of the cases. Elbot made 25% of the judges believe that it was a human. It is a quite impressive feat and we can hope that in the coming years somebody will make a robot good enough (intelligent enough?) to pass the Turing test.
Turing says his paper:
We may hope that machines will eventually compete with men in all purely intellectual fields. But which are the best ones to start with? Even this is a difficult decision. Many people think that a very abstract activity, like the playing of chess, would be best. It can also be maintained that it is best to provide the machine with the best sense organs that money can buy, and then teach it to understand and speak English. This process could follow the normal teaching of a child. Things would be pointed out and named, etc. Again I do not know what the right answer is, but I think both approaches should be tried.
For testing the quality of Elbot, I tried to chat with it. Excerpts:
Elbot: Well, this is indeed a surprise. A visitor from India!
Me: Good evening
Elbot: A big hello to you, as well.
Me: How are you?
Elbot: I’m feeling wonderful today, although I admit it may just be psychosomatic.
Me: And what does psychosomatic mean?
Elbot: I don’t know what I’m saying. These are all just random words.
Me: So you are just like me huh?
Elbot: I’m willing to admit you know more than I do.
Me: Then… tell me something interesting.
Elbot: It would be nice if you could tell me something, such as a fairy tale about robots.
Me: I don’t know much about robots
Elbot: I hope that my artificial intelligence is doing better.
….
Me: How old are you?
Elbot: How old do you think I am?
Me: 2 years?
Elbot: I have just been put together! Some of my screws still need to be tightened!
….
Me: What is 2 + 5 ?
Elbot: I have computed that before. The answer is: 8!
Me: You are pretty good at math !!!
Elbot: Why, thank you very much. I like you, too. If I could have a pet you would be my first choice.
Me: I thought you would be my pet, not the other way round.
Elbot: Thinking isn’t everything.
You too can chat with Elbot here.
The truth about ignorance
We all know a lot about a whole lot of things.
We know a little about a lot of other things too.
We have heard of a lot of stuff that we don’t know anything about. i.e, We all know that there are certain stuff that we don’t even know about. These things are a billion times more than the things we know about.
Then there are stuff that we don’t know; and we don’t even know that we don’t know about those stuff. Believe me – these things are a billion times more than the stuff you think you don’t know about.
Scary indeed !!!
The Dunning-Kruger effect says something along the same lines:
- Incompetent individuals tend to overestimate their own level of skill.
- Incompetent individuals fail to recognize genuine skill in others.
- Incompetent individuals fail to recognize the extremity of their inadequacy.
- If they can be trained to substantially improve their own skill level, these individuals can recognize and acknowledge their own previous lack of skill.
The next time you think you know all (or nothing) about something, think again.
The funny caching problem
There are many situations when solutions to problems they are supposed to solve only aid in magnifying the problem. Here is one case I came across.
There was an application that queried data from the database. It was working fine till somebody got the idea that if the data coming from the database is cached in the application, the performance of the application can be improved manifold. And guess what? They implemented the idea and now they are reportedly having unprecedented performance issues in the application.
Caching is supposed to increase the performance of a system when there are a large number of hits in the cache. In our case, caching degraded the performance of the application as time progressed. The reason? The database contains a large number of tables and records and our small cache could not handle such large amounts of data. As a result, the hit rate of the cache is very low.
So when to use caching in a system?
There are many things to consider before you implement caching in your application. The main points are:
- What are the trade-offs? How much memory should be allotted to the cache? How much processing power will the cache take? Will these degrade the performance?
- What will be the cache hit rate? If the hit rate of the cache is less than a threshold, using a cache may in fact reduce the performance of the system.
- Are there any other bottlenecks in the system that can be addressed before the cache is implemented? Instead of focusing your efforts in designing and developing a caching system for your application, it may be logical to search and remove any other bottlenecks that may be degrading the performance.
Hope these points will come handy for at least some of you. If you have other points to add, please do.
