User data can be dangerous. Whatever the user supplies as data, especially in a web application, cannot be assumed to be safe. On the contrary, there are many malicious users who try to exploit every security vulnerability in your application. XSS, CSRF, SQL Injection attacks are familiar to most of you. (If not, go figure it out and come back fast.) In order to protect your application from such attacks you need to sanitize user data so that it does not do anything harmful to your system.

A big question being discussed vigorously in the web development community is:

Where to sanitize the user data? Should it be done in the input stage where the data is being entered by the user or in the output stage where the data is being displayed to the user?

The solution, in my opinion, (and in the opinion of a large group of experts in this field) is to do dual sanitization. One validation and SQL escaping before going into the database and one sanitization (filtering and escaping) before going to the output.

So the process essentially boils down to validation in the input and escaping in the output. Here are the reasons why you should go by this method instead of escaping and sanitation in the input alone:

1. The way data needs to be sanitized depends on the context the data is intended to be used. For example, if the data is to be stored in a database, we need to escape the ‘ character to prevent SQL Injection attacks. If the data is to be displayed in the HTML output, we need to escape the < and > characters to prevent XSS attacks. In the input stage we cannot anticipate the ways in which the data is going to be used.  So it is better to sanitize the data just before the output stage when it is clear where the data is going.
2. You cannot always be sure that the data in the database is sanitized data. You cannot guarantee that it came from the sources we anticipated the data to come from. There is a chance that the data ended up in the database through a path where you have not placed your input sanitizer. What if a user directly edited the database to add some data? What if there are loopholes in your sanitizer? What if the data was placed by an SQL injection attack against your database? All these points tell us that we need to sanitize user data where it is being used – that is in the output stage.
3. There may be other applications which use the data from your database. For example an application written in COBOL may be using the data from the database to generate some reports with it. If the data already in the database is in the form of &gt;script&lt;&nbsp;hello&nbsp;world, the COBOL application will not able to make sense out of the data. It will have to implement its own decoder to read the data. This is a very painful process. We can avoid situations like this if we do not push processed data into the database.
4. It is always best to have pure unaltered data in the database so that it can be easily processed by all the applications using the data. Once we sanitize the data before it is stored in the database, there is no going back. It is really hard to get the original data supplied by the user back after doing all these filtering and escaping techniques. On the other hand, if we have unaltered data in the database it is easy to escape it later with respect to each application using the data.
5. According to the above points, data sanitization in the output is anyway needed for obvious reasons. If we are encoding the user data in the input as well as in the output, the data will be in a doubly encoded form and it will not be useful at all. There is no need for double sanitization anyway. So it is always recommended to encode your data to the target format just before passing the data to the target system.
6. Users have reported security holes with applications like phpMyAdmin when it displays database values without encoding to HTML format. The developers of phpMyAdmin anticipated the data in the user databases to be free of any malicious code, but it may not be the case. So your application needs output sanitization especially if you are using data form outside sources. Never trust any data coming your way.
7. Assume that you are using input sanitization. If there is some bug in the sanitizer, malicious data will creep into the database and now you have to fix the sanitizer and remove all the malicious data from your database. This can be a very tedious job. But if you were using output sanitizer, you just would have to modify the code to fix the security hole.

So how to do this two step sanization? Here is how:

1. User data comes in
2. Validate the data
3. If valid, do SQL escaping and store in the database. (mysql_real_escape_string( ) in PHP)
4. If invalid, reject the data. Don’t try to modify the data and push it into the database. This will do more harm than good. The user will think that the data went through successfully while the data in the database will be something else. So just accept or reject the user data. Don’t try to alter it.
5. Output: If the data is going to an HTML page, escape for HTML. (htmlentities( ) in PHP). If the data is going to a unix command line, escape for shell.(escapeshellarg( ) in PHP). If the data is going to a URL, URL encode the data.(urlencode( ) in PHP) etc.

In the validation step, check for the proper encoding of the data - URL/UTF-7/Unicode/US-ASCII etc. Then check if the data contains proper character-set. Allow only the characters which are really needed for the application. Put a limit the length of the input data. Remember that an attacker usually makes use of long strings to craft an attack. Check whether the data format is correct or not. Phone numbers should contain only numbers; email addresses should contain text in the specific email format etc.

Always use the methods or frameworks provided by your language/platform to do the escaping and encoding/decoding. Most of the languages out there support these operations. Java is an exception though: when you are using Java, you should write your own methods handle HTML encoding/decoding.

Finally, when sending the data to the web browser, remember to set the proper encoding for the web page. This can be done using the response header attribute or using meta tags. It is advisable to use both methods. Forgetting this step can aid some types of XSS attacks.

Other concerns

Some websites need to output user input as HTML itself – for example websites that allow HTML editing. In this case you cannot do encoding in your application. Remember to add proper filtering mechanisms to allow only the tags that are intended to be used. Always block potentially dangerous tags such as <script></script>

Read more at:

• Safe String Theory for the Web : Steven Wittens
• Input validation or output filtering, which is better? : Jeremiah Grossman
• Filter Input-Escape Output: Security Principle and Practice : Terry Chay
• Secure input and output handling : Wikipedia
• Web security: Google Code

I am increasingly coming across people who think robots.txt file can be used to prevent search engine crawlers from crawling sensitive data in their websites. Seriously.

This is just plain wrong. Data to be excluded using a robots.txt file is: unwanted, redundant or useless data. An entry in the robots.txt file cannot protect your sensitive data from going out. Sensitive data should not be left open in your website in the first place.

There are many malicious crawlers which crawl only the pages blocked by the robot.txt file in every website. I bet many interesting stuff will turn up in their search results.

Your choice of passwords shows how important the data secured by the password is. If the password for your email account is passw0rd, it means that the data in your email is not important enough (or that you don’t care much about the importance of the data). We all know that it is generally not a good idea to store user passwords in your database in the form of plain text. But in certain cases, you may be compelled to store user passwords directly in the plain text form in the database.

This means that all the user passwords are secured by a single database password. If someone brute forces the database password, he/she can read all the user passwords, and users as we know, are infamous for using the same passwords for most of their accounts everywhere. Thus the attacker can access other accounts of the user elsewhere.

As a developer/ DBA, it is your job to secure the data in your database. One thing you can do is to make the brute forcing of the database password as hard as brute forcing all the passwords in the database. This can be done by choosing the length of your database password wisely.

What should be the length of your database password?

Minimum length of database password = Average length of user passwords contained in the db *  log2 (number of user passwords) / 8

For example, if you are storing 1024 passwords of average length 8 characters, your database password should be at least 10 characters in length.

Any system will leak information in every possible way if we don’t carefully plug the holes and design the system.

The last day I was attending a telephone conference. The host dialed the phone number in a big conference enabled phone which made big beep sounds for every key he pressed. The IVRS directed him to enter the password for the system. He dialed the long password and looked very happy that he had a very good password for the system. Meanwhile, I calculated and wrote down the password in my notepad.

How did I do that?

Simple! When he dialed the numbers, the phone beeped according to the keys pressed. If you have slight experience in recognizing the sounds, you can decode the message that is being sent.

You can do this for virtually every mobile phone if “Key Tones” are enabled. You can recognize messages sent, passwords entered and numbers dialed this way.

The moral: Remember to switch off the fancy Key Tones in your phones and computers.

All of us know the importance of privacy. Privacy is one of the corner stones of trust and security in any business, online or not. We all try to ask for privacy in the services we use in day to day life and every company worth their salt try to impress customers with their privacy statements. However it looks like neither the the authorities of a state nor a very prominent bank in India does know even the basics of privacy and security.

What you are looking at is the complete details of the voters list in the state of Andhra Pradesh. Complete details means every single thing the state knows about you.

Anyway I will tell where I got this data from. I did not hack into the secure servers of AP. I did not crack the pass-phrase of the DBA. Apparently the authorities sold their waste papers to a vendor. Incidentally this vendor made paper plates using these and sold it a local cafe. I got the paper plates whenever I ordered something from there.

I took some more samples from there. Then came another shocking truth. Among the data there was data corresponding to the details of the customers of a famous bank. Looking at the samples it was obvious that the papers contained data of all the customers of the bank from all parts of India.

I don’t know how to react to this. Just wanted to let you know that the next time you are doing business with anyone, ask them what they do with their waste papers.

Cross-site scripting (XSS) and SQL injection attacks are real and growing threats in the web. Malicious users try to exploit any kind of security vulnerability they find in web applications. That is the reason why every single input field in your web input form must be validated.

Many web developers ignore validating the data in drop down lists, option buttons, check boxes etc because these form controls can have only a predefined set of values. Usually these set of values are provided by the server. So the developer assumes that the value returned to the server from the user will also contain any of these pre-defined values, and hence they need not be validated. This is a serious mistake.

Intentionally or by mistake, a user can send any data to your server. User can manipulate the data in your web page and return values that were not anticipated and thereby exploit your system.

See an example:
Below is a drop down list:

The allowed values are 100,200,300 and 400.

If you want to add a new value 500 to the list, it is pretty simple. There is a great plug-in for Firefox named Firebug. Using Firebug you can edit virtually any value in the DOM of the webpage.

The result – a malicious user can do cross-site scripting attacks or SQL injections attacks against your website if you are not doing proper validation of input data.

No amount of client side validation can substitute for server side validation. Client side validation is used for giving back error messages to the user to notify him of any error before submitting to the server. Once the data is submitted to the server, it must be validated and any potentially dangerous data must be stripped off.

The rule of thumb must be:

Every piece of data coming from the user must be strictly validated.

Simple. Isn’t it?

Security measures which are just to show-off security and which does not provide any real security are called security theatre. The problem with security theatre is that it will give you a false sense of security and at the same time, do nothing to increase your security. Worse, they may reduce your security. Here are the examples:

1) Fake TV

What is it?

FakeTV accurately simulates the light output of a real television. The effect of scene changes, fades, swells, flicks, on-screen motion, and color changes look just they came from a real TV. From outside the house, it looks just like someone is watching a real television. The potential burglar thinks the home must be occupied, so he moves on to an easier target.

Here is a tip for burglars: Get a list of people who bought Fake TV. Most probably they will be away from home frequently. You know what to do next.

2) Liquid Ban

Reuters reports that all public swimming pools in Olympic co-host city Shanghai will check shampoos, body wash and other liquids before allowing entry to guard against explosions or other “terrorist attacks”.

Pool guests who bring these items must allow them to be opened and inspected. Security personnel will smell them to see whether they are safe or not.

Schneier thinks this is a stupid idea.

Here is a tip for cheaters: Bring something like chloroform with you. Guards will smell it and faint. You are then free to do anything you want.