Background:   My day job involves security testing web-applications.

The team I work in uses both automated scanners, along with a few humans testing (minimum of 2)…   A good tester should know the weaknesses of the automated testers..

The problem with automated testers, is, simply put, they are not human.   That is they will not have intuition that a given function in a website is vulnerable.   When testing manually I find I get a feeling a function is vulnerable and then I concentrate on this perceived weakness..

Automated testers also only typically test some predefined vulnerabilities and although constantly being improved, they are far from perfect..

Automated testers do have their place however in greatly reducing the workload when security testing, allowing a lot of the tests which would usually take weeks to complete manually to be completed with a degree of confidence automatically.

My personal experience is they have a 25-30% valid findings rate of the vulnerabilities that we report in our assessments when setup correctly (less if not).

This is quite annoying in the context that some (thankfully not all) independant commercial testers we have encountered over the years believe an automated test is a sufficient test to charge a fortune for, and rely wholly on.   If we had done with with our internal tests we would have missed several critical “information disclosure” type bugs which were remarkably simple to spot.

Items that automated testers do NOT handle well that you should definitely therefore concentrate on in manual testing:

• Priviledge elevation’s - of all kinds.    Is the application checking priviledges in the right places correctly.     My experience is the document library is usually the weak point in any priviledge checks -  Several apps I  personally have tested recently do not check permissions when users download files…  the authentication cookies are present, but are not checked by the download functions.    Appscanners do have tools to check for privledge elevation and these also should be used (though the usability of these could in my opinion be improved).
• Authentication weaknesses - ie do ALL functions check for presence of the Authentication cookie correctly.     Some app developers take shortcuts - one recently thought simply a Referer header is a good enough method of security for one function in an application… and wasn’t checking the cookies passed at all.
• Weakness in “token generation” - that is where internal identifiers are used for security (ie msgID’s) to determine who has access to what - are those message ID’s predictable.     Generate messages in sequence and see if you can spot a pattern.    As a developer, please don’t write your own crypto if you are not a specialist in the field.    Use AES, SHA, MD5 - thus a standard well regarded library if possible.
• To a lesser extent SQL injection.    Although security test tools do concentrate on this - they have on ocassion missed things.    If you manually debug the database while trying manual SQL injection you can sometimes find things the automated tester has missed.   Often findings from this can then lead to finding XSS flaws…

For guidance in developing secure applications developers really should be looking to OWASP.   Don’t make common issues such as XSS, SQL Injection, XSRF, priviledge escalation possible - it has better guides than I could write on the subject….   The Webgoat project is also a nice way to actually learn how to test for vulnerabilities if you have never considered Web Application vulnerabilities before.
If you are seriously looking into Application security you also of course need to consider source analysis.   Thats an entire different subject.

Part of my current role (in fact the main piece now) is Web Application Security Testing. Which means I get paid to hack around with corporate and non corporate web apps (ie, Apps we buy vs app’s we build).

Web application bugs although currently looked by some to not be serious are gathering in momentum and becoming more common - only recently a lot of websites were compromised by Chinese hackers using SQL injection. XSS in particular can also be used to great effect in just one example of many to send a session cookie off site to hacker base - and thus giving them access to the logged in users data.

Over the past year I’ve tested around 40 apps in total, some complex, some simple. Major security defects have been found in all apart from one application during this time.
The fact is, regardless of the language an Web application is written in, it typically is vulnerable to one of the below 3 in my findings.

• SQL Injection
• Cross Site Scripting (XSS)
• Privilege Escalation

The above is not a full list, but its the basics and believe it or not even in 2008 SQL Injection is still the most common flaw we find! For a better view of what you should be doing to stop this, OWASP is a good website to start with.

Of these bugs SQL Injection and Cross site scripting are usually the easier to spot, and also to get developers to fix - and it is surprisingly easy to fix these first two by not trusting user input, and filtering it before it hits database in case of SQL injection, or is formatted back to users ( in case of XSS both preferably). There are good tools to test both in a semi-automated way in the form of a firefox extension here - we use this in combination with commercial tools to test for this in combination with manual testing (for SQL injection typically) with database traces running. Manual testing is far easier if you can see the queries being executed on the database (though you can’t do this in a black-box test where you have no access to remote database obviously.).

Privilege escalation however can be more tricky to both test and find in my experience at least. Its almost always a manual test - as the commerical and free tools do not do as good as job at finding this as they do the XSS/SQL bugs. In my testing I have found that some developers however still seem to think that simply hiding menu’s from a lesser priviledged user is a way to secure their application (though thankfully this is a minority!)..

One application I tested recently did at least get this right, but used a very predictable bash64 encoding to hash message ID’s within the messaging in the app (where critical data was being passed over this) They wrongly assumed the algorithm they devised was strong enough to protect themselves….. as it didn’t look predictable to the developers (who had no experience of that kind of work). The problem was they had negected to do a check when the page was loaded as to whether user had rights to read that message. That simple fix was enough to secure the app…

Where I work at least, we finally are integrating Web security at project design and initial build stages - so finally security is being taken seriously at day one (This is reducing the critical bugs found at testing) - Is this happening elsewhere?

Overall though is it not time for Web developers to take security more seriously? My hit rate on serious defects is showing that in some cases its the last thing on their mind when developing - and those that do make a effort, they sometime miss the mark leading to a nasty bug.

For information, the tools I use daily are : IBM Appscan, Paros, BURP, XSS/SQL-Inject, althugh these are not the only ones - I have a list that fills my screen - of various proxy’s, header modifiers, request modifiers etc, encryption tools etc.   I may write a blog post on a simple test methodology in the future that I use for the more simple applications tested if anyone is interested?