In-depth

To Beta or not to Beta?

Software beta testing could be dying out, or at least mutating into something unrecognisable.

When is a beta not a beta? When it's being used in production! The meaning of the term 'beta test' is changing, and perhaps even disappearing - and it's customers, alongside software developers, who are to blame.

Beta versions of Vista have been out in the field for months and Microsoft is already releasing security patches for them. "It's utterly off the wall. Surely the whole point about a beta is that you play with it, you feed back to the vendor, and they then release the final code," says Ken Munro, managing director of penetration testing cvompany SecureTest. "In the past, betas were controlled programs with privileged access. Now, people are rolling out betas for everything," he adds. "You end up with these operating systems in beta, running out in the entire world. What if there's a worm?"

There are 3.5 million beta testers running Microsoft Office 12, says Microsoft Office product manager Darren Strange. "Only 100 [early adopter] customers are allowed to use it in production," he argues. "Our advice to people is that you shouldn't be running it on your production machine. So that if your email goes wrong, you could always go back to yoiur live machine." So theoretically, just shy of 3.5 million people are running Office 12 on a second machine sitting along side their other PC. Yeah, right.

Web-based beta The situation is the same, if not worse, with Web-based applications. Google News was in beta for four years, Gmail is still a beta.

Advertisement
Advertisement - Article continues below
Advertisement - Article continues below

As there is no online distribution, the notion of software versioning becomes even more arbitrary and the idea of 'just in time programming' - where the line between development code and live code blurs or disappears - becomes more commonplace.

"A lot of the development environments created for the just in time software model were not bult with the same level of security and robust development procedures, so we're starting to see a lot of vulnerabilities related to that. Some of them are in the frameworks themselves, and some of them are because just in time software develompent doesn't lend itself to secure development," says Vincent Weafer, director of development for security response at Symantec. "These frameworks are designed so that you can say at any point, 'I'm done'," he adds, describing a 'fix it tomorrow' ethos among some web programmers. "You find a lot of issues with web development and sloppy programming."

Bridging the gap with dynamic web applications As Ajax and rich Internet applications continue to evolve, room for vulnerabilities could grow, warns SecureTest's Munro. For example, if most of the application logic is located on the client, it becomes more tempting for sloppy server programmers to forego proper back-end data validation, and assume that it is all being done in the browser. Some may forget that JavaScript is hackable, and Flash files can be decompiled. If 'beta' software compromised in such a way is available for all to use, such vulnerabilities could have widespread effects.

The marrying of client-side software and Internet distribution also muddies the waters. If you can easily update software online at any time with post-release patches, then the whole concept of software versioning becomes more interpretive.

What's the difference between Microsoft releasing software patches for Vista in beta, and the inevitable patches that will appear afterwards? "If we're posting patches to beta, I guess there are just more of them," shrugs Strange.

Beta enterprise software But even in B2B software development where the user base is more controlled, the concept of beta is shifting. Agile software development methodologies, which have taken off considerably in the past five years, have altered the nature of testing, points out Mike Beedle, chief executive of software development consultancy e-Architects and an original author of the agile manifesto.

Advertisement - Article continues below

Traditionally, products were unveiled to beta testers after an internal development reached a certain stage of maturity. In agile methodologies, the testing is married to the development at a much earlier stage, so that customers get to see very early versions of the product. "You do everything all at once. You do a requirement analysis, design, development and testing all in one iteration," Beedle says, describing 'sprints' - bursts of development on specific elements of a software application often lasting around 2-4 weeks, which bring together customers and developers all the way through the process. "Every sprint is self contained. There's no such thing as a testing sprint."

If business to business and business to consumer software developers are all changing the notion of beta testing, then we can assume that the idea is essentially dying off. In agile environments, it is becoming a more integral part of the development process, while in consumer environments, the line between evaluation and productive use of a product is blurring to the point where the decision to move from 'beta' to 'live' is becoming increasingly arbitrary. Agile developers are changing beta testing concepts in a structured way. Other software developers riding this wave must ensure that they cling to strong development principles and don't throw software quality out with the versioning bathwater.

Featured Resources

How inkjet can transform your business

Get more out of your business by investing in the right printing technology

Download now

Journey to a modern workplace with Office 365: which tools and when?

A guide to how Office 365 builds a modern workplace

Download now

Modernise and transform your sales organisation

Learn how a modernised sales process can drive your business

Download now

Your guide to managing cloud transformation risk

Realise the benefits. Mitigate the risks

Download now
Advertisement

Recommended

Visit/security/internet-security/354417/avast-and-avg-extensions-pulled-from-chrome
internet security

Avast and AVG extensions pulled from Chrome

19 Dec 2019
Visit/security/354156/google-confirms-android-cameras-can-be-hijacked-to-spy-on-you
Security

Google confirms Android cameras can be hijacked to spy on you

20 Nov 2019

Most Popular

Visit/mobile/28299/how-to-use-chromecast-without-wi-fi
Mobile

How to use Chromecast without Wi-Fi

5 Feb 2020
Visit/security/34616/the-top-ten-password-cracking-techniques-used-by-hackers
Security

The top ten password-cracking techniques used by hackers

10 Feb 2020
Visit/operating-systems/microsoft-windows/354739/windows-7-bug-blocks-users-from-shutting-down-their-pcs
Microsoft Windows

Windows 7 bug blocks users from shutting down their PCs

10 Feb 2020
Visit/hardware/354723/coronavirus-starts-to-take-its-toll-on-the-tech-industry
Hardware

Coronavirus starts to take its toll on the tech industry

6 Feb 2020