Story image

Crowd testing in the Digital Age - why is it growing so fast?

11 Jul 2016

Crowd testing is becoming a popular tool for ensuring application quality in digital organisations. A 2015 Accenture report identified that 43% of European CIO’s are either currently using crowd testing or planning to do so in the next two years. That’s quite a growth signal! So why is it becoming popular in our ‘Digital Age’? Simon Anderson, Assurity Consulting’s Head of Platforms, explains.

Crowd testing is a crowd-sourced approach to software testing. Simply put, a crowd platform in the cloud is used to match organisations that need testing services with a crowd of people willing to test.

It’s not a new concept – Jeff Howe was credited with the term in a seminal 2006 Wired magazine article. Howe found many examples of organisations engaging a crowd and creating mutual benefit. It didn’t take long for someone to apply crowd to software testing. uTest, which began in 2007 and has since been renamed ‘Applause’, was the first. Many more have followed.

To date, the uptake by organisations has been modest. That’s mainly because crowd testing has obvious challenges in security, ensuring crowd behaviour and the vetting of identity and skills. But platforms and processes have improved vastly and the market is seeing this. But there’s one more fly in the ointment for crowd testing and it’s about how it’s been marketed; early strategies positioned crowd as a replacement for in-house and out-sourced testing, rather than a complementary service.       

This story explains a lot! An associate of mine holds a senior quality role in a large organisation with a highly complex IS architecture. He was visited by a rep from a crowd-testing company who stated that all the department’s testing needs could be met remotely by the crowd. At that point in the conversation, the representative lost all professional credibility.

The maturing of crowd platforms, services and processes – combined with its positioning as a complementary service – is painting a brighter future for crowd sourcing and crowd testing in particular. Gartner believes its time is now because “the on-demand driven world we live in and the overall impact of consumerization and new technologies affect the requirements on applications’ quality and testing in organizations”. Gartner sees “crowdsourcing services meeting the increased pressure on the internal quality departments in organizations”.

At Assurity, we agree with Gartner, especially in areas of digital testing where traditional teams are not well-positioned to cover testing across multiple devices and operating systems. Just as important is that measuring consumerisation – and the quality of it – is best done through real customer feedback and observation.

The crowd’s big advantage, of course, is the diversity of feedback from a specific demographic that matches customer profiles. Talk to a customer-like crowd that owns and uses your applications and you’ll build a much more relevant picture of use in real life. You’ll see a crowd using your apps in unexpected ways, places and times. You’ll learn some things you never knew about.     

The key, of course, is to engage a crowd that best fits your organisation’s needs. A crowd does not comprise of ageless, asexual, unemotional automatons. Nor are its members cultureless, without context or place. If your applications are intended for New Zealanders, then you need New Zealanders to test them.

At Assurity, we’re focused on developing that New Zealand crowd. We have members who are already professional testers and members who aren’t. Most crowd engagements consist of both. We’re building a crowd community that helps members learn from and share experiences with each other. We strongly believe that the best and most useful crowds are those who are motivated by more than just financial reward – they’re people who want to share knowledge, grow and feel good about helping New Zealand organisations.

In developing our own Crowd Testing service, we ran some pilot engagements, including one for a transformation project for a large IS organisation. Following the engagement, the Digital Test Lead believes that “crowd is not really about ‘does it work?’, it’s more about ‘what does our customer think?’”. There’s much more value in that, of course. And it’s why we think crowd testing has a bright future here.

Apple's AirPods now come with 'Hey Siri' functionality
The new AirPods come with a standard case or a Wireless Charging Case that holds additional charges for more than 24 hours of listening time.
NZ investment funds throw weight against social media giants
A consortium of NZ funds managing assets worth more than $90m are appealing against Facebook, Twitter, and Google following the Christchurch terror attacks.
Poly appoints new A/NZ managing director, Andy Hurt
“We’re excited to be bringing together two established pioneers in audio and video technology to be moving forward and one business – Poly."
NVIDIA announces Jetson Nano: A US$99 tiny, yet mighty AI computer 
“Jetson Nano makes AI more accessible to everyone, and is supported by the same underlying architecture and software that powers the world's supercomputers.”
Unity and NVIDIA announce real-time ray tracing across industries
For situations that demand maximum photorealism and the highest visual fidelity, ray tracing provides reflections and accurate dynamic computations for global lighting.
Slack doubles down on enterprise key management
EKM adds an extra layer of protection so customers can share conversations, files, and data while still meeting their own risk mitigation requirements.
NVIDIA introduces a new breed of high-performance workstations
“Data science is one of the fastest growing fields of computer science and impacts every industry."
Apple says its new iMacs are "pretty freaking powerful"
The company has chosen the tagline “Pretty. Freaking powerful” as the tagline – and it’s not too hard to see why.