Story image

Crowd testing in the Digital Age - why is it growing so fast?

11 Jul 16

Crowd testing is becoming a popular tool for ensuring application quality in digital organisations. A 2015 Accenture report identified that 43% of European CIO’s are either currently using crowd testing or planning to do so in the next two years. That’s quite a growth signal! So why is it becoming popular in our ‘Digital Age’? Simon Anderson, Assurity Consulting’s Head of Platforms, explains.

Crowd testing is a crowd-sourced approach to software testing. Simply put, a crowd platform in the cloud is used to match organisations that need testing services with a crowd of people willing to test.

It’s not a new concept – Jeff Howe was credited with the term in a seminal 2006 Wired magazine article. Howe found many examples of organisations engaging a crowd and creating mutual benefit. It didn’t take long for someone to apply crowd to software testing. uTest, which began in 2007 and has since been renamed ‘Applause’, was the first. Many more have followed.

To date, the uptake by organisations has been modest. That’s mainly because crowd testing has obvious challenges in security, ensuring crowd behaviour and the vetting of identity and skills. But platforms and processes have improved vastly and the market is seeing this. But there’s one more fly in the ointment for crowd testing and it’s about how it’s been marketed; early strategies positioned crowd as a replacement for in-house and out-sourced testing, rather than a complementary service.       

This story explains a lot! An associate of mine holds a senior quality role in a large organisation with a highly complex IS architecture. He was visited by a rep from a crowd-testing company who stated that all the department’s testing needs could be met remotely by the crowd. At that point in the conversation, the representative lost all professional credibility.

The maturing of crowd platforms, services and processes – combined with its positioning as a complementary service – is painting a brighter future for crowd sourcing and crowd testing in particular. Gartner believes its time is now because “the on-demand driven world we live in and the overall impact of consumerization and new technologies affect the requirements on applications’ quality and testing in organizations”. Gartner sees “crowdsourcing services meeting the increased pressure on the internal quality departments in organizations”.

At Assurity, we agree with Gartner, especially in areas of digital testing where traditional teams are not well-positioned to cover testing across multiple devices and operating systems. Just as important is that measuring consumerisation – and the quality of it – is best done through real customer feedback and observation.

The crowd’s big advantage, of course, is the diversity of feedback from a specific demographic that matches customer profiles. Talk to a customer-like crowd that owns and uses your applications and you’ll build a much more relevant picture of use in real life. You’ll see a crowd using your apps in unexpected ways, places and times. You’ll learn some things you never knew about.     

The key, of course, is to engage a crowd that best fits your organisation’s needs. A crowd does not comprise of ageless, asexual, unemotional automatons. Nor are its members cultureless, without context or place. If your applications are intended for New Zealanders, then you need New Zealanders to test them.

At Assurity, we’re focused on developing that New Zealand crowd. We have members who are already professional testers and members who aren’t. Most crowd engagements consist of both. We’re building a crowd community that helps members learn from and share experiences with each other. We strongly believe that the best and most useful crowds are those who are motivated by more than just financial reward – they’re people who want to share knowledge, grow and feel good about helping New Zealand organisations.

In developing our own Crowd Testing service, we ran some pilot engagements, including one for a transformation project for a large IS organisation. Following the engagement, the Digital Test Lead believes that “crowd is not really about ‘does it work?’, it’s more about ‘what does our customer think?’”. There’s much more value in that, of course. And it’s why we think crowd testing has a bright future here.

How blockchain will impact NZ’s economy
Distributed ledgers and blockchain are anticipated to provide a positive uplift to New Zealand’s economy.
25% of malicious emails still make it through to recipients
Popular email security programmes may fail to detect as much as 25% of all emails with malicious or dangerous attachments, a study from Mimecast says.
Human value must be put back in marketing - report
“Digital is now so widely adopted that its novelty has worn off. In their attempt to declutter, people are being more selective about which products and services they incorporate into their daily lives."
Wine firm uses AR to tell its story right on the bottle
A Central Otago wine company is using augmented reality (AR) and a ‘digital first’ strategy to change the way it builds its brand and engages with customers.
DigiCert conquers Google's distrust of Symantec certs
“This could have been an extremely disruptive event to online commerce," comments DigiCert CEO John Merrill. 
Protecting organisations against internal fraud
Most companies tend to take a basic approach that focuses on numbers and compliance, without much room for grey areas or negotiation.
Telesmart to deliver Cloud Calling for Microsoft Teams
The integration will allow Telesmart’s Cloud Calling for Microsoft Teams to natively enable external voice connectivity from within Teams collaborative workflow environment.
Jade Software & Ambit take chatbots to next level of AI
“Conversation Agents present a huge opportunity to increase customer and employee engagement in a cost-effective manner."