According to Wikipedia, there are some disadvantages of Crowdsourced Testing that must be handled to enable a successful result. Although this may be true, I personally regard this new way of testing as necessary. As of today, many websites and applications depend on network access and communication with other applications as well as servers. Verifying all use cases is practically impossible due to the impact of different operator’s network configurations, routers etc. Additionally there are obstacles, on the one hand practically solvable but on the other costly, such as the 10 000s types of devices and demands on support of numerous languages. For instance, consider how Candy Crush, produced by King, must be tested. Furthermore, all of us living in Escania can probably agree on the never ending flaws of Skånetrafiken’s application.
The first issue raised at Wikipedia is Confidentiality. Naturally, a growing number of involved individuals results in an increased risk of information leakage. When it comes to Applause, all testers will sign a general NDA. Moreover, the customer’s proper NDA may be signed before the tester has any knowledge of the features. All of this is managed automatically within the platform of Applause.
The next concern is the communication between all the testers. Applause has solved this by hiring project leaders, managing the test project. Through the platform of Applause, direct communication with the testers take place. Also, the customer has access to this platform in case they want to be involved in the complete test cycle.
The characteristics of reported bugs mainly depends on the payment model. Wikipedia sheds light on that a straightforward payment per bug, will result in testers identifying the simplest bugs (simple as in quick to detect and record). However, Applause has taken several measures in this matter. Firstly, testers are rewarded according to the value of each bug report. The customer will determine the value of each bug report. Secondly, proficient testers are rewarded a higher amount per bug report, and the best bug report may also yield a bonus. Moreover, duplicates of previously reported bugs will render a negative effect in the ranking linked to each tester. An all too low ranking will result in the tester not being invited to commercial test cycles, instead going back to basics trying to improve the ranking again.
Diverse time zones, cultures etc. requires an extended commitment from the project management. Through the experienced Project Leaders and Test Team Leaders of Applause, customers receive well documented test cycles concluded in a few days. Test cycles performed 24/7 decreases test time further, thus enabling a reduction of the time to market.
The last topic mentioned at Wikipedia is the risk of a limited test coverage, due to the testing not being planned and followed up in the same manner as in Waterfall projects or Agile processes. Following Applause’s experience of this type of testing, a platform has been established and project leaders are utilized to manage the testing in the same practice as in traditional QA models.