Skip to main content

Accessibility by Crowd

Accessibility by Crowd is our dedicated service for crowd-powered accessibility testing. It allows us to leverage a diverse pool of testers, including users with disabilities, to evaluate digital accessibility at scale.

In this section, you will find an overview of how the service works, the methodologies we use, and resources to support accessibility testing through the crowd. 🚀

Activity Details:

Through the involvement of experienced testers we test the accessibility of a system, through the use of assistive technologies (e.g. screen-reader, keyboard, zoom software) to simulate the experience of users with disabilities and identify digital barriers that may compromise the accessibility and usability of the system.

The testers adopt a structured and reproducible approach focusing on key aspects such as keyboard navigation, screen reader, error and notification handling, contrasts and scaling.

Thanks to this approach, the client has all the tools to reproduce the problem independently and can apply remediation suggestions to correct it. This activity can be carried out several times, planning a roadmap of checks and fixes, in order to improve the accessibility of their digital product in an effective and verifiable manner and also improve the degree of compliance.

Perimeter:

The accessibility testing activity carried out by expert testers focuses on a well-defined perimeter, which may include the entire website or app or specific sections and functionalities critical to the user experience, based on WCAG 2.2, verifying compliance criteria at levels A and AA (or AAA, if required). The testing of websites takes place on desktop and mobile environments in the different operating systems, the testing of mobile applications involves browsing both iOS apps

Deliverable:

Each identified issue is documented in detail in:

  • Platform: through the timely uploading of violations, accompanied by screenshots and screen recordings (as the functional dashboard) - here a demo;
  • (optional / add-on) Excel report attached to the campaign on the platform: with an overview of the violations found through graphs and a timely and more usable categorization of the violated success criteria

Pricing:

  • WEB: 4300€ (Starting from) - both desktop and mobile (Test + Statement)
  • APP: 5400€ (Starting from) - both iOS and Android (Test + Statement)

Useful links:

F.A.Q.​

  • What's the difference between Crowd and Overlays? 🇬🇧 Overlays are automated solutions that do not guarantee your site is compliant. With these tests, you get a real check with human experts who use assistive technologies during the checks. 🇮🇹 Gli overlay sono soluzioni automatiche che non garantiscono la conformità del tuo sito. Con questi test invece ottieni una verifica reale con esperti umani che utilizzano tecnologie assistive durante le verifiche.
  • How long does the test take? 🇬🇧 In a few days the platform with details is ready. More or less 4 days for expert testing and then the triange managing by the Web Accessibility Expert. 🇮🇹 In pochi giorni la dashboard con violazioni e dettagli è pronta. C'è una prima fase di test di circa 4 giorni da parte dei tester esperti e poi il triage del Web Accessibility Expert.
  • What does it consist of in practice? 🇬🇧 6/8 testers navigate the system with assistive technologies, covering the different touchpoints and operating systems and reporting any violations they encounter through the bug form. These violations are accompanied by screenshots, videos, details about the violated WCAG (principle, criterion and level) and a suggestion for remediation. At the end of this testing phase, which usually lasts 3/4 days depending on the perimeter, a triage manager verifies the correctness and completeness of each violation and update the platform. 🇮🇹 6/8 tester esperti navigano il sistema con le tecnologie assistive, coprendo i diversi touchpoint (Desktop e mobile per i siti, iOS e Android per le app) e sistemi operativi e riportando attraverso il bug form le violazioni che incontra. Queste violazioni sono corredate da screenshot, video, dettagli rispetto la WCAG violata (principio, criterio e livello) e suggerimento di remediation. Al termine di questa fase di test, che dura solitamente 3/4 giorni a seconda del perimetro, un triage manager verifica la correttezza e completezza di ogni violazione.
  • Do testers use automatic tools for analysis or do they use assistive technologies to perform manual tests? 🇬🇧 Our testers use assistive technologies, actually interacting with pages through keyboards and screen readers and simulating real user navigation. Automatic tools never guarantee complete analysis coverage, this is because they cannot simulate the real user experience in complete processes (e.g. form compilation flow or purchasing process), limiting themselves to analyzing the static page in aspects such as header structure or color contrast. A manual test instead guarantees a more in-depth analysis that is closer to the real navigation of the user with disabilities. 🇮🇹 I nostri tester utilizzano le tecnologie assistive, interagendo realmente con le pagine attraverso tastiere e screen reader e simulando la reale navigazione degli utenti. I tool automatici non garantiscono mai la totale copertura dell'analisi, questo perché non riescono a simulare la reale esperienza utente in processi completi (es. flusso di compilazione di un form o processo di acquisto), limitandosi ad analizzare la pagina statica in aspetti come struttura delle intestazioni o contrasto dei colori. Un test manuale garantisce invece un'analisi più approfondita e più vicina alla reale navigazione dell'utente con disabilità.
  • Does this test allow you to write the accessibility statement? 🇬🇧 This test alone is not enough to write the accessibility statement, but it is preparatory to the drafting, which can be foreseen as an add-on. The support for the accessibility statement consists in the further verification by a Web Accessibility Expert, starting from the test results, reviewing all the mandatory success criteria of WCAG 2.2 and then proceeding to the drafting of a text document for the accessibility statement. 🇮🇹 Questo test da solo non basta per poter scrivere la dichiarazione di accessibilità, ma è propedeutico alla stesura, che si può prevedere come add-on. Il supporto alla dichiarazione di accessibilità consiste nella verifica ulteriore da parte di un Web Accessibility Expert, a partire dai risultati del test, passando in rassegna tutti i criteri di successo obbligatori delle WCAG 2.2 e procedendo poi alla stesura di un documento di testo per la dichiarazione di accessibilità.
  • In Accessibility by Crowd is there a way to visualize/track in the platform the connection between a bug and the violated WCAG guideline? And if so, how? Currently it is visible in detail at the level of single bug but it is possible for the customer to create tags on each bug to facilitate filtering. The indication of the WCAG present in the description can be exported to Excel and integrated into Jira. We can also foresee the add-on of an accessibility report in Excel that includes graphs and the main details of the violation. For the future it has been requested to have a dedicated dashboard on app.unguess in which the violated principles and success criteria are immediately evident, both through distribution and incidence graphs and as information elements in the bug list. 🇮🇹 Attualmente in piattaforma è visibile il principio e criterio WCAG violato nei dettagli del singolo bug, inoltre il cliente può aggiungere tag (come sulla funzionale) per facilitarsi il filtraggio. I dettagli delle violazioni sono esportabili anche nell'excel automatico e integrabili sui tool di lavoro dei clienti. Si può comunque fornire tramite add-on un report Excel che include grafici e incidenze delle violazioni. Per il futuro verrà implementata una dashboard dedicata con grafici rispetto ai criteri e principi violati.
  • 🇬🇧 How do expert testers test against WCAG criteria? Does each tester test against the full criteria list? Or is it more criteria-free testing, and if so, how do we crowd-source the criteria coverage? testers navigate as if they were users with disabilities thanks to the use of assistive technologies and in addition they are able to understand the nature of the non-conformity with respect to WCAG and to address suggestions for remediation. This methodology does not ensure that all violations are identified, just as the technical verification of a Web Accessibility Expert does not. A complete coverage of the 50 criteria occurs in the case of a technical verification and in the support for the drafting of the accessibility statement but even in this case it is not guaranteed that all non-conformities are identified. 🇮🇹 Come avviene il test da parte dei tester esperti rispetto ai criteri WCAG? Ciascun tester testa seguendo la lista dei criteri completa? O si tratta di un test più svincolato dai criteri e se è così come garantiamo a livello di crowd la copertura dei vari criteri? I tester navigano come se fossero degli utenti con disabilità grazie all’impiego di tecnologie assistive e in aggiunta riescono a capire la natura della non conformità rispetto a WCAG e a indirizzare dei suggerimenti per la remediation. Questa metodologia non assicura che vengano individuate tutte le violazioni, così come non lo fa la verifica tecnica di un Web Accessibility Expert. Una copertura completa dei 50 criteri avviene in caso di verifica tecnica e nel supporto alla stesura della dichiarazione di accessibilità ma anche in questo caso non è garantito che vengano individuate tutte le non conformità.