We ran some user trials during December and January (see Anna Wojnarowska’s post of 10 January 2014). We were trialing laptops, tablets, email, collaboration and WiFi access, with a group of around 50 users.
The users gave us regular feedback on how the new tools affected their day to day working, which we are feeding into the solutions that we will take forward.
As we are now at the end of those trials, it’s a great time to share what we learned and what assumptions were confirmed by real people that work at the Cabinet Office.
What we’ll describe here falls into two broad themes:
- technology (what technology should we be transitioning to)
- training and support (how we should support the transition, and our users, in the new world)
The technology that we gave the users included: lightweight laptops and tablets; Google Apps for mail, calendar, document creation and collaboration; Evernote for notetaking and Libre Office for offline document creation. We also included a lightweight secure layer, device management, portable 4G WiFi and printers.
The most interesting discovery was how easily participants were able to start using the real-time document editing functionality in Google Docs. This was particularly apparent with one team who regularly collaborate on a briefing presentation. Normally this would mean emailing versions back and forth, combined with waiting for changes to be made, with documents inevitably getting out of sync. During the trials they were able to work on the same document at the same time, from multiple locations - resulting in higher quality output, achieved faster. While their existing technology forced them to work a certain way, the trial technology enabled them to work how they wanted, and for the better.
The downside to this came when they needed to collaborate with people outside of the trials - those people not using the same technology. This highlighted the need to make sure the applications and file formats work together, but also to think about how our users work with other departments and with people outside of the Cabinet Office.
While there was enthusiasm for Google Docs, and it was well received, many users did not see it as a replacement for the products they are used to. Users keenly felt some differences in functionality - from small features they take for granted (such as automatic capitalisation of some words), to challenges with formatting and document presentation.
The trial showed the benefits of Google Docs, but also that users would find it hard to adjust without support. Interestingly, very few users commented on not being able to use Outlook; and then only that their filing system within Outlook had no obvious parallel within Google Mail.
As well as new applications, we provided new devices. The most obvious difference was the introduction of tablet devices - both Android and iOS.
A few users reported significant benefits from the tablets: note-taking during meetings, using them to present work, carrying all their documents with them, working on the train - to name a few. However, while the devices themselves (particularly the iPads) were popular, a large number of users questioned whether they would actually help them work, either because they didn’t need to move about that much, or because they relied too heavily on the laptop-based tools which provided functionality the tablet couldn’t match. Some users found the tablet didn’t help them much - this confirmed our belief that tablets wouldn’t benefit everyone.
Training and support
We wanted to maximise the value of the training we provided, so we gave users the devices prior to the training sessions and encouraged them to try them out first, before coming to the sessions.
We’ve seen how the success of the new technology depends on how prepared people are to use it, and in this regard training is really important - both initial training, and ongoing skill and capability development.
To provide support, initially we set up an online ticket system, and had a drop-in service where people could come along and get help.
In the event, very few people came along to the drop-in desk. It’s not clear exactly why this was the case - perhaps people are used to seeking support remotely, over the phone or via email. Our subsequent investigations into the support process suggest that people prefer ‘tech bar’ or drop-in services when they want to schedule a non-urgent or training request, and actually prefer to call if they have a technical fault.
Alongside the online support portal, we had provided a feedback form. Over the course of the trial, people stopped submitting tickets through the ‘proper’ channel, and instead raised them through the separate feedback form. This suggests that in future we should maintain a single point of contact for all feedback and support, and make sure we have a good system in place to sort through these - taking the burden away from the user and making it as easy as possible for them.
In response to the poor uptake of the drop-in service, we started doing ‘floorwalks’, where one of us would go and visit the areas where the trial teams were located. Participants had the opportunity to ask the questions they’d thought of but hadn’t emailed in, and they were able to get on-the-spot fixes for the issues they’d encountered. This approach worked really well, and is something we’re looking into for the roll-out and beyond.
Finally, we noticed how our users would always turn to each other for support, first and foremost. This is something we’re going to encourage, and we’ll think of ways we can support this - for example, with designated super users, community support tools, and recognition for people who are consistently helping their colleagues.
The end of the trial
As we had predicted, our users were extremely reluctant to return any of the equipment! Some of the clearest evidence of the difference the technology made came when we asked for the kit back - without exception, every team on the trial asked to keep the equipment beyond the trial end date.
Overall, we’re very happy with what we achieved during the trials. The uptake we’ve seen and the feedback we’ve received has been really encouraging. We’ll take all of this forward to shape how and what we roll out.