Geoff-Hart.com:
Editing, Writing, and Translation

Home Services Books Articles Resources Fiction Contact me Français

You are here: Articles --> 2011 --> Getting feedback on our work
Vous êtes ici : Essais --> 2011 --> Getting feedback on our work

Getting feedback on our work

by Geoff Hart

Previously published as: Hart, G. 2011. Getting feedback on our work. <http://techwhirl.com/columns/getting-feedback-on-our-work/>

In November 2009, I attended an STC Montreal presentation by John Druker, from SAP Labs Canada. Like many of us, John struggles with the task of getting feedback from his readers about the documentation that he and his colleagues produce. Unlike most of us, John has made a considerable effort to find ways to get that feedback and incorporate it in his work. This article is based on John's presentation, supplemented by my personal experience. John works primarily with internal clients (SAP employees), but also with occasional external clients and paid consultants; these clients are a diverse group, ranging from neophytes to experts and including people from the many national cultures in which SAP works. But his general goals for feedback are relevant to technical communicators working in any situation: getting constructive criticism on the work he's producing, its format, and the overall process being used to produce the work.

Although any feedback is valuable, the constructive type is most valuable because it's actionable: it gives you insights into how you can improve what you're doing. It must also be something that becomes part of the information development process so it can be used to continuously improve that process and its outputs, rather than being a post facto way to fix problems after it's too late to mitigate their impacts. By obtaining actionable feedback and integrating it with the information development process, it becomes possible to use feedback as a means of ongoing improvement rather than a way to make ad hoc fixes.

Tools for getting feedback

John's group at SAP uses surveys, interviews, round tables, and Web 2.0 tools to gather information from their audience: For all of these methods, it's important to remember that different jurisdictions have different laws related to privacy and the preservation of a respondent's anonymity. Anonymity can be particularly important for practical rather than purely legal reasons: people are more likely to respond honestly if they feel that nobody will be able to harm them for expressing their opinions. It's also important to understand that most people are not inherently experts in any of these types of information gathering, and that some training will be required before you begin gathering information. This is particularly true if many people will be involved as (say) interviewers, since the training will provide intellectual and other tools that both let people do their job right and help them to provide more consistent and usable data.

Planning the campaign

To gather feedback most effectively, you should plan a "campaign" (a planned and carefully organized activity, not something ad hoc). Planning should start with a consideration of which audience members to prioritize, how to motivate them (get "buy in"), and how to get the feedback early enough to make a difference. In forums such as techwr-l, I've dealt with the latter point from the perspective of asking for feedback early in the documentation development process so that what you learn early can make your subsequent efforts more efficient; any problems you identify and correct early in the process are problems you won't repeat (and therefore won't need to correct repeatedly) later in the process. John described a familiar three-step process for such campaigns:

If you have a multilingual or multicultural audience, it's important to keep differences among these groups in mind. Apart from the cultural issues I described earlier, it's worthwhile finding ways to gather information in the native language of the participants. This will increase the accuracy of the feedback because participants make fewer interpretation errors while deciphering your questions, and fewer errors while choosing words with which to frame their response. Both factors will also help them feel more comfortable participating, and seeing that you're interested in helping them to respond (i.e., by translating your questions into their linguistic and cultural terms) will motivate them more strongly to participate.

Before deciding what questions to ask, you must decide what things you're trying to improve. Typical questions focus on the quality of the documentation, including its ease of use; these are most relevant if you primarily want to do a better job of what you're already doing. But if you're more interested in whether you're doing the right things, you might want to ask questions about how frequently people use your documentation, what parts of it they use, and what specific problems they encounter while using it; this will tell you whether you're wasting your time creating information that people never use and missing opportunities to provide information that they'd like to use (more comprehensive documentation). My article Accentuate the Negative provides some insights into this process.

One interesting point that John raised is what's been broadly described elsewhere as "corporate memory". In his own feedback campaigns, he's often found that experts in a technology tend to create their own documentation and their own work patterns, and stop using the standard documentation that comes with a product. This information is important knowledge both for newcomers, who can benefit from the hard-won knowledge of experts, and for when you're creating documentation for experts, who are likely to have similar needs to experienced users. When I proposed that this kind of thing is precisely what wikis are good at capturing, and that SAP should find ways to motivate these experts to capture their knowledge, John agreed that this was something they'd been thinking about at SAP. The notion of creating tag clouds, as in Delicious, would also be relevant here.

A careful consideration of how you'll compile and analyze the data you gather should be an important part of your planning. For example, an interview process may be well-designed and may provide exactly the answers you need, but if it takes prohibitively long to extract those facts from the interview transcripts, you may have to choose a different approach. Holding one or two interviews may reveal which of your questions are most useful and what questions you forgot to ask, and you can then ask the most effective questions via an automated survey that will gather responses into a database for ease of analysis. The pre-test phase of your feedback campaign will help you test your process to ensure that you obtain usable data and that you can gain this data in a reasonable amount of time. Triage may be necessary; you can always return (as in the third step of the campaign) to gather more details if something important turns up during your analysis.

Summarizing and presenting the results

When it comes time to report your feedback, don't just present reams of charts and tables: if you want your manager (or other stakeholders such as authors and editors) to read the information and give you permission to make changes or even if you want them to change their own behavior, turn the data into a compelling story. Start with a concise, clear explanation of the context and the problem, then provide the most important data that demonstrates the problem is real. Then recommend potential solutions, and provide data from your results that support those solutions.

Because feedback should be an ongoing part of your process improvement, try to find ways to make the process "sticky". Here, John used "sticky" as a marketing buzzword that can be understood most simply as finding ways to make people "stick to it" (i.e., to continue being willing to participate). Making it easy to participate in your feedback process (e.g., create short surveys or interviews, delivered at times that are convenient to participants) and desirable (e.g., by providing rewards and personal interaction) are good starting points. But perhaps more important is giving participants the sense that you're actually listening to them and doing something about what you hear. People will naturally stop participating if they see participation as a waste of their time. But if they see changes being made that make their lives easier or more enjoyable, this will provide a strong incentive to keep participating. Even when you can't make any changes, explaining why may encourage them to keep participating. So always provide follow-up after you've obtained feedback: thank the participant, and tell them the results of your work (what you're planning to do), and offer opportunities to continue providing feedback to improve the results.


©2004–2017 Geoffrey Hart. All rights reserved