|
–Geoff-Hart.com:–Editing, Writing, and Translation —Home —Services —Books —Articles —Resources —Fiction —Contact me —Français |
You are here: Resources --> 2005 -->
Designing an effective review process
Vous êtes ici : Ressources --> 2005 --> Designing
an effective review process
by Geoff Hart, STC Associate Fellow, Montreal chapter
Previously published as: Hart, G. 2006. Designing an effective review process. Intercom July/August 2006:18–21.
Whether you're a writer or editor, coping with review processes is a fact of life. Unfortunately, these processes seem to spontaneously develop over the years, with no seeming logic behind their development. Even where a process has been carefully designed and developed, the process can gradually become atherosclerotic over the years. At a former employer, we used a well-defined documentation review process that had been developed over the company's 25-year existence, but that failed to achieve timely and effective reviews simply because it had become part of the wallpaper: everybody knew it was there, but no longer "saw" it and appreciated it.
How can you develop an effective review process where none formerly existed, or reinvigorate an existing process that desperately needs an extreme makeover? By understanding the key tasks in any review process, and determining how the unique characteristics of your situation will affect those tasks. In this article, I'll show you how to do this, with a very real likelihood that you'll end up with a process that is much less painful for everyone.
The essential elements of any review process are as follows:
The benefits of careful design are well known to professionals ranging from carpenters to engineers, and most writers are also familiar with the concept. Yet many writers, often because of tight deadlines that lead to skipped steps, don't take the time to properly plan the documents they're going to write. This is particularly true of professionals such as engineers and scientists who are forced to write as part of their job, but who are not primarily professional writers.
In my experience, hours of time can be saved for each participant in the review process and during each step in the review process through the simple act of planning. For writers, the obvious plan is an outline, and investing some time to develop an outline can generate significant paybacks all by itself. The problem with outlines, of course, is that if they're flawed, the resulting documents will contain those flaws, and the flaws will propagate through each subsequent phase of the review until someone spots the problem. As a result, you can generate significant return on the investment in this planning stage by rigorously reviewing the outline before anyone begins to write. At a former employer, several colleagues who were first and foremost researchers rather than writers had enormous difficulty planning what they intended to say in their reports. As a result, the reports became long, poorly organized, rambling efforts to present every thought rather than only the key points—and key points were often omitted. The result was that at each step of the review process, from writing to final review, large amounts of time were wasted trying to correct these problems.
I solved much of the problem by working with the author right from the start to carefully analyze the goal of the communication and the facts that had to be presented in the document. As the editor in this workplace, the hour or so I spent developing a strong outline saved me many more hours later in the process, generating a clear return on investment for me because the well-designed outline drastically reduced the time required to repair a poor first draft of the document. Others, including managers, quickly recognized the time savings this approach generated for them. My initial success with only a few writers formed the basis for a major change to our document creation process: the new process included a formal planning meeting during which all stakeholders (the author, the editor, the manager responsible for the information development, and the manager responsible for final approval of the document) rigorously reviewed the outline before the author began writing.
As a writer, I often joke that I have a compost pile where I store my works in progress: with a little time, what starts out as raw byproducts of the digestion process (to phrase this delicately) soon becomes valuable fertilizer. Authors often forget to set aside this necessary time that would allow them to review their own writing with some critical distance. Indeed, my favorite all-time review comment appeared on a document I received for editing, where the research director's distinctive handwriting said only one thing: "Have you read your own manuscript?" Clearly, the author had not.
Where possible, remind authors to set aside their first draft for at least a day before returning to it and beginning to revise it. Even a short time away from the manuscript provides the critical distance you need to revise it well. Encourage authors, particularly those who don't write as their primary job function, to plan to include this time in their scheduling. There will always be plenty of other things to keep them busy until they can return to the manuscript.
All publishers who care about the reactions of their readers insist on some form of quality control before they release a document for public inspection; this is generally referred to as an internal review, as distinct from the external review discussed in the next section. The internal review usually takes the form of editing in organizations that recognize the importance of well-edited writing, or of peer review in organizations that are less serious about quality or lack the budget for an editor. This step should never be eliminated, since it's the only way to catch and fix obvious problems before others encounter them. If your organization's reputation is an important asset, this phase cannot be skipped.
Internal review takes two forms. The first, which is present in all organizations, is a straightforward technical review. The goal of this review is to catch significant errors of fact or logic that would prevent the safe or successful use of a product or that would prevent or interfere with understanding of information. This review should also identify problems that make it unnecessarily difficult to understand the writing, but that is often a secondary goal. These problems lead to the second form of internal review, namely an editorial or peer review. Here, the goal is to take a technically correct document and make it more readable. This second review is less well appreciated, and many organizations have no professional editor to do this work; in that case, the writers may perform peer review of their own work, or the original writer may review and improve their own work based solely on the feedback obtained during the technical review.
Over the years, I've found that performing an editorial review before the technical review offers a large payback. In the absence of such a review, technical reviewers often spend far more time correcting grammar, punctuation, formatting, and other aspects of the writing than they spend focusing on the content. In contrast, providing a well-edited document for review removes all these obstacles to reading, allowing the technical reviewer to focus on the content—which is, after all, their area of expertise. Performing the editorial review first will not entirely eliminate comments related to the quality of the writing, but it will allow reviewers to focus more of their attention on the technical aspects than if their concentration is constantly being interrupted to correct typos. This approach increases both the effectiveness of the review (more technical errors are caught and fixed) and the speed of the review (less time is wasted fixing minor errors), and thus generates an obvious return on investment for the editor's time.
A well-done internal review can catch both major and minor problems, and can create a greatly improved product, but it has one large failing: the people who perform the internal review are highly familiar with the subject they're writing about, and thus, may not be representative of the typical reader, who will be far less familiar with the specific information. Moreover, organizations gradually tend to develop a set of standard operating assumptions that become part of their subconscious thought process and that subsequently bias how they communicate about a topic. Even experienced editors, who are experts in finding and resolving the problems that arise from assumptions and implicit understandings, are vulnerable to this kind of problem, particularly after many years spent absorbing an organization's standard practices and assumptions. The fresh set of eyes provided by an outsider external to the organization solves both problems: the reader's naiveté reveals any problems that result from the implicit assumptions made by the insiders who created the document.
External review is an essential part of some publishing processes, particularly in the sciences. For example, it's not possible to publish scientific research papers in most journals before the manuscript has been exhaustively and often ruthlessly critiqued by experts in the field. In other fields, reports being written for people who have the authority to reject a report or the activity based on the report (e.g., government regulators), and these people can be enlisted as reviewers to ensure that the final report will be acceptable. In the absence of this review, a lucrative contract may not be awarded or a key project may be stalled. In still other cases, external review is a luxury rather than a necessity: the value of the review is recognized, but because of time constraints or other problems, the review is not performed.
As a result of these concerns, a documentation manager or editor should advocate strongly to include some form of external review in the overall review process. To succeed, you must first understand the reasons why these reviews are not being performed. If the key issue is time, then careful planning (scheduling plus project management to track progress towards deadlines) may free up the time required to permit an external review. If the key issue is political, then understanding the politics behind how the organization makes decisions can provide insights into solutions; in one case, the obstacle was embarrassment over the quality of the materials being sent for external review, and the solution was to edit the materials beforehand. And if the issue is nothing more than apathy, sometimes it's possible to arrange an external review on your own authority, without asking for permission. It's often the case that managers who won't formalize the process of external review happily accept the improved quality that results from such reviews—provided that the review process doesn't interfere with meeting their own deadlines or lead to any problems that require their attention.
Whether you conduct rigorous internal and external reviews, or only perform some cursory quality control, at least one final review stage is necessary. Since it's inevitable that some changes will have been made as a result of early reviews, someone must ensure that the author has responded correctly to the review suggestions. Although this can be delegated to the author, my experience has shown that left to their own devices, authors often respond inappropriately, misunderstand review suggestions, or introduce new errors while responding to the reviews. In nearly 20 years of working as an editor, I've only rarely found an author with a perfect record; some simply ignore comments they disagree with or misunderstand, and most introduce additional errors even when they respond appropriately. (It's human nature to grow bored with a document after revising it for the tenth time, and that boredom weakens the author's focus and inevitably leads to errors.)
As a result, the ideal review process includes a final editorial review. This lets you polish the document to a fine glow and produce the maximum possible quality. But at a minimum, this final review gives you one last chance to spot any problems that have made it through all previous stages of the review process. These problems are legion, and cannot simply be assumed to have been fixed; examples include orphaned phrases such as "graphic artist: please replace this figure with a new one" and authors who, knowing that there would be no final review, took the opportunity to slip scurrilous or offensive statements into their manuscript. You can understand this part of the process by comparing it with a product assembly line: before a product is boxed and shipped to the consumer, someone always tests it to ensure that it's working right. For computers, this may involve a 24-hour "burn-in", with the product turned on for this long to ensure that it won't burn out as soon as it's plugged in, supplemented by a suite of software tests that rigorously and automatically test all key features of the installed software; for automobiles, it may involve a test drive from the end of the assembly line to the vehicle transporter that will take the car to a dealer.
In traditional print publishing, this final review occurred after layout of the document and was referred to as proofreading (i.e., reading the document as proof that the final results were correct). With modern all-electronic production processes, and particularly with online publishing (e.g., help systems and Web sites), there's a pernicious assumption that if you've carefully controlled the process up to this point, there will be no additional errors to find. I've found few editors who believe this to be the case. Indeed, when a former employer suggested doing away with this aspect of the publishing process, I spent a few months documenting (and fixing) the errors that resulted. Despite careful control of quality during the internal and external reviews, and an experienced and skilled desktop publisher, I routinely found dozens of small errors and often more than one significant error that would not have been caught without a final proofreading.
The final step in any publishing process is some form of management signoff. Depending on the nature of the organization and of the people involved, this step may be a simple "rubber stamp" exercise; in other cases, the manager who will be responsible for the consequences of publishing the document will rigorously examine the document down to the level of individual punctuation. In either case, all the quality control you've performed thus far in the process will be justified: if there are no errors for the manager to detect, they will gradually come to trust you, and will spend less time focusing on the minutiae. This makes better use of a manager's expensive time, avoids costly delays at the end of the publication process, and builds a sense of trust and cooperation.
The planning meeting I described earlier in this article is a key success factor. If the manager has carefully reviewed the documentation plan right at the start, this greatly decreases the risk of finding the results unacceptable at the end of the process. In particular, the manager has less incentive to justify their participation in the review process by insisting on major last-minute changes because they have already insisted on these changes at the start of the process. It's certainly true that sudden insights at the last possible moment can lead to extensive changes in the documentation, and this is one justification for a final management signoff. But more often, the approval task is simple and fast for a document that has survived all the previous steps in the review, and provides yet another example of how building quality and consensus into the initial stages of documentation development pays off with time savings and increased quality later in the process.
For products such as software that are evolving continuously, the writing and review process will be repeated (iteration) for the lifetime of the product—as often as twice per year for decades with modern software. For other situations, documents may be revised only occasionally, often after a long delay; this is the case with dictionaries and technical specifications, for example. Iterative review is often seen as a necessary evil, but in truth, it should be seen as a final reality check on your process. Reader reviews of your information between versions provide key insights into what you're doing right—and perhaps more importantly, into what you're doing wrong. Developing a mechanism for such reviews permits continuous improvement of the quality of the materials you produce, particularly if you can address those reviews in the down-time between releases of the document.
Reader feedback can be difficult to obtain, particularly where you don't have direct access to these readers. But there are always ways to find out whether the material you've produced is meeting expectations. Review articles in the popular press or in specialty magazines are a great source of information, whether you're producing software or best-selling popular science books. Best-selling third-party books such as the Pogue Press/O'Reilly "missing manuals" often provide keen insights into more effective ways to communicate or things that you've missed in your own documentation. If you lacked the time or resources to have your materials professionally edited, why not hire an editor during the down-time before the next writing and review cycle begins? Use the insights provided by that professional to do a better job in your next iteration of the writing. Look for online communities who discuss what you've written to see what they're saying, and ask leading questions to generate discussions about where you've succeeded and where you've failed.
Last but not least, talk to your colleagues in technical support and training. They deal directly with the people who use the information you create, and can provide important insights into where that information fails to meet reader needs.
This article neglects many important issues that arise from the details of this process. Among other things, each company and context will have certain unique characteristics that change how you look at the review process. For example, sometimes it's possible to conduct simultaneous reviews of a document to greatly reduce the time compared with sequential reviews. This raises the issues of how to reconcile conflicting review comments and how to obtain the synergies that are possible when reviewers can interact and conduct dialogues as they simultaneously review a document. Tools such as wikis (see, for example, http://wiki.org/) and blogs (e.g., http://www.livejournal.com/) allow readers and publishers to interactively create and update information, and exciting new tools such as XMetal Reviewer (http://www.xmetal.com/en_us/products/xmetal_reviewer/index.x) will increasingly solve this problem.
In addition, as I noted at the start of this article, review processes eventually fossilize with the passage of time, and what was once a powerful living organism transforms into a burdensome chunk of stone. Periodically, the stakeholders involved in any review process should take the time to confirm that it is still serving its intended purpose, and that people are still following the process. An organization's context may have changed, requiring the addition of new review steps (e.g., by the organization's lawyer, to confirm compliance with new regulations) or the removal of old steps (e.g., no more management approval if they trust you to do the job right). Technological change may also let you improve your process. Even if no changes are necessary, the dialogue between stakeholders that results from this review process is itself valuable because it keeps the lines of communication open and instills a sense of teamwork that is often lacking.
My goal in writing this article is not to suggest that reviews are easy, but rather to explain the goals and constraints of the key components of a review process. In so doing, I've illustrated how to take advantage of each phase of review, while minimizing its constraints. As my examples have shown, you can obtain remarkable payback from investing the time to understand the process and to look for ways to make it work better. I encourage you to explore some of the ideas I've presented, and to report your results to readers of this magazine. I look forward to your insights!
©2004–2024 Geoffrey Hart. All rights reserved.