View the original post
Knowing how to test and measure your copy is an essential skill.
After you finish writing the copy, how sure are you that your readers will understand it? Can they comprehend the message? What’s on their head when they read it?
No, your other fellow writers, product managers, product designers, and your VP don’t count. I mean your actual readers.
In this article, I want to share my team’s customized method in validating UX Copy.
Why do you need to validate your copy?
The short answer, to know how well it works.
The long answer, in a business environment, you need to convince other stakeholders why and how the copy you write will work, how our copy will perform in delivering results in interactions, responses, or whatever success metric you use. Unless you are a gifted writing warrior or don’t care about success metrics, you need to validate your copy.
What are the methods you can use?
You can utilize many methods to test or validate your UX Copy, such as A/B Testing, Usability Testing, Cloze, Direct Surveys, Think Aloud, etc.
Why do my team and I make our validation method?
We’ve been exploring the array of practices offered by other fellow writers on the www. But, we experienced inconveniences such as:
- At the moment, there are limitations from technical and bureaucratic limitations to do A/B testing.
- We feel that Usability Testing focuses more on the whole experience and flow.
- Cloze Tests and Direct Surveys don’t fit our needs.
- Think Aloud doesn’t work well when the respondent doesn’t speak much.
From those inconveniences, our so-called Copy Comprehension Test is born.
The Copy Comprehension Test Step by Step
The objective of this process is to get more understanding of how well your copy works. Along the way, you will found that this qualitative method gave you other exciting insights. Here’s the step-by-step to do the test.
Step 1. Plan The Test
Before you do the test, you need to create a research plan document that consists of the research background, objective, question list, participant criteria, and timeline.
💡 Tip: Don’t ask too many questions and only ask open-ended questions.
Step 2. Prepare The Test Materials
Two things need to be prepared, mock-up and scorecard.
You need mock-ups with the copy you want to test to give full context to the participants. You can also provide a mock-up using the Invision app or Maze when necessary.
Then provide a copy comprehension scorecard. The scorecard is where you put a score to the copy; whether it is easy to understand, makes the user confused, or prone to misinterpretation.
Step 3. Recruit Participants
To conduct the test you will need five participants from different backgrounds. Find people who fit the research criteria you’ve set before and make an appointment with them.
This part is tricky and time-consuming. First, you have to find the people who fit the research criteria; then, you have to contact them, invite them, and schedule a time. But there is no assurance that they will come to the appointment. Sometimes they will reschedule, opt-out, or even ghost.
💡 Tip: Have a respondent bank, so every time you do research or your current respondent bailed, you don’t need to recruit for another respondent again. You would already have people that fit your criteria at your disposal.
Step 4. Conduct Test
Today is the day! You have to stay neutral and make the participants feel at ease. You should not push, direct, or give any clues to make them understand our copy. Focus on listening to respondents’ answers rather than filling the scorecard. You can record the session and fill the scorecard later.
💡 Tip: Follow up with the respondent a day or an hour before the session. Make sure they are ready as in; have a proper internet connection, know how to use Google Meet/Zoom, are in a quiet environment, etc.
Step 5. Process and Analyze Data
After you’ve finished the test, it’s time to compile and tidy up the data into the scorecard of each participant. Then, compare the comprehension score of each section from every user using an affinity diagram.
At the end of the process, you will know which section or copy needs immediate action and which ones are good to go.
Step 6. Finalize
Take necessary action from the analysis result to accomplish your success matrix.
Pros, Cons, and Conclusions
This comprehension test costs a lot of resources, time, and money. The test itself takes approx. one to two weeks to perform. One interview session can take up to 90 minutes, and you need to do five sessions to make sure the result is valid. You also need to give compensation to the respondents for giving their time.
While the comprehension test costs a lot of time initially, it can save your time in the long run. You will have solid data when the stakeholders are questioning the copy validity. The copy will be more likely to deliver results in interactions, responses, or whatever success metric you may use. So, there will be no need for revisions in the future.
The process will also give us a lot of insights and inputs, not only for the copy but sometimes for the user flow, information architecture, and user interface design.
While the copy comprehension test costs a lot of effort, time, and money, it is necessary to be part of the UX writing process. The benefits of the process outrun the cost.
Do you need to do this on every project? Ideally, yes. The test can be exhausting and time-consuming; you need to assess the weight of the projects and the timeline given before deciding to do the comprehension test.
Please read my complete UX Writing Process here. I wrote this article before the Copy Comprehension Test was born; hence I still use Usability Testing.
Sincerely, a UX writer who need constant validation (LOL).
- My team, Edward Gilang, Senny Ciu, Zahra Amalia, and Ayu Lestari, helped build and run the test.
- Ernest Dimitria, the team’s editor who helped me to edit this article.