Revolutionizing Technical Writing: Measuring Effectiveness Through GAI-Driven Support Outcomes

Whenever I ask the question, how do you measure the performance of your writers, to any technical writing leader, the answers I receive are never satisfactory. People measure the quantity of content put out by the writers, how grammatically correct the articles are, the number of hits a page gets on the website, or the number of feedback emails.  

If you ask the very same people to define the role of a technical writer, the answers will be along the lines of helping customers use the product effectively or informing users about features in the product.  

The answers are at complete odds to the incentives. 

The aim of a technical writer is to inform the user on how to complete a task using the product at hand. If we agree on this premise and use that at the starting point, then counting articles and grammar errors don’t get us to the destination.  

People behave in ways that optimize for the incentives at play.  

What should those incentives be? I posit that they should be along the lines of effectiveness of the content. But that has been difficult, not impossible, to measure until now. Generative ASI (GAI) changes all of that. Now all we need to do is get the bot to ingest all the content and then sit back and watch it work its magic. By placing the GAI bot as the first line of the support, we can finally show how the content from the Help site deflects support tickets and frees up bandwidth for support engineers to focus on complex queries. 

During my first implementation of a GAI bot, we saw the bot jump in to 80% of the conversations and resolve close to 40% of those at 70% CSAT with no human intervention needed.  

Now writing teams can quantify the benefit of their work in relation to the company bottom-line.  

This also changes how we measure the effectiveness of the writing team. By focusing on ticket resolutions and CSAT, we can put the right incentives in place. A writer is no longer looking to create content for the sake of creating content. Instead, they are focused on creating the right content to ensure more resolutions by the bot.