There are many projects, products, and services to choose from when making decisions about the tools you and your team use.
One of the aspects many of us look at to help assess if a tool is useful to us is the documentation, but how do you judge what good documentation is?
A tool might offer exciting features, yet, the developer experience of the documentation might be lacking. When evaluating a new tool, the developer experience matters as much as the offered features.
This post aims to help you identify key factors to look for, how that might reflect on the project, and how well it will meet your expectations.
Onboarding and Getting Started
If you are learning about a new tool, then the getting started guide for any project is typically the first document you read.
A bad experience with it is likely to put any casual reader off of investigating further. First impressions matter, so below are four factors to look for when evaluating the getting started experience for documentation.
Self-serve option available
Trying the time-honored “getting started guide” is typically the first point of call for developers assessing a new technical tool. There are some tools (especially in the SaaS world) that rely on onboarding you through a front end first and taking you through the first steps from there instead.
In either case, prioritize tools that allow you to create an account and test yourself without the need to make an appointment with a sales representative or fill in excessive forms.
There might be some valid reasons for doing this, such as a private beta or that the project is new. But often, it’s purely to force you through a sales process and bombard you with sales messages. This isn’t directly related to documentation, but often you also can’t view documentation until you are through this process, which makes it hard to evaluate at all.
Time to getting started
If you evaluate multiple options, then how long it takes you to follow a getting started guide from the beginning to a satisfactory conclusion is one measure to measure by.
While the real time can depend a lot on the tutorial’s complexity, for comparable tools or guides, a shorter time is a good sign that the team has spent time reducing barriers, testing steps, and checking that the language they used is clear.
Real-world examples
Getting started guides tend to focus on simplicity and speed. Often this is intentional to make a tool appear “easy” to use. However, if the examples and steps the guide follows are unrealistic for any “real world” use case for the tool, then is it a good getting started guide?
Dependencies and Prerequisites
Certain programming languages are notorious for having deep and complex dependency trees. This means that a project might leverage code from a 3rd party (which is common), but then that tool also has its dependencies.
Following a getting started guide should not have to take you down a complex path of conflicting dependency versions or surprising undocumented prerequisites you have to figure out. This can show that the developers or writers of the documentation have not tested examples from scratch on a “vanilla” new user set up.
Ideally, a project should work with the most popular or most supported (and LTS, for example) versions of languages and operating systems. If they don’t, it can show a lack of meaningful commitment to development or specific platforms.
At the least, the documentation should state which versions you need and a rough idea of when to expect support for newer or more popular versions.
Continuing the journey
A great getting started guide is a big plus for any project you are evaluating, but rarely does a getting started guide help you apply anything to a real-world production application.
One of the hardest parts of creating documentation is taking a reader on this crucial journey, and it’s generally where most people get frustrated with documentation. Some projects have a much clearer and defined use case than others (for example, a payment API versus a CMS), but there are some general things to look for.
A good getting started guide should suggest common pages or paths for you to follow next. These could be generic, such as key components of the tool, finding further information on a feature, or following typical use cases based on user research. It’s near impossible for documentation to suit every possible use you might have, but it can help you assemble the knowledge you need.
Code examples
Not wanting to disappoint anyone reading this who writes documentation, but it’s an adage that developers often don’t read the documentation thoroughly but rather scan for code examples. Good documentation is structured to ensure that important text surrounds code examples, so there is more chance you notice it.
Code examples are often a source of annoyance for anyone reading documentation, and I’m sure you have come across one that is incomplete or doesn’t work.
There are different types of code examples, and in each case, their purpose should be clear.
Some code examples are part of a tutorial or guide, and we expect the reader to follow them in order and build a full piece of code that works as expected. In these cases, the examples should tell you all you need to know and not leave anything to assumption to result in a working example.
Some code examples are illustrative to show a concept or implementation detail, which means they do not work without replacing placeholder values or extra code. Documentation should make these clear and suggest what you need to do to make them usable.
While not always necessary, documentation should explain or show you the expected output for code examples, so you know that you have implemented it successfully.
Finally, if a project supports multiple programming languages or implementation options, then the documentation should show code examples for all of these. However, this is a lot of work, but examples for the most widely used options are essential, and guidance on using the other supported options enough.
Reference - SDK and API documentation
Filling in the documentation gaps is generally the role of reference documentation, which includes API endpoint and function documentation, architecture explanations, and other aspects of the project readers might find useful to know if they need to.
Firstly, is reference documentation available, or do you need to dig through the codebase to find it? If it does exist, how complete or useful is it? Often teams autogenerate reference documentation, and that’s fine, but often descriptions come from code comments, and code comments are not always useful to a reader.
Reference documentation can be more abstract than other documentation as it describes individual pieces of a project instead of how to use them. However, good reference documentation still explains what that piece is and how it works. Below you find an excellent example.
The Stripe API documentation is a common example of useful and usable autogenerated (but using highly customized tooling) API documentation. Not wanting to publicly shame any project, this tweet from Ricardo Ferreira sums up the problem with documentation generated from comments perfectly.
Language
Good language is a big topic, and it doesn’t necessarily affect the quality of a project. It can show that the team behind it goes the extra mile to care, which is a good sign.
Acceptable documentation should be free of spelling mistakes and other typographical errors. Excellent documentation should use a consistent voice, terminology, and style. What constitutes “correct” in these cases is personal and opinionated, but consistency is key.
There are a handful of popular and commonly used style guides, such as the Google and Microsoft style guides. Whether that documentation follows one of these style guides over another, or its own custom style guide is not what’s important. However, if a company follows a style guide, it shows they have given that extra thought and care to their documentation.
Bonus points
These discussion points don’t quite fit into any of the sections above but show that a team has an extra eye for detail.
How does the documentation look? Something that looks fantastic and uses the latest tricks in web development may not equal useful and useable documentation, but if it’s both of those, then even better.
How often is the documentation updated? There are some good reasons for not updating documentation regularly, such as an older project or problem-free documentation, but it can also indicate a dormant project that’s best avoided.
Good documentation shows a team cares
We all know that documentation is hard, especially for solo developers or smaller teams who also need to focus on features and bug fixes. However, when there are many options to choose from when evaluating new tools, frameworks, and services, and sometimes weighing them up on features alone is inconclusive. Therefore, the documentation’s developer experience matters.
Good developer experience and documentation indicate how much a team and community involved with a project care about their users. More importantly, it indicates how much work could be involved in implementing projects you are evaluating.
If tutorials are poorly written, or don’t make sense. If code examples don’t work, or important details are lacking. Will you find similar problems and a lack of concern in a project after investing your time and business in it?
