The Document is the Program
 by Hairy Larry, in Editorials - Sat, Jan 1st 2000 23:59 PDT

Over the years, desktop computer users have thrown up a number of straw men to explain why they can't use Free operating systems. The community has shot them down one by one, from "It doesn't support my hardware" to "There's no business software available", but there's still one complaint that too often goes unanswered: "I can't make heads or tails of this manual!" In today's editorial, Hairy Larry discusses the need for documentation and the three types of documents he thinks are essential to any program.

Just how important is documentation to the success of your program? Without a doubt, the documents are the most important pieces of your code.

Specifications Documents

Before you write a line of code, you should have at least a preliminary version of your specifications document. This does not mean that you shouldn't write toy programs as feasibility studies. (It's nice to know that something is possible before you commit to a lot of work.) It does mean that before you start on your .001 Alpha version, you should have a pretty good idea of where you are going.

And here's the reason why: Algorithms + Data Structures = Programs. Or, as I have heard it said, the database is the program. I really don't know of any substantial programming project that doesn't rely on data.

Getting your data structure right is hard. Mistakes in your data structure lead to massive rewrites. You have to put a lot of thought into how your database will work and how your code can access it. How can you even think about this until you have a pretty good idea about where you are going?

Of course your specifications document will change. It's a good idea to add version control to your specs from the outset. I'm sure you plan on enhancing your program by adding new features and bringing out new versions. These features will have to be specified, so track the changes in your specs just like you track the changes in your code.

Programmer Documents

I learned that even on projects on which I was the only programmer, I would often come back a month later (or the next day) and wonder "Just what was I doing there?" Now I'm assuming that if you are working on an open source project you are hoping that other programmers might get involved. How can you possibly expect them to figure out "What is he doing there?"

Each piece of source code should be documented at the top. Describe its overall function, what modules call it, and what modules it calls. I find it helpful to write this before I start coding to focus my attention on the problem at hand. In the source, each block of code should be documented with inline comments. What does this variable do? What algorithm am I using here? Why does this piece of code even exist? These questions and more should all be answered by inline comments.

You also need a single large document for the entire project. This document should describe the data structure and the overall structure of the program. Each module and its function should be included. You also need a list of all the documents for the project. Include a way to get in touch with you. Just because you understand what you are trying to say doesn't mean that the programmers reading the document will. Make it easy for them to ask questions and include your clarifications in the document.

By providing good programmer documents, you make it possible (not easy, just possible) for other programmers to come on board and help you with your project.

End User Document

The end user document (AKA the Manual) is the most important single piece of your entire project. It's more important than the data structure. It's more important than the code. It's more important than the documents described above. It is also, unfortunately, the most neglected.

Write your manual in plain, simple English. (French, German, Japanese, etc.) Use no jargon. Assume nothing. Describe the purpose of the program. List the major areas of functionality. Include step-by-step instructions on how to get from point A to point B.

How can you possibly expect the user without a manual to know that he has to click on the frammistran before going to the screen where he enters the stuff describing the fujitron? It may be obvious to you, but it's certainly not obvious to me.

I know most users won't read the manual. Instead of shouting "RTFM, RTFM", include context-sensitive help. Online help is the other side of the end user documents. Let your user hit the F1 key and view the portion of the manual appropriate to the task at hand. This is certainly easier than digging out a book or searching through a text file.

Quality end user documentation requires both a single large manual with all the documentation that a user can read or refer to and context-sensitive help at the touch of the finger when he needs it.

Closing

When a student in the Department of Math and Physics, I remember well the gearheads sneering at the English Majors who were having so much trouble with their required math courses. At the same time, there were no required writing courses for the Computer Science degree beyond Freshman English.

After I graduated, I realized that the most important skill leading to advancement was not the ability to do calculus. It was not being able to churn out tons of hot shot code. It was not the ability to design elegant data schema.

The most important skill is the ability to write well and communicate. It doesn't matter how good you are and how hard you work. If you can't communicate what you are doing and why you are doing it, you will be passed over for advancement. The reason for this is simple. The managers making the promotion decisions are not computer scientists; they are managers. They can't read code. They read English. If you can't write English, you just remain a cog in the machine.

Applying this to the world of open source software, it doesn't matter how good your code is. Most of your users will never read your code. They will read your documents. If your documents are well written, the users will find it easy to run your program. If not (or worse, if there's no documentation at all), potential users will hit a stumbling block and falter. They will then decide that the program is hard to use and look for another one. After all, they haven't made a major financial investment, so it's easy for them to walk away and try something else.

Hairy Larry (hairylarry@deltaboogie.com) was a student of music and mathematics at Arkansas State University where he finally graduated with a B.S. in computer science. He remains active in music through his band, Hairy Larry and The Flying Hingarians, and his radio show, "Something Blue". He is chief gearhead for the web sites TV Soup and Delta Boogie.
tvsoup.com
deltaboogie.com

T-Shirts and Fame!

We're eager to find people interested in writing editorials on software-related topics. We're flexible on length, style, and topic, so long as you know what you're talking about and back up your opinions with facts. Anyone who writes an editorial gets a freshmeat t-shirt from ThinkGeek in addition to 15 minutes of fame. If you think you'd like to try your hand at it, let jeff.covey@freshmeat.net know what you'd like to write about.

[Comments are disabled]

Referenced categories Topic :: Documentation Topic :: Software Development :: Documentation

 Comments

[»] Documentation
by Don Hansford - Jan 22nd 2000 06:33:16

OK, here we go with a fairly typical Newbies point of view. I am not a CS major, but I am a PC technician. I have been using DOS/Win for more years than I care to remember. About 12 months ago, I started hearing more & more about Open Source software. To cut a long story short, I am now running exclusively on Open Source (FreeBSD) and having a pretty good time of it all up. But the biggest let-down is with the lack of clear documentation with applications! As has been said before, man pages have their place, and I browse them assiduously, but they are not the best environment for Mr Average User. Installations aren't that big a problem (now!), though they were at the beginning. Setting up the programme to actually do what you installed it for seems to be the most neglected area, at least for me. Some of the options available are either just plain cryptic or downright geeky in their approach, "Enable Alpha bump-mapping? Y/N" I have no idea what Alpha bump-mapping is, though it's probably something I wouldn't want my daughter to consider doing! Some (relatively) brief explanations of what each option allows/disallows would be nice. You don't need to go to the lengths that, say, The GIMP went to (a truly excellent manual), but at least something would be nice. It is my aim to have some of the professional offices (Medical, Veterinary, Dental) in my area, start using Open Source OS's and software in their daily operations. Unfortunately, the lack of easy to access documentation makes it very hard for us to implement many of the appropriate applications, so Open Source misses out again, in favour of Redmond, who, if nothing else, has plenty of documented ways around the shortcomings of their offerings! End Rant.

[reply] [top]

[»] Re: I Utterly Disagree
by gregab - Jan 15th 2000 02:18:58

One simple example of a "human software" error report, followed by a serious question:

This program can't do <whatever>. Contact your system administrator.

What if I AM the system administrator? Don't you human software me, OK?

[reply] [top]

[»] Re: I Utterly Disagree
by gregab - Jan 15th 2000 02:17:46

One simple example of a "human software" error report, followed by a serious question:

This program can't do . Contact your system administrator.

What if I AM the system administrator? Don't you human software me, OK?

[reply] [top]

[»] Re: I utterly disagree
by naChoZ - Jan 12th 2000 09:04:25

Big, huge, gaping holes in your logic there. First, you're forgetting a big one, end users are basically stupid. Anyone who has been in the support side of things knows this. While even in the most poorly written of software, a user should at least be able to figure out how to, say, save a document in a word processor, but what about advanced features? Using the same word processor example, most users will not even know what a mail merge *is*, let alone be able to find it intuitively. Most end users are people who cannot operate computers without constant hand-holding and, for some of them, the only hand-holding available to them is the manual.

As far as comments in code go, you're basically saying that it's poor form to use them in your code. Comments aren't poor form used as a crutch to hold up your code, they're also for people who either don't code or are trying to learn to code who will have trouble following the flow of it without help. One part of the point of open source software is so that we can learn from each other, not prevent people from being able to learn and then indignantly scoff at those less experienced.

--
Andy Harrison ICQ: 123472 AIM/Y!: AHinMaine

[reply] [top]

[»] I utterly disagree
by Richard Clark - Jan 5th 2000 01:40:16

Totally. Utterly. If the code needs comments to be readable, it has been badly written. If the program needs a manual to be usable, it has been badly implemented.

Software is not like a car, it is not like a building, the very design of the program, and its implementation, should shout out how to use it louder than any manual or specification document.

Do you need a manual to work out how to instruct a human to write something? no, don't be stupid, you just tell them what you want. no matter how complex the problem, you don't need a manual, you start by trying with the biggest words, and drop down to shorter words for those concepts that are not immediately understood. Why everyone continues with this manual bullshit when even Microsoft has cottoned on to human software is beyond me.

To reiterate:
If a user can't use your software effectively without resorting to a manual, your software has a bug.
If a programmer can't read your code and understand what it does without screeds of comments (Some comments, obviously, are appropriate), then your code needs rewriting.

Stop writing dumb software.

[reply] [top]

[»] Language and Communication
by PotatoMan - Jan 4th 2000 19:56:03

The comments about who wrote this editorial are interesting in light of the subject matter. It seems clear that the real reason the authorship of this editorial is vague is that the poster's name has been placed where the byline should go.

Native speakers (and readers) of a language have certain expectations built into their wetware. These are the parsing rules for the grammar. Miscommunication results when these rules are violated. Hairy Larry is correct when he says you must learn to communicate in order to succeed. If you misuse language, others will be confused about what you mean, and you miss an opportunity to persuade or inform them.

It has been a source of puzzlement to me that no programmer would expect a compiler to 'understand' his meaning when he writes 'foreash' rather than 'foreach', yet if he writes documentation that uses 'enstans' in place of 'instance' he expects this to be understandable. Or, more pointedly, if the byline reflects the poster rather than the author, misunderstanding is bound to occur.

I believe the first step in better documentation is to learn how to write prose correctly (is it written 'function()' or '(function)'?). Proper spelling would also be a good thing (is it 'open FILE, filename' or 'open FILE filename'?). You learned C and Perl and Lisp; surely you can learn your own language as well. It's just one more hack.

[reply] [top]

[»] Documentation: Windows to Linux for the novice
by kaysi - Jan 2nd 2000 19:18:39

I sure know, as being the end user, that documentation sure does make a big difference - as I comb through the web and numerous books, I find myself somewhat overwhelmed. However; one way or another I am going to make the transition. In my office we have been testing a couple different versions of Linux on some of our extra PCs. And, slowly but surely I *think* I am starting to get some of this! However; not being a programmer much of this is over my head. So I continue to read and read and read. Perhaps one of these days it will finally click for me!

[reply] [top]

[»] Tutorial on Literate Programming
by jeff covey - Jan 2nd 2000 14:27:27

Ok, so who knows about Literate Programming and is willing to write a freshmeat tutorial on it? The usual incentives apply: fame, t-shirt, two-week vacation in Aruba (pick two of three).

--
vs lbh pna ernq guvf, lbh'er n trrx.

[reply] [top]

[»] The document drives the OS
by Hairy Larry - Jan 2nd 2000 12:35:17

hmm...

Writing documents to make it easier for Windows users to migrate to Linux.

What an interesting idea.

[reply] [top]

[»] Documentation
by Andrew Armstrong - Jan 2nd 2000 10:20:05

Some software manuals simply tell you what each function does, some also include what the function is for, and if you have nothing to do for several weeks you can learn all you need from the latter. Better manuals tell you how to do many common tasks, but it is difficult to anticipate every task that may be required. My background is technical but not programming : I design electronic hardware, and write html and javascript. (Also write magazine articles on electronics) My experience so far has been that I can make some progress trying to make Linux do the tasks I want done, then I hit a brick wall. After much searching I make a little progress to the next brick wall. A writer of operating systems or device drivers would probably find it easy, but most _users_ start from a less technical background than myself, and would probably fail to make a Linux system work. (I haven't completely succeded yet). The other half of what is going on is that when you install new software or hardware using Windows, it works more often than not, even if you are not an expert. Generally the program automatically recognises the hardware, and you don't need to know very much at all. Equally, the file system is superficially more comprehensible. Directories called "My Documents" while a little twee, do convey their intended function. Not so with Linux. I had to search for some time to find an exposition of the various directory structures, and I still don't have it completely clear in my mind. To work with something beyond following a few instructions by rote, it is useful to have a mental picture or story that describes the machinery in useful terms. For Windows, users don't care abvout binary or magnetisation of bits of hard disc. The rarely know about the FAT. But a picture of files and directories, almost like a filing cabinet, maps well enough to what the system does for it to be useful. I could give more examples, but I hope that this illustrates the point I am trying to make: that a description from which the user could build a mental model would be helpful for people to use Linux as a tool rather than as an end in itself. To attract Windows users, it should probably start from the Windows mindset, for example: In Windows you do this by the following method . .because . . .but in Linux . . .

[reply] [top]

[»] literate programming builds cathedrals
by Liam Quin - Jan 2nd 2000 04:04:08

I rather liked the CACM article in which Jon Bentley invited Don Knuth to solve a problem using Web... the result is 16 columns long and uses a previously unpublished data structure.

Jon then invited Doug McIlroy to review the result. Doug gave a 6-line shell script that actually came somewhat closer to solving the original problem.

Literate Programming encourages programming, i think, that's like a mathematical proof: almost completely self-contained.

It's better to redefine a problem so it's easier to solve than to write a complex program.

It's better to reuse than to invent.

Where you need to write a complex self-cpntaind program, literaet programming is surely an excellent approach. Where you need to write a short program that reuses other components and that can be understood and repaired in seconds or minutes, something simpler may be better.

Often, well-chosen variable names are better than pages of (usually out-of-date) comments.
AddToTableOfContents(Name, Page) will almost always win out over attc(buffer, i). A program is called robust if it survives being hacked by people who don't have time or inclination to read much of it.

Requirments documents are excellent, but they should be short, or people won't read them.

The programs easiest to modify are the most honest -- they are written so that you can understand a few lines of code, and then change them, without having to read every function that's called, every method, look at every class.

The needs of small projects are different from the needs of large projects. Most projects are small, and most large projects ought to have been small too. And then you don't need so much documentation.

The manual is the most important. If the manual and the cod disagree, if it's hard to document what the program does, it's almost always best to fix the program.

[reply] [top]

[»] Appreciate the comments
by Hairy Larry - Jan 2nd 2000 00:06:54

I appreciate the comments. I am not familiar with Literate Programming although I am familiar with Donald Knuth. More info on this would be good. Self documenting systems have always been a kind of dream of mine. Breaking out the Requirements document before program specification is such a good idea. And I thought the comments on the different kind of manuals and the importance of documenting work flow were very important.

In fact work flow is so important that it should be a design consideration for interactive programs. Important jobs should be specified screen by screen and workstation to workstation in very preliminary planning. Of course the work flow will be streamlined and the specs will be improved upon. Still, if you are not previsualizing how your program will be used and making the end user experience integral to your design you will almost certainly have trouble adding desirable end user features and implementing efficient work flows.

[reply] [top]

[»] Required Documents...
by Don Johnson - Jan 1st 2000 17:21:33

My $0.02:

There are four types of documents I find necessary for any project: (1) Requirements Documentation, (2) Design Documentation, (3) User Guide, (4) Program Documentation. I think UML is a great way to organize the first two, the third needs lots of pictures, and (since I mainly use Perl) a combination of comments and POD (plain old documentation) is best for the fourth.

REQUIREMENTS DOCUMENTATION

You should "design" the problem domain. This means carefully documenting the problem you intend to solve in a requirements document. Even for (especially for?) open source projects it's critical to communicate the purpose of the system. The audience for the requirements documentation includes other programmers and problem solvers.

DESIGN DOCUMENTATION

You should "design" the solution domain. This means carefully documenting your solution to the requirements. Even for (especially for?) open source projects it is critical to communicate the architecture and design solutions of the system. The audience for the design documentation includes other programmers and problem solvers.

USER GUIDE

I completely agree that the user manual is critical. But, how many of you have bought a "Netscape for Dummies" book? (Not most, I imagine). How many have spent much time looking at a user guide for any web browser? Sometimes complex functions can be so well hidden behind intuitive interfaces that the need for a user guide is minimized. I believe the importance of the user guide depends on the problem and the system implementation. Sometimes it's more important than others.

PROGRAM DOCUMENTATION

I love literate programming. Using the plain old documentation (POD) features of Perl, I can put the code and program documentation in the same place: run the code, publish the documentation. I believe program documentation is especially important to the open source community. Our goal should be to share our knowledge before we prove our cleverness. Just like any language, learning to program involves reading and writing. I consider learning from other people's code more important than writing my own. I enjoy reading well documented code. Sometimes I avoid reading clever, puzzling, undocumented code.

[reply] [top]

[»] Read first, no gripe necessary!
by John LeMay - Jan 1st 2000 12:39:52

If people would take the time to read the entire post, instead of reading only what looks interesting, they would note Jeff identifies the writer at the beginning of the article and a bio follows at the end. Perhaps a course in reading comprehension should be offered to CS folks as well!

[reply] [top]

[»] credits for articles
by robr - Jan 1st 2000 11:59:44

Perhaps that's a sign that there needs to be a much more visible sign of who wrote what. It's kind of confusing to see it on the main page and it looks like it's written by you, with only a very small bio down at the end of it.

[reply] [top]

[»] It's not my editorial. :)
by jeff covey - Jan 1st 2000 11:27:07

Just a reminder since I've received confused emails and now a post on this editorial... Just because I post the editorials doesn't mean that I write all of them. The author's name is given at the top of each one, and a brief bio is given at the bottom. Don't put other people's opinions in my mouth. :)

--
vs lbh pna ernq guvf, lbh'er n trrx.

[reply] [top]

[»] The Document is the Program
by Steve McAllister - Jan 1st 2000 11:26:24

How true this is and not just in the source code. Think about how many cases of a good program/concept being poo-pooed because of documentation. I have refused to use a program because of the docs, only to find someone using the program and telling me how great it is. And what about the many cases of developers forcing users to jump through hoops to get the programs running. Often times it would just take a small shell script, RPM etc... to simplify this. An hour or two of putting together an RPM or other installer, would save hundreds of hours of user time. This is especially important with all the new Linux users. New users hate and don't understand how to compile a program. There is an elitist attitude that only computer science majors should install a program. This is the major thing scarring new users away from Open Source. As a non programmer I get frustrated downloading a tarball only to find that the install instructions are written with the assumption that the user has programming knowledge. COMPUTER USERS AREN'T ALWAYS PROGRAMMERS. Developers need to be aware of this and documents should be written for the lowest common denominator.

[reply] [top]

[»] Literate Programming
by willadams - Jan 1st 2000 10:57:38

Is there no CS curriculum in the US which touches upon this?

Why is it that there are so few programming environments support it?

For those who've never heard of it, Literate Programming is Dr. Donald Knuth's (author of _The Art of Computer Programming_ series) concept of a program as formatted text with documentation integral to its structure.

TeX and METAFONT were written in said system and two of the books in his series _Computers & Typesetting_ are merely the typeset source for the programs TeX and METAFONT respectively.

William

[reply] [top]

[»] Overview docs sorely needed
by Dan Maas - Jan 1st 2000 10:53:04

I agree with Jeff here, especially regarding the second type of documentation. One of the biggest obstacles I encounter when working with open source software is the lack of a brief overview covering the basic data structures, control flow, etc. Many times I've started to code an enhancement to one of my favorite OSS projects, only to be stopped by an utter lack of understanding of how the program is put together. Detailed, frequent comments in the code are helpful, but they only give you the micro picture - understanding how a single function works isn't too hard, but grasping the overall structure of the entire program is nearly an impossible task without some sort of "introductory developer documentation."

Luckily the solution is simple: all I really want is a page or two of HTML describing the basic modules, classes, structures, etc. and how they fit together; that way I'll at least know where to start coding, and what not to touch.

Let's say I'm developing a simple GUI paint program. This is the sort of doc I want to see:

"FooBarPaint consists of two primary modules:
1) GUI code - this is written to Gtk+. The GUI code is in foobar_gui*.c. We're using a separate thread for the interface so don't forget mutexes when accessing GUI objects. Remember that we decided on 32 bits per pixel as the internal representation of images, regardless of the display depth.

2) Paint operations - this code carries out the paint functions like airbrush, pencil, lines, etc. Each tool resides in foobar_(toolname).c. If you're going to write a new tool, remember it needs to be registered with the GUI (check foobar_airbrush.c for an example). When a tool is selected and the user clicks on an image, the tool's activate() callback is called. Do your work in activate(); remember to save the data you modify and pass it to undo() when finished."

etc, etc...

[reply] [top]

[»] I AGREE!
by synap - Jan 1st 2000 08:21:36

I agree with this !! i fully agree.

[reply] [top]