Documentation Generator
The D programming language enables embedding both contracts and test code along side the actual code, which helps to keep them all consistent with each other. One thing lacking is the documentation, as ordinary comments are usually unsuitable for automated extraction and formatting into manual pages. Embedding the user documentation into the source code has important advantages, such as not having to write the documentation twice, and the likelihood of the documentation staying consistent with the code.
Some existing approaches to this are:
- Doxygen which already has some support for D
- Java's Javadoc, probably the most well-known
- C#'s embedded XML
- Other documentation tools
D's goals for embedded documentation are:
- It looks good as embedded documentation, not just after it is extracted and processed.
- It's easy and natural to write, i.e. minimal reliance on <tags> and other clumsy forms one would never see in a finished document.
- It does not repeat information that the compiler already knows from parsing the code.
- It doesn't rely on embedded HTML, as such will impede extraction and formatting for other purposes.
- It's based on existing D comment forms, so it is completely independent of parsers only interested in D code.
- It should look and feel different from code, so it won't be visually confused with code.
- It should be possible for the user to use Doxygen or other documentation extractor if desired.
Specification
The specification for the form of embedded documentation comments only specifies how information is to be presented to the compiler. It is implementation-defined how that information is used and the form of the final presentation. Whether the final presentation form is an HTML web page, a man page, a PDF file, etc. is not specified as part of the D Programming Language.
Phases of Processing
Embedded documentation comments are processed in a series of phases:
- Lexical - documentation comments are identified and attached to tokens.
- Parsing - documentation comments are associated with specific declarations and combined.
- Sections - each documentation comment is divided up into a sequence of sections.
- Special sections are processed.
- Highlighting of non-special sections is done.
- All sections for the module are combined.
- Macro and Escape text substitution is performed to produce the final result.
Lexical
Embedded documentation comments are one of the following forms:
- /** ... */ The two *'s after the opening /
- /++ ... +/ The two +'s after the opening /
- /// The three slashes
The following are all embedded documentation comments:
/// This is a one line documentation comment. /** So is this. */ /++ And this. +/ /** This is a brief documentation comment. */ /** * The leading * on this line is not part of the documentation comment. */ /********************************* The extra *'s immediately following the /** are not part of the documentation comment. */ /++ This is a brief documentation comment. +/ /++ + The leading + on this line is not part of the documentation comment. +/ /+++++++++++++++++++++++++++++++++ The extra +'s immediately following the / ++ are not part of the documentation comment. +/ /**************** Closing *'s are not part *****************/
The extra *'s and +'s on the comment opening, closing and left margin are ignored and are not part of the embedded documentation. Comments not following one of those forms are not documentation comments.
Parsing
Each documentation comment is associated with a declaration. If the documentation comment is on a line by itself or with only whitespace to the left, it refers to the next declaration. Multiple documentation comments applying to the same declaration are concatenated. Documentation comments not associated with a declaration are ignored. Documentation comments preceding the ModuleDeclaration apply to the entire module. If the documentation comment appears on the same line to the right of a declaration, it applies to that.
If a documentation comment for a declaration consists only of the identifier ditto then the documentation comment for the previous declaration at the same declaration scope is applied to this declaration as well.
If there is no documentation comment for a declaration, that declaration may not appear in the output. To ensure it does appear in the output, put an empty declaration comment for it.
int a; /// documentation for a; b has no documentation int b; /** documentation for c and d */ /** more documentation for c and d */ int c; /** ditto */ int d; /** documentation for e and f */ int e; int f; /// ditto /** documentation for g */ int g; /// more documentation for g /// documentation for C and D class C { int x; /// documentation for C.x /** documentation for C.y and C.z */ int y; int z; /// ditto } /// ditto class D { }
Sections
The document comment is a series of Sections. A Section is a name that is the first non-blank character on a line immediately followed by a ':'. This name forms the section name. The section name is not case sensitive.
Summary
The first section is the Summary, and does not have a section name. It is first paragraph, up to a blank line or a section name. While the summary can be any length, try to keep it to one line. The Summary section is optional.
Description
The next unnamed section is the Description. It consists of all the paragraphs following the Summary until a section name is encountered or the end of the comment.
While the Description section is optional, there cannot be a Description without a Summary section.
/*********************************** * Brief summary of what * myfunc does, forming the summary section. * * First paragraph of synopsis description. * * Second paragraph of * synopsis description. */ void myfunc() { }
Named sections follow the Summary and Description unnamed sections.
Standard Sections
For consistency and predictability, there are several standard sections. None of these are required to be present.
- Authors:
- Lists the author(s) of the declaration.
- Bugs:
- Lists any known bugs.
- Date:
- Specifies the date of the current revision. The date should be in a form parseable by std.date.
- Deprecated:
- Provides an explanation for and corrective action to take if the associated declaration is marked as deprecated.
- Examples:
- Any usage examples
- History:
- Revision history.
- License:
- Any license information for copyrighted code.
- Returns:
- Explains the return value of the function. If the function returns void, don't redundantly document it.
- See_Also:
- List of other symbols and URL's to related items.
- Standards:
- If this declaration is compliant with any particular standard, the description of it goes here.
- Throws:
- Lists exceptions thrown and under what circumstances they are thrown.
- Version:
- Specifies the current version of the declaration.
/**
* Authors: Melvin D. Nerd, melvin@mailinator.com
*/
/**
* Bugs: Doesn't work for negative values.
*/
/**
* Date: March 14, 2003
*/
/** * Deprecated: superseded by function bar(). */ deprecated void foo() { ... }
/**
* Examples:
* --------------------
* writeln("3"); // writes '3' to stdout
* --------------------
*/
/**
* History:
* V1 is initial version
*
* V2 added feature X
*/
/** * License: use freely for any purpose */ void bar() { ... }
/** * Read the file. * Returns: The contents of the file. */ void[] readFile(char[] filename) { ... }
/**
* See_Also:
* foo, bar, http://www.digitalmars.com/d/phobos/index.html
*/
/**
* Standards: Conforms to DSPEC-1234
*/
/** * Write the file. * Throws: WriteException on failure. */ void writeFile(char[] filename) { ... }
/**
* Version: 1.6a
*/
Special Sections
Some sections have specialized meanings and syntax.
- Copyright:
- This contains the copyright notice. The macro COPYRIGHT is set to the contents of the section when it documents the module declaration. The copyright section only gets this special treatment when it is for the module declaration.
- Params:
- Function parameters can be documented by listing them in a params section. Each line that starts with an identifier followed by an '=' starts a new parameter description. A description can span multiple lines.
- Macros:
- The macros section follows the same syntax as the Params: section. It's a series of NAME=value pairs. The NAME is the macro name, and value is the replacement text.
- Escapes=
- The escapes section is a series of substitutions which replace special characters with a string. It's useful when the output format requires escaping of certain characters, for example in HTML & should be escaped with &.
- The syntax is /c/string/, where c is either a single character, or multiple characters separated by whitespace or commas, and string is the replacement text.
/** Copyright: Public Domain */ module foo;
/*********************************** * foo does this. * Params: * x = is for this * and not for that * y = is for that */ void foo(int x, int y) { }
/**
* Macros:
* FOO = now is the time for
* all good men
* BAR = bar
* MAGENTA = <font color=magenta>$(DOLLAR)0</font>
*/
/**
* ESCAPES = /&/AddressOf!/
* /!/Exclamation/
* /?/QuestionMark/
* /,/Comma/
* /{ }/Parens/
* /<,>/Arrows/
*/
Highlighting
Embedded Comments
The documentation comments can themselves be commented using the $(DDOC_COMMENT comment text) syntax. These comments do not nest.
Embedded Code
D code can be embedded using lines beginning with at least three hyphens (ignoring whitespace) to delineate the code section:
/++
+ Our function.
+
+ Example:
+ ---
+ import std.stdio;
+
+ void foo()
+ {
+ writeln("foo!"); /* print the string */
+ }
+ ---
+/
Note that the documentation comment uses the /++ ... +/ form so that /* ... */ can be used inside the code section.
Inline Code
Inline code can be written between backtick characters (`), similarly to the syntax used on GitHub, Reddit, Stack Overflow, and other websites. Both the opening and closing ` character must appear on the same line to trigger this behavior.
Text inside these sections will be escaped according to the rules described above, then wrapped in a $(DDOC_BACKQUOTED) macro. By default, this macro expands to be displayed as an inline text span, formatted as code.
A literal backtick character can be output either as a non-paired ` on a single line or by using the ` macro.
/// Returns `true` if `a == b`. void foo() {} /// Backquoted `<html>` will be displayed to the user instead /// of passed through as embedded HTML (see below). void bar() {}
Embedded HTML
HTML can be embedded into the documentation comments, and it will be passed through to the HTML output unchanged. However, since it is not necessarily true that HTML will be the desired output format of the embedded documentation comment extractor, it is best to avoid using it where practical.
/**
* Example of embedded HTML:
*
* <ol>
* <li><a href="http://www.digitalmars.com">Digital Mars</a></li>
* <li><a href="http://www.classicempire.com">Empire</a></li>
* </ol>
*/
Emphasis
Identifiers in documentation comments that are function parameters or are names that are in scope at the associated declaration are emphasized in the output. This emphasis can take the form of italics, boldface, a hyperlink, etc. How it is emphasized depends on what it is - a function parameter, type, D keyword, etc. To prevent unintended emphasis of an identifier, it can be preceded by an underscore (_). The underscore will be stripped from the output.
Character Entities
Some characters have special meaning to the documentation processor, to avoid confusion it can be best to replace them with their corresponding character entities:
| Character | Entity |
|---|---|
| < | < |
| > | > |
| & | & |
It is not necessary to do this inside a code section, or if the special character is not immediately followed by a # or a letter.
No Documentation
No documentation is generated for the following constructs, even if they have a documentation comment:
- Invariants
- Postblits
- Destructors
- Static constructors and static destructors
- Class info, type info, and module info
Macros
The documentation comment processor includes a simple macro text preprocessor. When a $(NAME) appears in section text it is replaced with NAME's corresponding replacement text.
For example:/** Macros: PARAM = <u>$1</u> MATH_DOCS = <a href="http://dlang.org/phobos/std_math.html">Math Docs</a> */ module math; /** * This function returns the sum of $(PARAM a) and $(PARAM b). * See also the $(MATH_DOCS). */ int sum(int a, int b) { return a + b; }
The above would generate the following output:
<h1>test</h1> <dl><dt><big><a name="sum"></a>int <u>sum</u>(int <i>a</i>, int <i>b</i>); </big></dt> <dd>This function returns the <u>sum</u> of <u><i>a</i></u> and <u><i>b</i></u>. See also the <a href="http://dlang.org/phobos/std_math.html">Math Docs</a>. </dd> </dl>
The replacement text is recursively scanned for more macros. If a macro is recursively encountered, with no argument or with the same argument text as the enclosing macro, it is replaced with no text. Macro invocations that cut across replacement text boundaries are not expanded. If the macro name is undefined, the replacement text has no characters in it. If a $(NAME) is desired to exist in the output without being macro expanded, the $ should be replaced with $.
Macros can have arguments. Any text from the end of the identifier to the closing ‘)’ is the $0 argument. A $0 in the replacement text is replaced with the argument text. If there are commas in the argument text, $1 will represent the argument text up to the first comma, $2 from the first comma to the second comma, etc., up to $9. $+ represents the text from the first comma to the closing ‘)’. The argument text can contain nested parentheses, "" or '' strings, <!-- ... --> comments, or tags. If stray, unnested parentheses are used, they can be replaced with the entity ( for ( and ) for ).
Macro definitions come from the following sources, in the specified order:
- Predefined macros.
- Definitions from file specified by sc.ini's or dmd.conf DDOCFILE setting.
- Definitions from *.ddoc files specified on the command line.
- Runtime definitions generated by Ddoc.
- Definitions from any Macros: sections.
Macro redefinitions replace previous definitions of the same name. This means that the sequence of macro definitions from the various sources forms a hierarchy.
Macro names beginning with "D_" and "DDOC_" are reserved.
Predefined Macros
These are hardwired into Ddoc, and represent the minimal definitions needed by Ddoc to format and highlight the presentation. The definitions are for simple HTML.
B = <b>$0</b>
I = <i>$0</i>
U = <u>$0</u>
P = <p>$0</p>
DL = <dl>$0</dl>
DT = <dt>$0</dt>
DD = <dd>$0</dd>
TABLE = <table>$0</table>
TR = <tr>$0</tr>
TH = <th>$0</th>
TD = <td>$0</td>
OL = <ol>$0</ol>
UL = <ul>$0</ul>
LI = <li>$0</li>
BIG = <big>$0</big>
SMALL = <small>$0</small>
BR = <br>
LINK = <a href="$0">$0</a>
LINK2 = <a href="$1">$+</a>
LPAREN = (
RPAREN = )
BACKTICK = `
DOLLAR = $
DEPRECATED = $0
RED = <font color=red>$0</font>
BLUE = <font color=blue>$0</font>
GREEN = <font color=green>$0</font>
YELLOW = <font color=yellow>$0</font>
BLACK = <font color=black>$0</font>
WHITE = <font color=white>$0</font>
D_CODE = <pre class="d_code">$0</pre>
D_INLINECODE = <pre style="display:inline;" class="d_inline_code">$0</pre>
D_COMMENT = $(GREEN $0)
D_STRING = $(RED $0)
D_KEYWORD = $(BLUE $0)
D_PSYMBOL = $(U $0)
D_PARAM = $(I $0)
DDOC = <html><head>
<META http-equiv="content-type" content="text/html; charset=utf-8">
<title>$(TITLE)</title>
</head><body>
<h1>$(TITLE)</h1>
$(BODY)
<hr>$(SMALL Page generated by $(LINK2 http://dlang.org/ddoc.html, Ddoc). $(COPYRIGHT))
</body></html>
DDOC_COMMENT = <!-- $0 -->
DDOC_DECL = $(DT $(BIG $0))
DDOC_DECL_DD = $(DD $0)
DDOC_DITTO = $(BR)$0
DDOC_SECTIONS = $0
DDOC_SUMMARY = $0$(BR)$(BR)
DDOC_DESCRIPTION = $0$(BR)$(BR)
DDOC_AUTHORS = $(B Authors:)$(BR)$0$(BR)$(BR)
DDOC_BUGS = $(RED BUGS:)$(BR)$0$(BR)$(BR)
DDOC_COPYRIGHT = $(B Copyright:)$(BR)$0$(BR)$(BR)
DDOC_DATE = $(B Date:)$(BR)$0$(BR)$(BR)
DDOC_DEPRECATED = $(RED Deprecated:)$(BR)$0$(BR)$(BR)
DDOC_EXAMPLES = $(B Examples:)$(BR)$0$(BR)$(BR)
DDOC_HISTORY = $(B History:)$(BR)$0$(BR)$(BR)
DDOC_LICENSE = $(B License:)$(BR)$0$(BR)$(BR)
DDOC_RETURNS = $(B Returns:)$(BR)$0$(BR)$(BR)
DDOC_SEE_ALSO = $(B See Also:)$(BR)$0$(BR)$(BR)
DDOC_STANDARDS = $(B Standards:)$(BR)$0$(BR)$(BR)
DDOC_THROWS = $(B Throws:)$(BR)$0$(BR)$(BR)
DDOC_VERSION = $(B Version:)$(BR)$0$(BR)$(BR)
DDOC_SECTION_H = $(B $0)$(BR)$(BR)
DDOC_SECTION = $0$(BR)$(BR)
DDOC_MEMBERS = $(DL $0)
DDOC_MODULE_MEMBERS = $(DDOC_MEMBERS $0)
DDOC_CLASS_MEMBERS = $(DDOC_MEMBERS $0)
DDOC_STRUCT_MEMBERS = $(DDOC_MEMBERS $0)
DDOC_ENUM_MEMBERS = $(DDOC_MEMBERS $0)
DDOC_TEMPLATE_MEMBERS = $(DDOC_MEMBERS $0)
DDOC_ENUM_BASETYPE = $0
DDOC_PARAMS = $(B Params:)$(BR)\n$(TABLE $0)$(BR)
DDOC_PARAM_ROW = $(TR $0)
DDOC_PARAM_ID = $(TD $0)
DDOC_PARAM_DESC = $(TD $0)
DDOC_BLANKLINE = $(BR)$(BR)
DDOC_ANCHOR = <a name="$1"></a>
DDOC_PSYMBOL = $(U $0)
DDOC_PSUPER_SYMBOL = $(U $0)
DDOC_KEYWORD = $(B $0)
DDOC_PARAM = $(I $0)
DDOC_BACKQUOTED = $(D_INLINECODE $0)
ESCAPES = /</</
/>/>/
/&/&/
Ddoc does not generate HTML code. It formats into the basic formatting macros, which (in their predefined form) are then expanded into HTML. If output other than HTML is desired, then these macros need to be redefined.
| Name | Description |
|---|---|
| B | boldface the argument |
| I | italicize the argument |
| U | underline the argument |
| P | argument is a paragraph |
| DL | argument is a definition list |
| DT | argument is a definition in a definition list |
| DD | argument is a description of a definition |
| TABLE | argument is a table |
| TR | argument is a row in a table |
| TH | argument is a header entry in a row |
| TD | argument is a data entry in a row |
| OL | argument is an ordered list |
| UL | argument is an unordered list |
| LI | argument is an item in a list |
| BIG | argument is one font size bigger |
| SMALL | argument is one font size smaller |
| BR | start new line |
| LINK | generate clickable link on argument |
| LINK2 | generate clickable link, first arg is address |
| RED | argument is set to be red |
| BLUE | argument is set to be blue |
| GREEN | argument is set to be green |
| YELLOW | argument is set to be yellow |
| BLACK | argument is set to be black |
| WHITE | argument is set to be white |
| D_CODE | argument is D code |
| D_INLINECODE | argument is inline D code |
| DDOC | overall template for output |
DDOC is special in that it specifies the boilerplate into which the entire generated text is inserted (represented by the Ddoc generated macro BODY). For example, in order to use a style sheet, DDOC would be redefined as:
DDOC = <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
<html><head>
<META http-equiv="content-type" content="text/html; charset=utf-8">
<title>$(TITLE)</title>
<link rel="stylesheet" type="text/css" href="style.css">
</head><body>
<h1>$(TITLE)</h1>
$(BODY)
</body></html>
DDOC_COMMENT is used to insert comments into the output file.
Highlighting of D code is performed by the following macros:
| Name | Description |
|---|---|
| D_COMMENT | Highlighting of comments |
| D_STRING | Highlighting of string literals |
| D_KEYWORD | Highlighting of D keywords |
| D_PSYMBOL | Highlighting of current declaration name |
| D_PARAM | Highlighting of current function declaration parameters |
The highlighting macros start with DDOC_. They control the formatting of individual parts of the presentation.
| Name | Description |
|---|---|
| DDOC_COMMENT | Inserts a HTML comment. |
| DDOC_DECL | Highlighting of the declaration. |
| DDOC_DECL_DD | Highlighting of the description of a declaration. |
| DDOC_DITTO | Highlighting of ditto declarations. |
| DDOC_SECTIONS | Highlighting of all the sections. |
| DDOC_SUMMARY | Highlighting of the summary section. |
| DDOC_DESCRIPTION | Highlighting of the description section. |
| DDOC_AUTHORS | Highlighting of the authors section. |
| DDOC_BUGS | Highlighting of the bugs section. |
| DDOC_COPYRIGHT | Highlighting of the copyright section. |
| DDOC_DATE | Highlighting of the date section. |
| DDOC_DEPRECATED | Highlighting of the deprecated section. |
| DDOC_EXAMPLES | Highlighting of the examples section. |
| DDOC_HISTORY | Highlighting of the history section. |
| DDOC_LICENSE | Highlighting of the license section. |
| DDOC_RETURNS | Highlighting of the returns section. |
| DDOC_SEE_ALSO | Highlighting of the see-also section. |
| DDOC_STANDARDS | Highlighting of the standards section. |
| DDOC_THROWS | Highlighting of the throws section. |
| DDOC_VERSION | Highlighting of the version section. |
| DDOC_SECTION_H | Highlighting of the section name of a non-standard section. |
| DDOC_SECTION | Highlighting of the contents of a non-standard section. |
| DDOC_MEMBERS | Default highlighting of all the members of a class, struct, etc. |
| DDOC_MODULE_MEMBERS | Highlighting of all the members of a module. |
| DDOC_CLASS_MEMBERS | Highlighting of all the members of a class. |
| DDOC_STRUCT_MEMBERS | Highlighting of all the members of a struct. |
| DDOC_ENUM_MEMBERS | Highlighting of all the members of an enum. |
| DDOC_TEMPLATE_MEMBERS | Highlighting of all the members of a template. |
| DDOC_ENUM_BASETYPE | Highlighting of the type an enum is based upon |
| DDOC_PARAMS | Highlighting of a function parameter section. |
| DDOC_PARAM_ROW | Highlighting of a name=value function parameter. |
| DDOC_PARAM_ID | Highlighting of the parameter name. |
| DDOC_PARAM_DESC | Highlighting of the parameter value. |
| DDOC_BLANKLINE | Inserts a blank line. |
| DDOC_ANCHOR | Expands to a named anchor used for hyperlinking to a particular declaration section. Argument $1 expands to the qualified declaration name. |
| DDOC_PSYMBOL | Highlighting of declaration name to which a particular section is referring. |
| DDOC_PSUPER_SYMBOL | Highlighting of the base type of a class. |
| DDOC_KEYWORD | Highlighting of D keywords. |
| DDOC_PARAM | Highlighting of function parameters. |
| DDOC_BACKQUOTED | Inserts inline code. |
For example, one could redefine DDOC_SUMMARY:
DDOC_SUMMARY = $(GREEN $0)
And all the summary sections will now be green.
Macro Definitions from sc.ini's DDOCFILE
A text file of macro definitions can be created, and specified in sc.ini:
DDOCFILE=myproject.ddoc
Macro Definitions from .ddoc Files on the Command Line
File names on the DMD command line with the extension .ddoc are text files that are read and processed in order.
Macro Definitions Generated by Ddoc
| Macro Name | Content |
|---|---|
| BODY | Set to the generated document text. |
| TITLE | Set to the module name. |
| DATETIME | Set to the current date and time. |
| YEAR | Set to the current year. |
| COPYRIGHT | Set to the contents of any Copyright: section that is part of the module comment. |
| DOCFILENAME | Set to the name of the generated output file. |
| SRCFILENAME | Set to the name of the source file the documentation is being generated from. |
Using Ddoc to generate examples from unit tests
Ddoc can automatically generate usage examples for declarations using unit tests. If a declaration is followed by a documented unit test, the code from the test will be inserted into the example section of the declaration. This avoids the frequent problem of having outdated documentation for pieces of code.
To create a documented unit test just add three forward slashes before the unittest block, like this:
/// unittest { ... }For more information please see the full section on documented unit tests.
Using Ddoc for other Documentation
Ddoc is primarily designed for use in producing documentation from embedded comments. It can also, however, be used for processing other general documentation. The reason for doing this would be to take advantage of the macro capability of Ddoc and the D code syntax highlighting capability.
If the .d source file starts with the string "Ddoc" then it is treated as general purpose documentation, not as a D code source file. From immediately after the "Ddoc" string to the end of the file or any "Macros:" section forms the document. No automatic highlighting is done to that text, other than highlighting of D code embedded between lines delineated with --- lines. Only macro processing is done.
Much of the D documentation itself is generated this way, including this page. Such documentation is marked at the bottom as being generated by Ddoc.
Links to D documentation generators
A list of current D documentation generators which use Ddoc can be found on our wiki page.