As part of the new fanr infrastructure, there is need to completely replace the existing documentation infrastructure. The existing docCompiler pod grew up in an ad-hoc way, and lacks a solid design. More importantly it was written to use the reflection APIs to iterate the pods, types, and slots. This makes it unsuitable for use in generating API docs from a fanr repository.
To refactor the entire design correctly, the following changes are planned:
redesign the docapi text file generated by compiler
create new compilerDoc pod
remove docCompiler pod
Move RichTextStyling from fwt to gfx pod
Move syntax styling from fluxText into new syntax pod
change distribution to not include HTML docs by default
add switch to generate HTML docs after install
Some of these changes are difficult, but not impossible to maintain backward compatibility. If you are using docCompiler or the docapi files generated by the compiler, then please contact me. I doubt anyone is using these, but if you are I will work with you to ensure clean upgrade to new design.
New docapi format
The existing docapi text files packaged into the pod zip files largely contain only the fandoc text for the type and slots. There is a bit of meta-data related to field and parameter default expressions too. But the old format relies on reflection to figure out all the other stuff required for API docs (signatures, facets, etc). For documentation engines working off a fanr repository we need a simple way to generate a complete model of the types, slots, facets, and docs without using reflection or having to decode fcode. Since the vast majority of the content is the fandoc itself, I decided best to just stick to a text format. But we will add all the facets and full signatures. The new format will look like this:
@sys::Transient
public const sys::Str? someField := <<"default expr">>
This is the fandoc of someField.
\
Examples:
here is a code section of the fandoc
The format is easy to read and pretty efficient to easily parse.
New compilerDoc pod
This will be new from-the-ground up redesign of how to model and generate API documentation. It will have three main jobs:
a new set of namespace classes for modeling pods, types, and slots
parser for docapi format
writers for generating HTML for pod index, type api docs, fandoc chapters, and syntax styled code
It will follow conventions used by compiler for modeling Fantom reflection structures but will only define what is needed for documentation. The model will be lazily loaded directly from pod files using the new docapi format. This will make the APIs easy to use for local environments and for fanr repositories.
Syntax Styling
The current docs generate styled HTML for Fantom source code. But they don't do anything for JS, Java, C#, etc. We actually already a have a complete solution for parsing different programming languages for color coding the syntax. This bit of infrastructure is handled by fluxText to allow color coding of a whole bunch of languages: ActionScript, Windows Batch, C++, C#, CSS, Fantom, Java, JS, Perl, Python, Ruby, Sedona, Bash, and XML.
The problem is we need to elegantly pull this code out of fluxText into a pod lower in the dependency chain. First we need to model styling itself. Today this is done via fwt::RichTextStyle, but this is a bit high in the dependency tree for such a core structure. So we move this class exactly as it is into gfx::TextStyle. We will make fwt::RichTextStyle extend this class and deprecate RichTextStyle.
Then we can pull the syntax styling infrastructure out of fluxText into a new pod called syntax (or maybe someone can come up with better name?). It will reuse same etc files which power fluxText, just make it more reusable. This new syntax pod will only have a dependency on sys and gfx.
With this code pulled out, we can have compilerDoc depend on syntax and add support for color coding of example code and source code.
Distro
Today the HTML docs and HTML pages for color coded source code add 4MB to the distro size which is 25% of the overall size. Everything needed to recreate this HTML is packaged up in the zip files themselves. So I'm thinking that we change the distro to not include HTML by default. Rather we will have a simple web server program to browse HTML on a local http port, or a script which will generate HTML offline onto the file system after installation.
In the near future many of the pods will be loaded from fanr rather than bundled in the distro, so that is another angle to consider.
If anyone has thoughts or concerns, please let me know. And I said before if you are using docCompiler or the existing docapi files please let me know!
brian Thu 11 Aug 2011
As part of the new fanr infrastructure, there is need to completely replace the existing documentation infrastructure. The existing
docCompilerpod grew up in an ad-hoc way, and lacks a solid design. More importantly it was written to use the reflection APIs to iterate the pods, types, and slots. This makes it unsuitable for use in generating API docs from a fanr repository.To refactor the entire design correctly, the following changes are planned:
Some of these changes are difficult, but not impossible to maintain backward compatibility. If you are using docCompiler or the docapi files generated by the compiler, then please contact me. I doubt anyone is using these, but if you are I will work with you to ensure clean upgrade to new design.
New docapi format
The existing docapi text files packaged into the pod zip files largely contain only the fandoc text for the type and slots. There is a bit of meta-data related to field and parameter default expressions too. But the old format relies on reflection to figure out all the other stuff required for API docs (signatures, facets, etc). For documentation engines working off a fanr repository we need a simple way to generate a complete model of the types, slots, facets, and docs without using reflection or having to decode fcode. Since the vast majority of the content is the fandoc itself, I decided best to just stick to a text format. But we will add all the facets and full signatures. The new format will look like this:
@sys::Transient public const sys::Str? someField := <<"default expr">> This is the fandoc of someField. \ Examples: here is a code section of the fandocThe format is easy to read and pretty efficient to easily parse.
New compilerDoc pod
This will be new from-the-ground up redesign of how to model and generate API documentation. It will have three main jobs:
It will follow conventions used by compiler for modeling Fantom reflection structures but will only define what is needed for documentation. The model will be lazily loaded directly from pod files using the new docapi format. This will make the APIs easy to use for local environments and for fanr repositories.
Syntax Styling
The current docs generate styled HTML for Fantom source code. But they don't do anything for JS, Java, C#, etc. We actually already a have a complete solution for parsing different programming languages for color coding the syntax. This bit of infrastructure is handled by fluxText to allow color coding of a whole bunch of languages: ActionScript, Windows Batch, C++, C#, CSS, Fantom, Java, JS, Perl, Python, Ruby, Sedona, Bash, and XML.
The problem is we need to elegantly pull this code out of fluxText into a pod lower in the dependency chain. First we need to model styling itself. Today this is done via fwt::RichTextStyle, but this is a bit high in the dependency tree for such a core structure. So we move this class exactly as it is into
gfx::TextStyle. We will make fwt::RichTextStyle extend this class and deprecate RichTextStyle.Then we can pull the syntax styling infrastructure out of fluxText into a new pod called
syntax(or maybe someone can come up with better name?). It will reuse same etc files which power fluxText, just make it more reusable. This newsyntaxpod will only have a dependency onsysandgfx.With this code pulled out, we can have
compilerDocdepend onsyntaxand add support for color coding of example code and source code.Distro
Today the HTML docs and HTML pages for color coded source code add 4MB to the distro size which is 25% of the overall size. Everything needed to recreate this HTML is packaged up in the zip files themselves. So I'm thinking that we change the distro to not include HTML by default. Rather we will have a simple web server program to browse HTML on a local http port, or a script which will generate HTML offline onto the file system after installation.
In the near future many of the pods will be loaded from fanr rather than bundled in the distro, so that is another angle to consider.
If anyone has thoughts or concerns, please let me know. And I said before if you are using docCompiler or the existing docapi files please let me know!