Solving the Midgard documentation issues
-
Henri Bergius
Solving the Midgard documentation issues
Fri February 27 2009 13:57:05 UTCHi, all
Vinland is coming soon, and I wanted to start tackling the biggest
persistent issue with Midgard: the lack or bad quality of
documentation.
The problem mostly is that Midgard is a 10 year old software project,
and documentation has been gathered from very many sources, targeted
at very many different phases of Midgard. If you look at MidCOM
documentation, most of it has been written for pre 2.4 MidCOM. While
some of that has been updated, it is confusing to figure out what
parts are valid and what are not.
Same thing goes with all the tutorials and "recipes" and user
instructions for Midgard: some of the material is quite good, but it
is all jumbled together into the Midgard wiki without clear indication
what is valid and where.
The only area where things are reasonably good is the Midgard API
documentation. There Piotras has been doing very good work in
maintaining the docs on how the new API (MgdSchema, midgard_connection
etc) works.
In last Gathering we appointed Solt and Neithan as documentation
maintainers, but it appears they haven't had the time to do any work
on this. And in any case, the current amount of obsolete documentation
is just overwhelming.
So, how to make a fresh start?
Ragnaroek's documentation viewer (shipped as part of the packages) is
a quite good start, I think. But Ragnaroek is so huge package, getting
all the documentation in there would be a huge task.
With Vinland we can do things better. But only if we figure out the
right tools and methods *right now* and stick to them. If we let the
development go any further, we will be unable to catch up.
My proposal is the following:
* All documentation shall be maintained as .markdown files under the
"documentation" folder of each MidCOM package
* In addition, Midgard will have its own documentation, also shipped
as .markdown, installed under a standard location
* We will make a MidCOM-based documentation viewer, that will:
- Display all markdown documentation of all components (and Midgard)
- Reflect the MidCOM configuration system to know how the installed
components operate, and display all relevant data about each route in
a nice way
- Use PHP5 reflection API to generate API docs of all classes and
methods in Midgard, MidCOM and components
- Use Midgard reflector to generate docs about all installed MgdSchemas
Yes, this means we will kill the Midgard documentation wiki (hopefully
merging all the "good stuff" to the new format). And I think the same
should be done with Flack's wiki
(http://midgardwiki.contentcontrol-berlin.de/index.php/Main_Page) if
he agrees. The old documentation folder can then be replaced with this
new documentation browser component so there is also a set of docs
available without installing Midgard.
One place to maintain documentation, with proper version control
(git-svn <3), instead of multiple places should help us get the
documentation there, and kept up-to-date.
The cool thing about using PHP5 reflection
(http://fi.php.net/oop5.reflection) for API docs is that we can
generate very nice PhpDoc-like output tailored for our own needs, but
unlike PhpDoc, this is rendered "live", so updating an API
documentation page means just editing PhpDoc comment blocks in a PHP
file, saving and reloading the page. Just like you do with the code.
A major change to the Ragnaroek documentation browser is that we must
support subdirectories, and generate deeper table-of-contents based on
that. And I think images and other files related to the docs should be
stored in the documentation tree so Markdown editing becomes simpler
(no symlinks needed to enable preview).
Another bonus is that Midgard core packages (and language bindings)
would in this case ship their own documentation, making them more
useful also without MidCOM.
This is something we need to agree on. And soon.
Proposed steps:
* Discuss the idea on dev (next few days)
* Implement a first version of the doc browser (next week)
* Write basic documentation on MidCOM3 to the new format (before Vinland final)
* Commit to the decision (Midgard Gathering)
* Start porting docs if we so decide (during and after the Gathering)
/Henri
PS. Why .markdown instead of .txt? So that various tools can recognize
and render the content correctly. See for instance
http://www.mdk.org.pl/2009/2/10/quicklook-for-markdown
PPS. In addition to .markdown we may support also .html files if we
see that as sensible.
--
Henri Bergius
Motorcycle Adventures and Free Software
http://bergie.iki.fi/
Skype: henribergius
Jabber: henri.bergius@gmail.com
Microblogs: http://bergie.jaiku.com/ http://twitter.com/bergie
Sent from: Helsinki Southern Finland Finland.
_______________________________________________
dev mailing list
dev@lists.midgard-project.org
http://lists.midgard-project.org/mailman/listinfo/dev -
-
Re: [midgard-dev] Solving the Midgard documentation issues
Fri February 27 2009 14:28:20 UTCHenri Bergius writes:
> Hi, all
Hi!
> My proposal is the following:
>
> * All documentation shall be maintained as .markdown files under the
> "documentation" folder of each MidCOM package
> * In addition, Midgard will have its own documentation, also shipped
> as .markdown, installed under a standard location
What do you mean by 'Midgard' here? Core? Bindings? Midgard Concepts?
> - Use PHP5 reflection API to generate API docs of all classes and
> methods in Midgard, MidCOM and components
+10
With PHP5 reflection API we have the same doc format for C extension and
for pure PHP code.
Piotras
_______________________________________________
dev mailing list
dev@lists.midgard-project.org
http://lists.midgard-project.org/mailman/listinfo/dev -
-
Re: [midgard-dev] Solving the Midgard documentation issues
Fri February 27 2009 14:34:22 UTCHi,
On Fri, Feb 27, 2009 at 4:28 PM, Piotr Pokora <piotrek.pokora@gmail.com> wrote:
> What do you mean by 'Midgard' here? Core? Bindings? Midgard Concepts?
All of it :-)
Theoretically Midgard Core should contain documentation on core API,
concepts and such things, and for instance Midgard-PHP should contain
the PHP-level docs and instructions how to run with Apache, Lighty,
whatever.
> With PHP5 reflection API we have the same doc format for C extension and
> for pure PHP code.
Yep. Though, have you checked if you can supply "PhpDocs" also via the
extension? At least currently when reflected the MgdSchema classes do
not return anything via $method->getDocComment();
> Piotras
/Henri
--
Henri Bergius
Motorcycle Adventures and Free Software
http://bergie.iki.fi/
Skype: henribergius
Jabber: henri.bergius@gmail.com
Microblogs: http://bergie.jaiku.com/ http://twitter.com/bergie
Sent from: Helsinki Southern Finland Finland.
_______________________________________________
dev mailing list
dev@lists.midgard-project.org
http://lists.midgard-project.org/mailman/listinfo/dev -
-
Re: [midgard-dev] Solving the Midgard documentation issues
Fri February 27 2009 14:38:19 UTCHi!
I didn't take a look at Vinland yet and I guess I won't be able to do so
before the end of this year (still too much stuff to port to
Ragnaroek...), so consider this an outsider's point of view:
First of all, the concept of "self-documenting" packages (i.e. shipping
docs with the packages themselves) seems nice, but there should
definately be an online version as well, if only for increased
visibility on Google and to read about things not present in your own
installation.
Also, I'm a bit skeptical if this new documentation viewer will really
be able to include all the niceties of PHPdoc and wikis respectively.
F.x. in PHPdoc, I can render the source code and get a version where
each recognized class/function/variable name is linked to the respective
declaration and documentation (i.e. @param midcom_core_privilege_db will
always link to the docblock of the class and so on). Also, I cannot
stress this enough, IMHO a documentation without a builtin search
function is utterly useless, so if you really want to handle all
documentation as builtin, you'd have to force the users to install and
configure indexers as well.
Tracking the documentation in the repository is definately a good idea
and widely used AFAIK. Some projects include their entire website in the
VCS and then deploy it using custom build scripts. Maybe this would be
an idea for Midgard as well?
Bye,
Andreas
Henri Bergius schrieb:
> Hi, all
>
> Vinland is coming soon, and I wanted to start tackling the biggest
> persistent issue with Midgard: the lack or bad quality of
> documentation.
>
> The problem mostly is that Midgard is a 10 year old software project,
> and documentation has been gathered from very many sources, targeted
> at very many different phases of Midgard. If you look at MidCOM
> documentation, most of it has been written for pre 2.4 MidCOM. While
> some of that has been updated, it is confusing to figure out what
> parts are valid and what are not.
>
> Same thing goes with all the tutorials and "recipes" and user
> instructions for Midgard: some of the material is quite good, but it
> is all jumbled together into the Midgard wiki without clear indication
> what is valid and where.
>
> The only area where things are reasonably good is the Midgard API
> documentation. There Piotras has been doing very good work in
> maintaining the docs on how the new API (MgdSchema, midgard_connection
> etc) works.
>
> In last Gathering we appointed Solt and Neithan as documentation
> maintainers, but it appears they haven't had the time to do any work
> on this. And in any case, the current amount of obsolete documentation
> is just overwhelming.
>
> So, how to make a fresh start?
>
> Ragnaroek's documentation viewer (shipped as part of the packages) is
> a quite good start, I think. But Ragnaroek is so huge package, getting
> all the documentation in there would be a huge task.
>
> With Vinland we can do things better. But only if we figure out the
> right tools and methods *right now* and stick to them. If we let the
> development go any further, we will be unable to catch up.
>
> My proposal is the following:
>
> * All documentation shall be maintained as .markdown files under the
> "documentation" folder of each MidCOM package
> * In addition, Midgard will have its own documentation, also shipped
> as .markdown, installed under a standard location
> * We will make a MidCOM-based documentation viewer, that will:
> - Display all markdown documentation of all components (and Midgard)
> - Reflect the MidCOM configuration system to know how the installed
> components operate, and display all relevant data about each route in
> a nice way
> - Use PHP5 reflection API to generate API docs of all classes and
> methods in Midgard, MidCOM and components
> - Use Midgard reflector to generate docs about all installed MgdSchemas
>
> Yes, this means we will kill the Midgard documentation wiki (hopefully
> merging all the "good stuff" to the new format). And I think the same
> should be done with Flack's wiki
> (http://midgardwiki.contentcontrol-berlin.de/index.php/Main_Page) if
> he agrees. The old documentation folder can then be replaced with this
> new documentation browser component so there is also a set of docs
> available without installing Midgard.
>
> One place to maintain documentation, with proper version control
> (git-svn <3), instead of multiple places should help us get the
> documentation there, and kept up-to-date.
>
> The cool thing about using PHP5 reflection
> (http://fi.php.net/oop5.reflection) for API docs is that we can
> generate very nice PhpDoc-like output tailored for our own needs, but
> unlike PhpDoc, this is rendered "live", so updating an API
> documentation page means just editing PhpDoc comment blocks in a PHP
> file, saving and reloading the page. Just like you do with the code.
>
> A major change to the Ragnaroek documentation browser is that we must
> support subdirectories, and generate deeper table-of-contents based on
> that. And I think images and other files related to the docs should be
> stored in the documentation tree so Markdown editing becomes simpler
> (no symlinks needed to enable preview).
>
> Another bonus is that Midgard core packages (and language bindings)
> would in this case ship their own documentation, making them more
> useful also without MidCOM.
>
> This is something we need to agree on. And soon.
>
> Proposed steps:
>
> * Discuss the idea on dev (next few days)
> * Implement a first version of the doc browser (next week)
> * Write basic documentation on MidCOM3 to the new format (before Vinland final)
> * Commit to the decision (Midgard Gathering)
> * Start porting docs if we so decide (during and after the Gathering)
>
> /Henri
>
> PS. Why .markdown instead of .txt? So that various tools can recognize
> and render the content correctly. See for instance
> http://www.mdk.org.pl/2009/2/10/quicklook-for-markdown
>
> PPS. In addition to .markdown we may support also .html files if we
> see that as sensible.
>
_______________________________________________
dev mailing list
dev@lists.midgard-project.org
http://lists.midgard-project.org/mailman/listinfo/dev -
-
Re: [midgard-dev] Solving the Midgard documentation issues
Fri February 27 2009 14:43:14 UTCHenri Bergius writes:
> Hi,
Hi!
> On Fri, Feb 27, 2009 at 4:28 PM, Piotr Pokora <piotrek.pokora@gmail.com> wrote:
>> What do you mean by 'Midgard' here? Core? Bindings? Midgard Concepts?
>
> All of it :-)
>
> Theoretically Midgard Core should contain documentation on core API,
> concepts and such things, and for instance Midgard-PHP should contain
> the PHP-level docs and instructions how to run with Apache, Lighty,
> whatever.
Documenting API and concepts or howto is different thing.
Can we quickly convert markdown to pdf or html using command line scripts?
If I understood correctly markdown files will be shipped under doc
directory (with other subdirectories),
so there will be a need to convert them to other formats for binary
packages for example.
>> With PHP5 reflection API we have the same doc format for C extension and
>> for pure PHP code.
>
> Yep. Though, have you checked if you can supply "PhpDocs" also via the
> extension? At least currently when reflected the MgdSchema classes do
> not return anything via $method->getDocComment();
Didn't chek that. Will try asap.
Piotras
_______________________________________________
dev mailing list
dev@lists.midgard-project.org
http://lists.midgard-project.org/mailman/listinfo/dev -
-
Re: [midgard-dev] Solving the Midgard documentation issues
Fri February 27 2009 14:46:59 UTCHi,
On Fri, Feb 27, 2009 at 4:38 PM, Andreas Flack
<flack@contentcontrol-berlin.de> wrote:
> I didn't take a look at Vinland yet and I guess I won't be able to do so
> before the end of this year (still too much stuff to port to
> Ragnaroek...), so consider this an outsider's point of view:
I think whatever we decide with Vinland will also trickle down to
Ragnaroek eventually.
> First of all, the concept of "self-documenting" packages (i.e. shipping
> docs with the packages themselves) seems nice, but there should
> definately be an online version as well, if only for increased
> visibility on Google and to read about things not present in your own
> installation.
Fully agreed. That is why we can run the same documentation browser
component on m-p.org too.
> Also, I'm a bit skeptical if this new documentation viewer will really
> be able to include all the niceties of PHPdoc and wikis respectively.
> F.x. in PHPdoc, I can render the source code and get a version where
> each recognized class/function/variable name is linked to the respective
> declaration and documentation (i.e. @param midcom_core_privilege_db will
> always link to the docblock of the class and so on).
Some of this we can do with the reflection API. And this doesn't mean
we couldn't still generate PhpDocs, the point is just having a more
convenient viewer for the data always available.
For instance, the inheritance charts are something we definitely do
not want to reimplement :-)
> Also, I cannot stress this enough, IMHO a documentation without a builtin search
> function is utterly useless
Again, the documentation viewer running on m-p.org can be searchable.
And if we end up bundling Tracker as the Midgard indexing solution at
some point of Vinland+N, then we can start indexing the docs on every
install.
BTW, you did not comment on the possibility of using your wiki
contents in this new documentation system. What do you think?
> Andreas
/Henri
--
Henri Bergius
Motorcycle Adventures and Free Software
http://bergie.iki.fi/
Skype: henribergius
Jabber: henri.bergius@gmail.com
Microblogs: http://bergie.jaiku.com/ http://twitter.com/bergie
Sent from: Helsinki Southern Finland Finland.
_______________________________________________
dev mailing list
dev@lists.midgard-project.org
http://lists.midgard-project.org/mailman/listinfo/dev -
-
Re: [midgard-dev] Solving the Midgard documentation issues
Fri February 27 2009 14:47:47 UTCAndreas Flack writes:
> Hi!
Hi!
> I didn't take a look at Vinland yet and I guess I won't be able to do so
> before the end of this year (still too much stuff to port to
> Ragnaroek...), so consider this an outsider's point of view:
>
> First of all, the concept of "self-documenting" packages (i.e. shipping
> docs with the packages themselves) seems nice, but there should
> definately be an online version as well, if only for increased
> visibility on Google and to read about things not present in your own
> installation.
I think we should generate online one per every release.
> Also, I'm a bit skeptical if this new documentation viewer will really
> be able to include all the niceties of PHPdoc and wikis respectively.
> F.x. in PHPdoc, I can render the source code and get a version where
> each recognized class/function/variable name is linked to the respective
> declaration and documentation (i.e. @param midcom_core_privilege_db will
> always link to the docblock of the class and so on).
What to do with *PHP* classes, variables and methods declared and
defined in C extension?
Piotras
_______________________________________________
dev mailing list
dev@lists.midgard-project.org
http://lists.midgard-project.org/mailman/listinfo/dev -
-
Re: [midgard-dev] Solving the Midgard documentation issues
Fri February 27 2009 14:50:54 UTCHi,
On Fri, Feb 27, 2009 at 4:43 PM, Piotr Pokora <piotrek.pokora@gmail.com> wrote:
> Documenting API and concepts or howto is different thing.
> Can we quickly convert markdown to pdf or html using command line scripts?
To HTML we can, yes, and from HTML there is reasonably easy path to PDF.
Markdown is nice in the sense that there are multiple different
implementations of the parser. So you're not stuck with only the PHP
implementation MidCOM uses, but can also use other tools:
http://daringfireball.net/projects/markdown/
https://www.ohloh.net/tags/markdown/parser/text
> If I understood correctly markdown files will be shipped under doc
> directory (with other subdirectories),
> so there will be a need to convert them to other formats for binary
> packages for example.
Why? I think much of the docs shipped by other packages is also in
normal, shell-readable text format what Markdown also is.
Though yes, we may want to generate stuff like man pages out of it.
This is something we can probably do in our build scripts.
> Piotras
/Henri
--
Henri Bergius
Motorcycle Adventures and Free Software
http://bergie.iki.fi/
Skype: henribergius
Jabber: henri.bergius@gmail.com
Microblogs: http://bergie.jaiku.com/ http://twitter.com/bergie
Sent from: Helsinki Southern Finland Finland.
_______________________________________________
dev mailing list
dev@lists.midgard-project.org
http://lists.midgard-project.org/mailman/listinfo/dev -
-
Re: [midgard-dev] Solving the Midgard documentation issues
Fri February 27 2009 14:51:18 UTC>
> BTW, you did not comment on the possibility of using your wiki
> contents in this new documentation system. What do you think?
Well, yeah, no problem. I didn't enter a licence agreement for the wiki
because I was too lazy to read up on it, but I have no problem with that
whatsoever. The more (users/readers), the merrier :-)
Bye,
Andreas
>
>> Andreas
>
> /Henri
>
_______________________________________________
dev mailing list
dev@lists.midgard-project.org
http://lists.midgard-project.org/mailman/listinfo/dev -
-
Re: [midgard-dev] Solving the Midgard documentation issues
Fri February 27 2009 15:04:47 UTCHenri Bergius writes:
> Hi,
Hi!
> Yep. Though, have you checked if you can supply "PhpDocs" also via the
> extension? At least currently when reflected the MgdSchema classes do
> not return anything via $method->getDocComment();
God! Why did you create Zend developers?
Or maybe you did it as wonder of crap?
Part of Zend code:
if (ce->type == ZEND_USER_CLASS && ce->doc_comment) {
RETURN_STRINGL(ce->doc_comment, ce->doc_comment_len, 1);
}
ce stands for class entry. So even if I put some docs there it won't be
returned cause
classes registered in extensions are not ZEND_USER_CLASS.
Piotras
_______________________________________________
dev mailing list
dev@lists.midgard-project.org
http://lists.midgard-project.org/mailman/listinfo/dev -
