Where should you document your developments? Well, at several places, indeed, depending on the documentation we are talking about:
This is quite easy indeed. It’s excellent practice to add
docstring to your
python code. The good part of it is that tools like pyDev can automatically
read it. Also your python shell can (try
help()), and so does iPython
? for example). Python stores every docstring in
the special attribute
Pylint will, by default, complain for every method/class/function left without a docstring.
Releases documentation can be found in 2 places: release notes, and github wiki:
release notes are automatically created from the first comment in the pull requests, please describe the changes between BEGRINRELEASENOTES and ENDRELEASENOTES as presented by the template provided
The github wiki can contain a section, for each DIRACGrid repository, highlighting update operations, for example the DIRAC releases notes are linked from the DIRAC wiki main page.
As said at the beginning of this guide, this documentation is in git at
It is very easy to contribute to it, and you are welcome to do that. You don’t
even have to clone the repository: github lets you edit it online.
This documentation is written in
RST and it is compiled using
Some parts of the documentation can use UML diagrams. They are generated from .uml files
with plantuml. Sphinx support plantuml but ReadTheDocs
didn’t, so you have to convert .uml in .png with
java -jar plantuml.jar file.uml.
The agent, service and executor options are documented in their respective module docstring via literal include of their options in the ConfigTemplate.cfg:
.. literalinclude:: ../ConfigTemplate.cfg :start-after: ##BEGIN MyComponent :end-before: ##END :dedent: 2 :caption: MyComponent options
Around the section in the ConfigTemplate.cfg configuring the component the ##BEGIN MyComponent and ##END tags need set so that the include is restricted to the section belonging to the component. The options :dedent: and :caption: are optional, but create a nicer output.
The DIRAC documentation is created using
sphinx and makes use of the
diracdoctools to create the code documentation, command references, and
ConfigTemplates into one file. The
located in the
DIRAC/docs folder, but they can be
pip installed for use
outside of DIRAC.
For local development the easiest way to build the docs is by running these commands from a standard Python 3 DIRAC development environment (see Creating a development environment with Conda.)
pip install docs/ cd docs make htmlall python -m http.server -d build/html/
You can now browse the docs locally by going to the URL shown in your terminal.
In the sphinx configuration file (
source/conf.py), the functionality can
then be called to create code reference, command reference and concatenated CFG
if os.environ.get("READTHEDOCS") == "True": setUpReadTheDocsEnvironment(moduleName="DIRAC", location="../../src") # re-create the RST files for the command references LOG.info("Building command reference") from diracdoctools.cmd.commandReference import run as buildCommandReference buildCommandReference(configFile="../docs.conf") # singlehtml build needs too much memory, so we need to create less code documentation buildType = "limited" if any("singlehtml" in arg for arg in sys.argv) else "full" LOG.info("Chosing build type: %r", buildType) from diracdoctools.cmd.codeReference import run as buildCodeDoc buildCodeDoc(configFile="../docs.conf", buildType=buildType) # Update dirac.cfg LOG.info("Concatenating dirac.cfg") from diracdoctools.cmd.concatcfg import run as updateCompleteDiracCFG updateCompleteDiracCFG(configFile="../docs.conf")
The configuration for
diracdoctools is done via a configuration file
located in the docs folder. Just copy the file
adapt it to your needs. All options are mandatory unless otherwise stated. If
certain features are not used, simply leave the values empty
[Docs] # name of the module to documented, DIRAC, voDIRAC, ExtensionDIRAC module_name = DIRAC # path to the source folder relative to docs.conf source_folder = ../src/DIRAC [Code] # building code reference # where to place the code reference docs_target_path = ./source/CodeDocumentation # where Custom docstrings can be found, see # DIRAC/docs/diracdoctools/CustomizedDocs for an example customdocs_folder = ./diracdoctools/CustomizedDocs # add :private-members: to autodoc for matching module document_private_members = FCConditionsParser # add :no-inherited: to autodoc for matching module # Useful to avoid markup errors where the inherited members come from a module # whose docstrings are not reST formatted no_inherited_members = DIRAC.Core.Utilities.Graphs.GraphUtilities, DIRAC.DataManagementSystem.private.HttpStorageAccessHandler # only creating dummy files, because they cannot be safely imported due to sideEffects create_dummy_files = lfc_dfc_copy, lfc_dfc_db_copy, JobWrapperTemplate, PlotCache, PlottingHandler # do not include these files in the documentation tree ignore_folders = diracdoctools, /test, /scripts ignore_files = setup.py [CFG] # concatenating ConfigTemplates # which file to use as a base base_file = ../dirac.cfg # where to place the resulting file target_file = source/AdministratorGuide/Configuration/ExampleConfig.rst [Commands] # list of commands which are not executed to get doc files, rst files have to be # created by hand ignore_commands= # does not have --help, starts compiling externals dirac-compile-externals, # does not have --help dirac-install-client, # does not have --help dirac-framework-self-ping # scripts where the module docstring should be added to the documentation add_module_docstring = dirac-install # arbitrary number of these sections can be added, they result in folders with # rst files for each command matching the pattern. all options are mandatory, # except indexFile and prefix [commands.admin] # pattern to match in the full path of the command names pattern = admin, accounting, FrameworkSystem, framework, install, utils, dirac-repo-monitor, dirac-jobexec, dirac-info, ConfigurationSystem, Core, rss, transformation, stager # title of the section title = Admin # this list of patterns will reject scripts that are matched by the patterns above exclude = # the scripts in this list are added to the indexFile, but the rst file has to be provided manually manual = dirac-cert-convert.sh # where to place the rst files sectionPath = source/AdministratorGuide/CommandReference # optional: only if a hand curated rst file exists indexFile = source/AdministratorGuide/CommandReference/index.rst # optional: prefix to add when creating reference anchors, anchor will be prefix_scriptName prefix = admin [commands.dms] pattern = dms title = Data Management manual = exclude = sectionPath = source/UserGuide/CommandReference/%(title)s [commands.wms] pattern = wms title = Workload Management manual = exclude = sectionPath = source/UserGuide/CommandReference/%(title)s [commands.z_section3] pattern = dirac-proxy, dirac-info, myproxy, -resource- title = Others exclude = manual = dirac-cert-convert.sh sectionPath = source/UserGuide/CommandReference/%(title)s
For local testing of the documentation, the scripts can also be called
directly, like this example from the
htmlall: diracdoctools/scripts/dirac-docs-build-commands.py diracdoctools/scripts/dirac-docs-build-code.py diracdoctools/scripts/dirac-docs-concatenate-diraccfg.py $(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html @echo @echo "Build finished. The HTML pages are in $(BUILDDIR)/html."
The code reference is either created by calling
diracdoctools.cmd.codeReference or by invoking the script
dirac-docs-build-code.py. This creates an
rst file for each python
autodoc to document all classes inside those modules. The
actual documentation is build when sphinx is invoked, which must be able to
import the modules and all their dependencies
The command references can be created by calling
diracdoctools.cmd.commandReference, or by calling the
[commands.section] will result in a
list of commands with links to their documentation, which is based on the output
--help. The resulting
index.rst has to be included explicitly
in the documentation. If
indexFile is specified it has to contain all the
links itself, or warnings are generated for missing and superfluous entries.
The parsing of the
--help output is extremely limited and naive. You must not end a line with a
: unless you intend to create a verbatim block after it.
If you developed your own systems, you can concatenate all the settings defined
ConfigTemplate.cfg files into one large file. You need a base
dirac.cfg file and a location where to put the final result