NetBeans Architecture Answers for Module System API module

General Information

Question (arch-what): What is this project good for?
Answer: The Modules API lies at the core of NetBeans and describes how plug-in modules are added and managed. ModulesAPI
Question (arch-overall): Describe the overall architecture.
WARNING: Question with id="arch-overall" has not been answered!
Question (arch-usecases): Describe the main use cases of the new API. Who will use it under what circumstances? What kind of code would typically need to be written to use the module?
Answer: For general overview of the concepts, together with code samples, see chapter 2, of NetBeans Platform for Beginners by Jason Wexbridge and Walter Nyland. Gory technical details are explained on API overview page . Other interesting topic(s) follow:

How a classpath of my module is constructed?

The NetBeans is defacto a container that manages individual module's lifecycle and other runtime aspects. One of the important things is that it creates a runtime classpath for provided modules based on dependencies they specify in their manifests. The overview of the runtime infrastructure is a good starting place for everyone who wishes to learn more about the NetBeans runtime container behaviour.

Runtime compatibility patches

To maintain binary compatibility, method implementations may be injected at runtime, in a form of a superclass in the class' inheritance hierarchy. Modules compiled against older version of APIs which contains MethodReferences to methods removed from the oficial APIs will be then linked according to JVM Resolution algorithm to a matching method present in the superclass of the referenced type.

Annotations are used to instruct the ClassLoader to make transformations to the API classes. PatchFor causes the annotated class to be injected as a superclass of the API class identified by the annotation's value. ConstructorDelegate marks a method, which is called as constructor implementation in the case that it is necessary to preserve a constructor for binary compatibility.

Question (arch-time): What are the time estimates of the work?
WARNING: Question with id="arch-time" has not been answered!
Question (arch-quality): How will the quality of your code be tested and how are future regressions going to be prevented?
WARNING: Question with id="arch-quality" has not been answered!
Question (arch-where): Where one can find sources for your module?
WARNING: Question with id="arch-where" has not been answered!

Project and platform dependencies

Question (dep-nb): What other NetBeans projects and modules does this one depend on?
Answer: UtilitiesAPI And in the implementation only: FilesystemsAPI

The default answer to this question is:


These modules are required in project.xml:

• UtilitiesAPI - The module is needed for compilation. The module is used during runtime. Specification version 9.3 is required.
• WeakListener.setAccessible - The module is needed for compilation. The module is used during runtime. Specification version 9.3 is required.
• LookupAPI - The module is needed for compilation. The module is used during runtime. Specification version 8.0 is required.
Question (dep-non-nb): What other projects outside NetBeans does this one depend on?
Answer: None, other than any available SAX parser.
Question (dep-platform): On which platforms does your module run? Does it run in the same way on each?
Answer: Any.
Question (dep-jre): Which version of JRE do you need (1.2, 1.3, 1.4, etc.)?
Answer: 1.3
Question (dep-jrejdk): Do you require the JDK or is the JRE enough?
Answer: JRE

Deployment

Question (deploy-jar): Do you deploy just module JAR file(s) or other files as well?
Answer: The API portion is inside openide.jar; the implementation refers to this and is inside core.jar.
Question (deploy-nbm): Can you deploy an NBM via the Update Center?
Answer: Yes (as part of openide.nbm and core.nbm).
Question (deploy-shared): Do you need to be installed in the shared location only, or in the user directory only, or can your module be installed anywhere?
Answer: Must be installed in the shared location as it is part of the very core of NetBeans.
Question (deploy-packages): Are packages of your module made inaccessible by not declaring them public?
Answer:

The entire API is one public package. ModulesAPI

The implementation is in another package and is not considered public, though it is made available to autoupdate and apisupport as these special modules deal directly with the module system at a deeper level than the API provides for. CoreModulesAPI

Question (deploy-dependencies): What do other modules need to do to declare a dependency on this one, in addition to or instead of the normal module dependency declaration (e.g. tokens to require)?
WARNING: Question with id="deploy-dependencies" has not been answered!

Compatibility with environment

Question (compat-i18n): Is your module correctly internationalized?
Answer: Yes
Question (compat-standards): Does the module implement or define any standards? Is the implementation exact or does it deviate somehow?
Answer: It defines and implements the Modules API. No intentional deviations.
Question (compat-version): Can your module coexist with earlier and future versions of itself? Can you correctly read all old settings? Will future versions be able to read your current settings? Can you read or politely ignore settings stored by a future version?
Answer: Backward compatibility of setting storage for the list of configured modules is considered a design priority. The implementation of the Modules API uses an XML format to store this list; it has an associated versioned DTD, and the 1.0 format additionally supports expansion through arbitrary named parameters. Also, the modules themselves can be considered settings in that a user directory may contain them, so backward compatibility of the directory layout is maintained.
Question (compat-deprecation): How the introduction of your project influences functionality provided by previous version of the product?
WARNING: Question with id="compat-deprecation" has not been answered!

Access to resources

Question (resources-file): Does your module use java.io.File directly?
Answer:

Yes, module JARs and associated resource JARs must be real files. They are loaded as defined by module enablement XML files.

Module enablement XML files are loaded via Filesystems. The public API (partially) specifies them only in these terms. autoupdate uses Filesystems to manipulate them when necessary, though it relies on some additional implementation-specific knowledge of their format (which is fairly stable, especially since changes are limited by compatibility constraints on old user directories). The NetBeans build scripts use some additional implementation knowledge to pregenerate suitable XML files for modules included in the application distribution.

Question (resources-layer): Does your module provide own layer? Does it create any files or folders in it? What it is trying to communicate by that and with which components?
Answer: The Modules API defines some layer folders:
• Modules/*.xml (read from and written to)
• ModuleAutoDeps/*.xml (only read from)
Question (resources-read): Does your module read any resources from layers? For what purpose?
Answer: XML files are read from the two folders defined by the API. Note however that since these folders control the module system itself, contents generally should not be placed in module layers, but in the system/ subfolder of the installation or user directory (generally, in the NBM).
Question (resources-mask): Does your module mask/hide/override any resources provided by other modules in their layers?
Answer: No.
Question (resources-preferences): Does your module uses preferences via Preferences API? Does your module use NbPreferences or or regular JDK Preferences ? Does it read, write or both ? Does it share preferences with other modules ? If so, then why ?
WARNING: Question with id="resources-preferences" has not been answered!

Lookup of components

Question (lookup-lookup): Does your module use org.openide.util.Lookup or any similar technology to find any components to communicate with? Which ones?
Answer: All registered InstalledFileLocator instances are queried and used to implement InstalledFileLocator.getDefault().
Question (lookup-register): Do you register anything into lookup for other code to find?
Answer:
• TestModuleDeployer (from META-INF), used by apisupport
• InstalledFileLocator (from META-INF), used by its public default method
• an instance of every known ModuleInfo, via a custom lookup insertion (specified in API)
• an instance of ClassLoader, via a custom lookup insertion (used broadly)
Question (lookup-remove): Do you remove entries of other modules from lookup?
Answer: No.

Execution Environment

Question (exec-property): Is execution of your code influenced by any environment or Java system (System.getProperty) property? On a similar note, is there something interesting that you pass to java.util.logging.Logger? Or do you observe what others log?
Answer: netbeans.home - NetBeans installation directory. netbeans.user - User directory. netbeans.dirs - Additional installation component directories. netbeans.systemclassloader.patches - Classpath appended to the system class loader. Used for automated testing infrastructure. netbeans.classpath - May be used to prepend items to the same class loader as is used for openide.jar and core.jar, similarly to JARs found in lib/patches/. netbeans.cache.manifests - By default true, may be turned off to disable manifest caching. netbeans.patches.MODULE.CODE.NAME.BASE - Patch directories or JARs to add to the class loader for a module, besides patches/MODULE-CODE-NAME-BASE/*.jar. netbeans.preresolve.classes - If true, all module classes are forcibly loaded at startup, to help detect possible linkage errors. org.netbeans.core.modules.NbInstaller.noAutoDeps - Disables automatic module dependency upgrades. org.netbeans.core.modules - If set to 0, enables logging for the module system. org.netbeans.core.modules.NbInstaller.NO_COMPAT_AUTO_TRANSITIVE_DEPS - Disabled automatic transitive class loader dependencies for old modules. netbeans.modules.quiet - Avoids text messaging (other than ErrorManager logging) to the console from the module system. netbeans.moduleitem.dontverifyclassloader - Suppresses checks to ensure that module section classes are actually loaded from the module itself. netbeans.full.hack - Avoids using GUI when user-visible error conditions occur. netbeans.mainclass - Name of class from core.jar which will handle most of the startup sequence; by default, org.netbeans.core.Main. netbeans.security.nocheck - Suppress security checks in the VM, even from "untrusted" code. netbeans.classloader.verbose - Prints messages when resources or classes are loaded from JARs. netbeans.cache.layers -

Used to control the XML layer cache mechanism. Value may be a fully-qualified class name to load as a manager (implement org.netbeans.core.projects.cache.LayerCacheManager), or - to disable caching and always parse the XML layers directly. Current default is to use a binary cache manager.

Question (exec-component): Is execution of your code influenced by any (string) property of any of your components?
Answer: No.
Question (exec-ant-tasks): Do you define or register any ant tasks that other can use?
WARNING: Question with id="exec-ant-tasks" has not been answered!
Question (exec-classloader): Does your code create its own class loader(s)?
Answer: The implementation creates one class loader for each enabled module, plus a special "system" class loader.
Question (exec-reflection): Does your code use Java Reflection to execute other code?
Answer: Of course components present in modules are initially loaded using reflection. Otherwise there is little semantic use of it (transparent optimizations only). One exception: pending a solution to #29382, one portion of the Filesystems API is accessed via reflection when merging XML layers from modules.
Question (exec-privateaccess): Are you aware of any other parts of the system calling some of your methods by reflection?
Answer: No.
Question (exec-process): Do you execute an external process from your module? How do you ensure that the result is the same on different platforms? Do you parse output? Do you depend on result code?
WARNING: Question with id="exec-process" has not been answered!
Question (exec-introspection): Does your module use any kind of runtime type information (instanceof, work with java.lang.Class, etc.)?
WARNING: Question with id="exec-introspection" has not been answered!
Question (exec-threading): What threading models, if any, does your module adhere to? How the project behaves with respect to threading?
WARNING: Question with id="exec-threading" has not been answered!
Question (security-policy): Does your functionality require modifications to the standard policy file?
WARNING: Question with id="security-policy" has not been answered!
Question (security-grant): Does your code grant additional rights to some other code?
WARNING: Question with id="security-grant" has not been answered!

Format of files and protocols

Question (format-types): Which protocols and file formats (if any) does your module read or write on disk, or transmit or receive over the network? Do you generate an ant build script? Can it be edited and modified?
Answer:
• Module JAR files, as specified by the Modules API
• module-status-1_0.dtd - -//NetBeans//DTD Module Status 1.0//EN
• module-auto-deps-1_0.dtd - -//NetBeans//DTD Module Automatic Dependencies 1.0//EN
• filesystem-1_1.dtd - -//NetBeans//DTD Filesystem 1.1//EN
• a custom binary format for caching merged layer contents
• a custom binary format for caching module manifests
• a custom binary (serialized) format for caching folder lookup
• Java serialization for externalized ModuleInstalls (deprecated)
Question (format-dnd): Which protocols (if any) does your code understand during Drag & Drop?
Answer: N/A
Question (format-clipboard): Which data flavors (if any) does your code read from or insert to the clipboard (by access to clipboard on means calling methods on java.awt.datatransfer.Transferable?
Answer: N/A

Performance and Scalability

Question (perf-startup): Does your module run any code on startup?
Answer: Yes, of course - it finds and loads all modules.
Question (perf-exit): Does your module run any code on exit?
Answer: Delegates to exit handlers of enabled modules. Saves lookup cache.
Question (perf-scale): Which external criteria influence the performance of your program (size of file in editor, number of files in menu, in source directory, etc.) and how well your code scales?
Answer: Scalability of the standalone system seems reasonably good; even with a thousand modules (an order of magnitude higher than current reality) startup time and memory consumption in the module system itself are not large (7 seconds, 6 Mb).
Question (perf-limit): Are there any hard-coded or practical limits in the number or size of elements your code can handle?
Answer: No.
Question (perf-mem): How much memory does your component consume? Estimate with a relation to the number of windows, etc.
Answer: Not a lot. On modules which are well-formed (layers, JAR entries, complex interdependencies) but which do not run any code there is a modest incremental cost: 3.1msec and 4.5Kb per module at least measurement.
Question (perf-wakeup): Does any piece of your code wake up periodically and do something even when the system is otherwise idle (no user interaction)?
Answer: No.
Question (perf-progress): Does your module execute any long-running tasks?
Answer: Bulk module installation can be time-consuming. org.netbeans.core.ui.ModuleBean is a Swing-safe bean that serves as a nonblocking wrapper for all module system modifications and is used e.g. in the Modules node.
Question (perf-huge_dialogs): Does your module contain any dialogs or wizards with a large number of GUI controls such as combo boxes, lists, trees, or text areas?
Answer: No. Occasionally small dialogs only.
Question (perf-menus): Does your module use dynamically updated context menus, or context-sensitive actions with complicated and slow enablement logic?
Answer: No.
Question (perf-spi): How the performance of the plugged in code will be enforced?
WARNING: Question with id="perf-spi" has not been answered!