NetBeans Architecture Answers for Command Line Parsing API module

General Information

Question (arch-what): What is this project good for?
Answer:

GetOpts like infrastructure to parse command line arguments with the cooperative participation of various modules.

Question (arch-overall): Describe the overall architecture.
Answer:

SendOptsAPI allows anyone to parse an array of strings - e.g. a command line. The infrastracture of the module then locates all providers registered using the SendOptsSPI and distributes the parts of the command line to their handlers. It is expected that the handlers do not know about each other and are in fact provided by different modules. The goal of the sendopts framework is to get the description of the handlers, apply the gained knowledge to the actual content of the command line and distribute the parts of the command line arguments to the designated handlers. Beyond this optimal state the error handling and help composition is also supported by this infrastructure.

Question (arch-usecases): Describe the main use cases of the new API. Who will use it under what circumstances? What kind of code would typically need to be written to use the module?
Answer:

Just Parse the Command Line

There needs to be a simple API for someone who has an array of strings and wants to parse them. One does not need to search for providers, just ask the infrastructure to do the parse and get the result.
The correct way to achieve this is to call CommandLine.getDefault().process(args).

Parse the Command Line with Own Options

Since version 2.20 one can define own classes with fields and annotate them with @Arg annotation. Those classes can then be passed into a factory method that creates new command line. One can then process the arguments as many times as needed via the process method. Example:

public final class MyOption implements Runnable {
  @Arg(longName="hello")
  public String name;

  public void run() {
    System.out.println("Hello " + name + "!");
  }
  
  public static void main(String... args) {
    CommandLine line = CommandLine.create(MyOption.class);
    line.process(args);
  }
}

If the above main class is called with parameters --hello World it will print out Hello World!.

Short and Long options with or without an argument

The standard getopts supports short form of options - e.g. a dash followed with one letter - or long form using two dashes followed with a word. Moreover the long form is optimized for abbrevations. If there are no conflicts between multiple options, then one can only use double dash followed with a prefix of a long option.
When using the declarative annotation style one can always specify @Arg(longName="text", shortName='t'). The longName attribute is required, but if there is supposed to be no long version of the argument, it can be set to empty string.
One can create an Option by calling any of its factory methods (like withoutArgument) and provider char for the one letter option and/or string for the long getopts option.

Options with or without an argument

There are three types of options. Those without an argument, those with a required one and those with optional one. Each one can be created by appropriate factory method in the Option class.
When using the declarative annotation style one needs to annotate a field of type boolean to create an option without an argument.

Support for --

The getopts compliant command line parsers support --. If these characters do appear on the command line, the rest of it is treated as extra arguments and not processed. The sendopts infrastructure supports this as well.

Multiple Independent CLI Handlers

The handlers for the options need not know about each other and still have to be able to process the command line successfully. Any module which wishes to provide its own options can register its OptionProcessor with @ServiceProvider annotation. Alternatively the module can use the @Arg annotation of its fields and it will be registered as well.

Extensible Options Set

Q: How shall one write an OptionProcessor that recognizes set of basic options, however contains one open slot? The processor wants other modules to provide recognizers for that slot and wants to communicate with them. For example, by default the processor recognizes option --channel <name_of_the_channel> which describes a source of data, and stores such data into a sink. There can be multiple sinks - discard the output, save it to file, show it on stdout, stream it to network. The processor itself can handle the copying of data, but does not itself know all the possible sink types.

To implement OptionProcessor like this one shall define an additional interface to communicate with the sink providers:

   package my.module;
   public interface SinkProvider {
     /** gets the option (even composite) that this sink needs on command line */
     public Option getOption();

     /** processes the options and creates a "sink" */
     public OutputStream createSink(Env env, Map<Option,String[]> values) throws CommandException;
   }
       

Other modules would then registered implementations of this interface in the META-INF/services/my.module.SinkProvider files. The OptionProcessor itself would just look all the implementations up, queried for the sinks, and then did the copying:

   class CopyingProvider extends OptionProvider {
     public Option getOption() {
        List<Option> l = ...;
        for (SinkProvider sp : Lookup.getDefault().lookupAll(SinkProvider.class)) {
          l.add(sp.getOption());
        }

        // we need only one provider to be present
        Option oneOfSinks = OptionGroups.oneOf(l.toArray(new Option[0]));

        // our channel option
        Option channel = ...;

        // the channel option needs to be present as well as a sink
        return OptionGroups.allOf(channel, oneOfSinks);
     }

     public void process(Env env, Map<Option,String[]> values) throws CommandException {
        OutputStream os = null;
        for (SinkProvider sp : Lookup.getDefault().lookupAll(SinkProvider.class)) {
          if (values.containsKey(sp.getOption())) {
            os = sp.createSink(env, values);
            break;
          }
        }
        if (os == null) {
          throw CommandException.exitCode(2);
        }

        // process the channel option and
        // handle the copying to the sink os
     }
   }
       

Another possible approach how to allow sharing of one option between multiple modules is to expose the option definition and its handling code as an interface to other modules, and then let the modules to write their own OptionProcessors. Necessary condition is that each of the processor is uniquely identified by some additional option, so when the shared option appears the infrastructure knows which processor to delegate to. This is demonstrated in the SharedOptionTest which basically does the following:

   /** the shared option, part of an interface of some module */
   public static final Option SHARED = ...;
   /** finds value(s) associated with the SHARED option and 
   * creates a JPanel based on them */
   public static JPanel getSharedPanel(Map<Option,String[]> args) { ... }
       

Then each module who wishes to reuse the SHARED option and the factory method that knows how to process their values for their own processing can just:

  public static final class ShowDialog extends OptionProcessor {
    private static final Option DIALOG = Option.withoutArgument('d', "dialog");

    protected Set<Option> getOptions() {
        // the following says that this processor should be invoked
        // everytime --dialog appears on command line, if the SHARED
        // option is there, then this processor wants to consume it 
        // as well...
        return Collections.singleton(Option.allOf(DIALOG, Option.anyOf(SHARED)));
    }

    protected void process(Env env, Map<Option, String[]> optionValues) throws CommandException {
        JPanel p = getSharedPanel(optionvalues);
        if (p == null) {
           // show empty dialog
        } else {
           // show some dialog containing the panel p
        }
    }
  }
       

The other modules are then free to write other processors refering to SHARED, for example one can write ShowFrame that does the same, just shows the panel in a frame, etc. The infrastructure guarantees that the exactly one provider which matches the command line options is called.

Printing Full Help Text

Althrough the handlers are provided by independent parties, it must be possible to generate resonable and consistent help description from all of them, so for the end user it appears as well formated and easily understandable. That is why every option can be associated with a short description providing info about what it is useful for using Option.shortDescription method. When using the @Arg style, there is an additional @Description annotation which can be used to declaratively associate a localized display name and short description with the option. To get such descriptions for all available options one can use CommandLine.getDefault().usage(java.io.PrintWriter).

Finding and Reporting when Options Are Not Correct

In case the command line cannot be processed a clean error for programmatic consumation and also one that can be shown to the end user of the command line must be given. This is handled by throwing CommandException with appropriate message description and exit code.

Processing Extra Command Line Arguments

There can be non-option arguments in the command line and they can freely mix with the option ones. For example the getopts would treat the following command line arguments as the same:

       --open X.java Y.java Z.txt
       X.java Y.java --open Z.txt
       

if the option open handles extra arguments. The sendopts infrastructure must distinquish between them and pass the non-option ones to the only one handler (active because it processed an option) that knowns how to parse them. It is an error if more than one or no handler expresses an interest in extra arguments and those are given. One can register such option by using the Option.additionalArgument factory method.
When using the declarative annotation style one may annotate a field of type String[] which then means this field should be filled with all additional arguments.

Handling Input and Output

Handler's shall not use the input and output streams directly for their execution, they should rely on the framework provided ones. This allows NetBeans based application to transfer the I/O from second started instance to the master one which is already running. From the client side there is the CommandLine.getDefault().parse methods taking additional arguments like input and output streams. This gets transfered to providers as an Env argument of their methods.

Returning Exit Code

When Handler's get execute (in the order defined by the order of options on the command line), each of them can either execute successfully, or fail. If a handler succeeds, next one is executed, if it fails, the execution is terminated and its return code is returned to the caller. The error can be notified by creating and throwing CommandException.exitCode(int errorCode).

Processing Only Extra Command Line Arguments

Sometimes it is desirable to process non-option arguments like file names without providing any option. Handlers can declare interest in such arguments. It is an error if such non-options are provided and no or more than one handler is around to handle them. One can create such option by using Option.defaultArguments factory method. With the declarative annotation style one can annotate a field of type String[] and specify that it is supposed to be implicit.

Only those processor need to process the options are created

For purposes of usage in NetBeans, it is needed to not-initialize those handlers that are not really needed to process certain command line. The infrastructure decides which of them are going to be needed and instantiates only those. This is supported only when using the declarative annotation style - information about these options is recorded in declarative way and the system can decide without loading the provider classes whether they are present on the command line or not.

Complex Option Relations

Certain CLI processors may need more than one option before they can process the input. For example it is necesary to tune the radio and then also tell what to do with the output. It is unconvenient to process that as one option with argument(s), that is why one can use the OptionGroups.allOf, OptionGroups.someOf, for example like:

class PP extends OptionProcessor {
    private static Option tune = Option.requiredArgument(Option.NO_SHORT_NAME, "tune");
    private static Option stream = Option.requiredArgument(Option.NO_SHORT_NAME, "stream");
    
    public Set<Option> getOptions() {
      return Collections.singleton(
        OptionGroups.allOf(tune, stream)
      );
    }
    
    public void process(Env env, Map>Option,String[]> values) throws CommandException {
        String freq = values.get(tune)[0];
        String output = values.get(stream)[0];

        // XXX handle what is needed here
    }
}

When the two options are registered and command line like --tune 91.9 --stream radio1.mp3 is being processed, the PP's process method is going to get called with values 91.9 and radio1.mp3.
This kind of grouping is not currently supported with the declarative annotation style registration.

Alternative Options

Sometimes there may different ways to specify the same option and just one of them or none of them can be provided at given time. For example is there is a way to tune the radio with direct frequency or with name of the station. Just one can be provided and one is needed. This can be specified by using OptionGroups.oneOf factory methods:

Option freq = Option.requiredArgument(Option.NO_SHORT_NAME, "tune");
Option station = Option.requiredArgument(Option.NO_SHORT_NAME, "station");
Option tune = OptionGroups.oneOf(freq, station);    

The option tune then signals that just one of the station or freq options can appear and that they both are replaceable.
Question (arch-time): What are the time estimates of the work?
Answer:

Delivered for NetBeans 5.0.

Question (arch-quality): How will the quality of your code be tested and how are future regressions going to be prevented?
Answer:

There will be tests to verify consistency with UNIX standard getopts behaviour. Everytime one reports an error or finds difference between functionality of this API and functionality of getopts new tests will be created.

Question (arch-where): Where one can find sources for your module?
Answer:

The sources for the module are in the Apache Git repositories or in the GitHub repositories.

Project and platform dependencies

Question (dep-nb): What other NetBeans projects and modules does this one depend on?
Answer:
• org.openide.util - The sendopts module uses Lookup to get list of all registered OptionProcessors. This dependency is critical. If you want to use this API outside of NetBeans, also include the org-openide-util.jar module.
• org.netbeans.bootstrap - The sendopts module implements and registers the CLIHandler so it can be called back by the core/bootstrap whenever some one needs to parse the command line. This dependency is conditional, if the core/bootstrap module is not present, the whole library can still be used.
Question (dep-non-nb): What other projects outside NetBeans does this one depend on?
Answer:

XXX no answer for dep-non-nb

Question (dep-platform): On which platforms does your module run? Does it run in the same way on each?
Answer:

XXX no answer for dep-platform

Question (dep-jre): Which version of JRE do you need (1.2, 1.3, 1.4, etc.)?
Answer:

XXX no answer for dep-jre

Question (dep-jrejdk): Do you require the JDK or is the JRE enough?
Answer:

XXX no answer for dep-jrejdk

Deployment

Question (deploy-jar): Do you deploy just module JAR file(s) or other files as well?
Answer:

XXX no answer for deploy-jar

Question (deploy-nbm): Can you deploy an NBM via the Update Center?
Answer:

XXX no answer for deploy-nbm

Question (deploy-shared): Do you need to be installed in the shared location only, or in the user directory only, or can your module be installed anywhere?
Answer:

XXX no answer for deploy-shared

Question (deploy-packages): Are packages of your module made inaccessible by not declaring them public?
Answer:

XXX no answer for deploy-packages

Question (deploy-dependencies): What do other modules need to do to declare a dependency on this one, in addition to or instead of the normal module dependency declaration (e.g. tokens to require)?
Answer:

Nothing.

Compatibility with environment

Question (compat-i18n): Is your module correctly internationalized?
Answer:

XXX no answer for compat-i18n

Question (compat-standards): Does the module implement or define any standards? Is the implementation exact or does it deviate somehow?
Answer:

XXX no answer for compat-standards

Question (compat-version): Can your module coexist with earlier and future versions of itself? Can you correctly read all old settings? Will future versions be able to read your current settings? Can you read or politely ignore settings stored by a future version?
Answer:

XXX no answer for compat-version

Question (compat-deprecation): How the introduction of your project influences functionality provided by previous version of the product?
Answer:

This module replaces and stabilizes the previous CLIHandler offered by core/bootstrap module. Modules are now adviced to depend just on the API exported by this module.

Access to resources

Question (resources-file): Does your module use java.io.File directly?
Answer:

XXX no answer for resources-file

Question (resources-layer): Does your module provide own layer? Does it create any files or folders in it? What it is trying to communicate by that and with which components?
Answer:

XXX no answer for resources-layer

Question (resources-read): Does your module read any resources from layers? For what purpose?
Answer:

XXX no answer for resources-read

Question (resources-mask): Does your module mask/hide/override any resources provided by other modules in their layers?
Answer:

XXX no answer for resources-mask

Question (resources-preferences): Does your module uses preferences via Preferences API? Does your module use NbPreferences or or regular JDK Preferences ? Does it read, write or both ? Does it share preferences with other modules ? If so, then why ?
WARNING: Question with id="resources-preferences" has not been answered!

Lookup of components

Question (lookup-lookup): Does your module use org.openide.util.Lookup or any similar technology to find any components to communicate with? Which ones?
Answer:

OptionProcessor - The sendopts module uses Lookup to get list of all registered OptionProcessors.

Question (lookup-register): Do you register anything into lookup for other code to find?
Answer:

XXX no answer for lookup-register

Question (lookup-remove): Do you remove entries of other modules from lookup?
Answer:

XXX no answer for lookup-remove

Execution Environment

Question (exec-property): Is execution of your code influenced by any environment or Java system (System.getProperty) property? On a similar note, is there something interesting that you pass to java.util.logging.Logger? Or do you observe what others log?
Answer:

XXX no answer for exec-property

Question (exec-component): Is execution of your code influenced by any (string) property of any of your components?
Answer:

XXX no answer for exec-component

Question (exec-ant-tasks): Do you define or register any ant tasks that other can use?
Answer:

XXX no answer for exec-ant-tasks

Question (exec-classloader): Does your code create its own class loader(s)?
Answer:

XXX no answer for exec-classloader

Question (exec-reflection): Does your code use Java Reflection to execute other code?
Answer:

XXX no answer for exec-reflection

Question (exec-privateaccess): Are you aware of any other parts of the system calling some of your methods by reflection?
Answer:

XXX no answer for exec-privateaccess

Question (exec-process): Do you execute an external process from your module? How do you ensure that the result is the same on different platforms? Do you parse output? Do you depend on result code?
Answer:

XXX no answer for exec-process

Question (exec-introspection): Does your module use any kind of runtime type information (instanceof, work with java.lang.Class, etc.)?
Answer:

XXX no answer for exec-introspection

Question (exec-threading): What threading models, if any, does your module adhere to? How the project behaves with respect to threading?
Answer:

XXX no answer for exec-threading

Question (security-policy): Does your functionality require modifications to the standard policy file?
Answer:

XXX no answer for security-policy

Question (security-grant): Does your code grant additional rights to some other code?
Answer:

XXX no answer for security-grant

Format of files and protocols

Question (format-types): Which protocols and file formats (if any) does your module read or write on disk, or transmit or receive over the network? Do you generate an ant build script? Can it be edited and modified?
Answer:

XXX no answer for format-types

Question (format-dnd): Which protocols (if any) does your code understand during Drag & Drop?
Answer:

XXX no answer for format-dnd

Question (format-clipboard): Which data flavors (if any) does your code read from or insert to the clipboard (by access to clipboard on means calling methods on java.awt.datatransfer.Transferable?
Answer:

XXX no answer for format-clipboard

Performance and Scalability

Question (perf-startup): Does your module run any code on startup?
Answer:

XXX no answer for perf-startup

Question (perf-exit): Does your module run any code on exit?
Answer:

XXX no answer for perf-exit

Question (perf-scale): Which external criteria influence the performance of your program (size of file in editor, number of files in menu, in source directory, etc.) and how well your code scales?
Answer:

XXX no answer for perf-scale

Question (perf-limit): Are there any hard-coded or practical limits in the number or size of elements your code can handle?
Answer:

XXX no answer for perf-limit

Question (perf-mem): How much memory does your component consume? Estimate with a relation to the number of windows, etc.
Answer:

XXX no answer for perf-mem

Question (perf-wakeup): Does any piece of your code wake up periodically and do something even when the system is otherwise idle (no user interaction)?
Answer:

XXX no answer for perf-wakeup

Question (perf-progress): Does your module execute any long-running tasks?
Answer:

XXX no answer for perf-progress

Question (perf-huge_dialogs): Does your module contain any dialogs or wizards with a large number of GUI controls such as combo boxes, lists, trees, or text areas?
Answer:

XXX no answer for perf-huge_dialogs

Question (perf-menus): Does your module use dynamically updated context menus, or context-sensitive actions with complicated and slow enablement logic?
Answer:

XXX no answer for perf-menus

Question (perf-spi): How the performance of the plugged in code will be enforced?
Answer:

XXX no answer for perf-spi