NetBeans Architecture Answers for Database Explorer module

General Information

Question (arch-what): What is this project good for?
Answer:

This project provides access to objects defined in the Database Explorer. Documentation is available in the Javadoc.

Question (arch-overall): Describe the overall architecture.
Answer:

The Database module provides the DatabaseExplorerAPI which allows access to the database connections and drivers defined in the Database Explorer. It allows a client to retrieve the connection list and their properties, to create new connections and remove existing ones. The Database Explorer also manages a list of JDBC drivers used to connect to databases. The API provides access to these drivers and allows to create new and remove existing drivers.

The DatabaseExplorerLayerAPI allows for the declarative registration of database connections and JDBC drivers in the module layer. Database runtimes (which are representations of an instance of a database server) can also be registered in the layer.

Loaders-text-dbschema-Actions allows extending .dbschema files with own actions by registering them in the Loaders/text/x-dbschema/Actions folder. Note that this folder is actually provided by the org-netbeans-modules-dbschema.jar module.

Loaders-text-sql-Actions allows extending .sql files with own actions by registering them in the Loaders/text/x-sql/Actions folder. Note that this folder is actually provided by the org-netbeans-modules-db-core.jar module.

This module also provides a SQLSupportAPI which provides utilities for working with SQL such as quoting identifiers.

Question (arch-usecases): Describe the main use cases of the new API. Who will use it under what circumstances? What kind of code would typically need to be written to use the module?
Answer:

Registering JDBC ../db.drivers

An external module can register JDBC drivers. A typical example is a module which provides integration with a database server. In this case the module contains the JDBC driver for that database server and uses the Database Explorer API to add it do the Database Explorer.

Another client of this API could be a module providing integration with a J2EE application server. Sometimes a J2EE application server bundles a database server for improving the out-of-the-box experience. When the server is registered in the IDE the JDBC drivers for the bundled database server are added to the Database Explorer.

The drivers are registered by making calls on JDBCDriverManager or by registering an XML file which describes the driver in the module layer. The XML file is described by the JDBC Driver DTD. An example of a registration file describing the JDBC driver for PostgreSQL follows:

    <?xml version='1.0'?>
    <!DOCTYPE driver PUBLIC '-//NetBeans//DTD JDBC Driver 1.0//EN' 'http://www.netbeans.org/dtds/jdbc-driver-1_0.dtd'>
    <driver>
      <name value='postgresql-7'/>
      <display-name value='PostgreSQL (v7.0 and later)'/>
      <class value='org.postgresql.Driver'/>
      <urls>
        <url value='file:/folder1/folder2/drivers/pg74.1jdbc3.jar'/>
      </urls>
    </driver>
   

This file should be registered in the Databases/JDBCDrivers folder of the module layer. To addres a bundled JAR inside the IDE the nbinst protocol can be used in the URLs: nbinst:/modules/ext/bundled-driver.jar.

Get the underlying JDBC Driver instance for a JDBCDriver

You can use the JDBCDriver.getDriver() method to obtain a reference to the underlying JDBC Driver instance. This is useful if you want to use the registered drivers but create your own JDBC connections independent of the Database Explorer.

Retrieving the list of JDBC ../db.drivers

When creating a new connection the JDBC driver which it should use can be specified. A list of all the registered JDBC drivers can be retrieved using JDBCDriverManager.getDrivers().

Registering database runtimes

An external module can register new database runtimes. A database runtime is an abstraction of a database server instance (usually bundled with the IDE, an integration module or with a J2EE server). It allows a database server instance to be started and stopped when a connection to this instance is made in the IDE. Database runtimes are represented by the DatabaseRuntime SPI interface and are registered in the Databases/Runtimes of the module layer.

Creating database connections

A module can create new database connections (for example to a bundled database). New connections can be added by calling DatabaseConnection.create() to create a new DatabaseConnection instance and then ConnectionManager.addConnection() to add the connection to the Database Explorer.

New connections can also be added by registering them in the module layer. The format of the registration file is described by the Database Connection DTD. An example of a registration file describing a connection to a PostgreSQL database follows:

    <?xml version='1.0'?>
    <!DOCTYPE connection PUBLIC '-//NetBeans//DTD Database Connection 1.1//EN' 'http://www.netbeans.org/dtds/connection-1_1.dtd'>
    <connection>
      <driver-class value='org.postgresql.Driver'/>
      <driver-name value='postgres-7'/>
      <database-url value='jdbc:postgresql:test'/>
      <schema value='public'/>
      <user value='test'/>
      <password value='cGFzc3dvcmQ='/>
    </connection>
   

This file should be registered in the Databases/Connections folder of the module layer.

The password element is optional, but if it is included, its value must be the Base64 encoding of the UTF-8 representation of the password. Note that the UTF-8 representation of passwords composed entirely of ASCII characters is the same as their ASCII representation, so for such passwords all that needs to be done is to convert them to Base64.

Base64 encoding serves as a simple scrambling to prevent accidental revelation of the password. It is not indended to offer any real security. You can protect the password by assigning appropriate file protections to the connection XML file.

Retrieving and displaying the list of database connections

Sometimes the list of connections needs to be displayed somewhere else in the IDE than the Runtime tab. A typical example is the SQL Editor, which allows the user to select the database connection which the SQL statement will be executed against in a combo box in the editor toolbar. The list of connections can be obtained by calling ConnectionManager.getConnections(), which returns an array of DatabaseConnection instances.

The client usually needs to show the display name of the connection. The display name can be retrieved using the DatabaseConnection.getDisplayName() method.

Retrieving the properties of database connections

Sometimes a client needs to retrieve the connection properties, such as the driver class. An example could be a module for a J2EE server creating a connection pool. The properties can be retrieved using the getDriverClass(), getDatabaseURL(), getSchema(), getUser() and getPassword() methods of the DatabaseConnection class.

Showing the New Database Connection dialog

Usually when displaying a list of connections (usually in a combo box), the last item is "New Connection", which displays the standard New Database Connection dialog of the Database Explorer. This can be achieved by calling one of the ConnectionManager.showAddConnectionDialog() methods.

Remove a database connection

A user of this API may want to remove a connection from the list of connections registered by the Database Explorer. This is done using ConnectionManager.removeConnection()

Connecting to a database

A component which provides database functionality (such as the SQL Editor) will need to connect to a database. This can be achieved using the DatabaseConnection.showConnectionDialog() method and the java.sql.Connection instance can be retrieved using the getJDBCConnection() method.

If you want to connect to the database without showing a dialog or any kind of UI, you can use the DatabaseConnection.connect() method.

Test a database connection for validity

You may want to test to make sure the underlying physical JDBC connection obtained from a DatabaseConnection is either valid or null. This is done using the DatabaseConnection.getJDBCConnection(boolean test) method, which validates the underlying connection before returning it. If the connection is invalid, it marks the DatabaseConnection as disconnected and returns null.

Displaying the database connections in the UI

A component which provides database functionality (such as the SQL Editor or a module providing support for data sources) will need to let the user select the a database connection, usually through a combo box. This can be achieved using the DatabaseExplorerUIs.connect() method. The JComboBox passed to the method will be filled with the list of connections as returned by ConnectionManager.getConnections(), followed by a separator and a New Database Connection item which will display the dialog for adding a new database connection when selected.

Drag and drop support for database objects

A component might need to allow database tables from the Database Explorer to be dragged to a visual editor. An API is provided in DatabaseMetaDataTransfer containing DataFlavors for database objects and nested classes encapsulating those database objects during a drag and drop transfer.

Get support for working with SQL identifiers

A component might need support for working with SQL identifiers. In particular, it's important to know when to quote a SQL identifier. The SQLIdentifiers.Quoter class is provided for this.

Question (arch-time): What are the time estimates of the work?
Answer:

Most code is already written. About 5 man-days are still needed for the registration of database runtimes and cleanups and about two weeks shall be spent writing tests. The milestone by which this API should be stable is promo-G.

Question (arch-quality): How will the quality of your code be tested and how are future regressions going to be prevented?
Answer:

All Javadoc-specified functionality should be covered by unit tests.

Question (arch-where): Where one can find sources for your module?
Answer:

The sources for the module are in the Apache Git repositories or in the GitHub repositories.

Project and platform dependencies

Question (dep-nb): What other NetBeans projects and modules does this one depend on?
Answer:

Lookup - JDBC drivers and database runtimes are registered in the default lookup.

org.openide.actions - Needed in the Database Explorer UI.

org.openide.filesystems - Neded for writing JDBC driver registration files.

org.openide.util - Multiple usages (bundles, request processor).

org.openide.modules - For installing a ModuleInstall.close() method which disconnects the connected connections upon IDE shutdown.

org.openide.nodes - Needed in the Database Explorer UI.

org.openide.dialogs - Needed in the Database Explorer UI.

org.openide.windows - Needed in the Database Explorer UI (the Execute Command top component).

org.openide.loaders - Neded for writing JDBC driver registration files.

org.netbeans.api.progress - Needed in the Database Explorer UI.

Question (dep-non-nb): What other projects outside NetBeans does this one depend on?
Answer:

None.

Question (dep-platform): On which platforms does your module run? Does it run in the same way on each?
Answer:

No known platform dependencies.

Question (dep-jre): Which version of JRE do you need (1.2, 1.3, 1.4, etc.)?
Answer:

The module runs on JRE 1.4 and higher.

Question (dep-jrejdk): Do you require the JDK or is the JRE enough?
Answer:

JRE is enough.

Deployment

Question (deploy-jar): Do you deploy just module JAR file(s) or other files as well?
Answer:

There are two JAR files: the module JAR file modules/org-netbeans-modules-db.jar and a library used by the module located at modules/ext/ddl.jar.

Question (deploy-nbm): Can you deploy an NBM via the Update Center?
Answer:

Yes.

Question (deploy-shared): Do you need to be installed in the shared location only, or in the user directory only, or can your module be installed anywhere?
Answer:

Anywhere.

Question (deploy-packages): Are packages of your module made inaccessible by not declaring them public?
Answer:

Only API and SPI packages are exported.

Question (deploy-dependencies): What do other modules need to do to declare a dependency on this one, in addition to or instead of the normal module dependency declaration (e.g. tokens to require)?
Answer:

Nothing.

Compatibility with environment

Question (compat-i18n): Is your module correctly internationalized?
Answer:

Yes.

Question (compat-standards): Does the module implement or define any standards? Is the implementation exact or does it deviate somehow?
Answer:

No.

Question (compat-version): Can your module coexist with earlier and future versions of itself? Can you correctly read all old settings? Will future versions be able to read your current settings? Can you read or politely ignore settings stored by a future version?
Answer:

The module can read old settings, but doesn't store version numbers.

Question (compat-deprecation): How the introduction of your project influences functionality provided by previous version of the product?
Answer:

XXX no answer for compat-deprecation

Access to resources

Question (resources-file): Does your module use java.io.File directly?
Answer:

No.

Question (resources-layer): Does your module provide own layer? Does it create any files or folders in it? What it is trying to communicate by that and with which components?
Answer:

Yes. The module contains a standard layer which describes menu items, actions and services. The folders Databases/Connections, Databases/JDBCDrivers and Databases/Runtimes, where database connections, JDBC drivers and database runtimes are registered, are created.

Question (resources-read): Does your module read any resources from layers? For what purpose?
Answer:

Only resources registered by this module are read.

Question (resources-mask): Does your module mask/hide/override any resources provided by other modules in their layers?
Answer:

No.

Question (resources-preferences): Does your module uses preferences via Preferences API? Does your module use NbPreferences or or regular JDK Preferences ? Does it read, write or both ? Does it share preferences with other modules ? If so, then why ?
Answer:

XXX no answer for resources-preferences

Lookup of components

Question (lookup-lookup): Does your module use org.openide.util.Lookup or any similar technology to find any components to communicate with? Which ones?
Answer:

FolderLookup is used to locate database runtimes. The Databases/Runtimes folder in the default filesystem is searched for implementations of the DatabaseRuntime interface.

Question (lookup-register): Do you register anything into lookup for other code to find?
Answer:

No.

Question (lookup-remove): Do you remove entries of other modules from lookup?
Answer:

No.

Execution Environment

Question (exec-property): Is execution of your code influenced by any environment or Java system (System.getProperty) property? On a similar note, is there something interesting that you pass to java.util.logging.Logger? Or do you observe what others log?
Answer:

No.

Question (exec-component): Is execution of your code influenced by any (string) property of any of your components?
Answer:

No.

Question (exec-ant-tasks): Do you define or register any ant tasks that other can use?
Answer:

No.

Question (exec-classloader): Does your code create its own class loader(s)?
Answer:

A class loader is created for loading JDBC drivers.

Question (exec-reflection): Does your code use Java Reflection to execute other code?
Answer:

No.

Question (exec-privateaccess): Are you aware of any other parts of the system calling some of your methods by reflection?
Answer:

No.

Question (exec-process): Do you execute an external process from your module? How do you ensure that the result is the same on different platforms? Do you parse output? Do you depend on result code?
Answer:

No. Database runtime implementations could execute such actions though (especially executing external scripts which start/stop a database server).

Question (exec-introspection): Does your module use any kind of runtime type information (instanceof, work with java.lang.Class, etc.)?
Answer:

No.

Question (exec-threading): What threading models, if any, does your module adhere to? How the project behaves with respect to threading?
Answer:

The API is thread-safe. Most methods are not required to run on the event queue (although the implementation of methods which display an UI switches to the event thread where necessary). The methods which are required to run on the event queue say so in the Javadoc and exceptions are thrown if this condition does not hold. Events are fired synchronously.

Question (security-policy): Does your functionality require modifications to the standard policy file?
Answer:

No.

Question (security-grant): Does your code grant additional rights to some other code?
Answer:

No.

Format of files and protocols

Question (format-types): Which protocols and file formats (if any) does your module read or write on disk, or transmit or receive over the network? Do you generate an ant build script? Can it be edited and modified?
Answer:

The database connections, JDBC drivers and database runtimes are saved as XML files whose DTDs are published.

Question (format-dnd): Which protocols (if any) does your code understand during Drag & Drop?
Answer:

None.

Question (format-clipboard): Which data flavors (if any) does your code read from or insert to the clipboard (by access to clipboard on means calling methods on java.awt.datatransfer.Transferable?
Answer:

Implementations of PasteType representing database tables and indexes are placed on the clipboard.

Performance and Scalability

Question (perf-startup): Does your module run any code on startup?
Answer:

No.

Question (perf-exit): Does your module run any code on exit?
Answer:

Yes. The connected database connections are disconnected.

Question (perf-scale): Which external criteria influence the performance of your program (size of file in editor, number of files in menu, in source directory, etc.) and how well your code scales?
Answer:

Performance scales linearly with the number of connections and ../db.drivers, but usually the numbers are small.

Question (perf-limit): Are there any hard-coded or practical limits in the number or size of elements your code can handle?
Answer:

None known.

Question (perf-mem): How much memory does your component consume? Estimate with a relation to the number of windows, etc.
Answer:

A small amount of memory is necessary for the database connections, drivers and runtimes. The amount of memory needed for displaying the structure of a database is directly proportional to the number of elements in the database.

Question (perf-wakeup): Does any piece of your code wake up periodically and do something even when the system is otherwise idle (no user interaction)?
Answer:

No.

Question (perf-progress): Does your module execute any long-running tasks?
Answer:

Yes, any database call can be a long-running task. However, these calls are not executed in the event thread.

Question (perf-huge_dialogs): Does your module contain any dialogs or wizards with a large number of GUI controls such as combo boxes, lists, trees, or text areas?
Answer:

No.

Question (perf-menus): Does your module use dynamically updated context menus, or context-sensitive actions with complicated and slow enablement logic?
Answer:

No.

Question (perf-spi): How the performance of the plugged in code will be enforced?
Answer:

Some methods of the database runtimes may take a long time (especially the start method). These methods are never called in the event thread and a progress dialog is displayed.