Firebird Documentation IndexFirebird 3.0.6 Release NotesChanges to the Firebird API and ODS → Application Programming Interfaces
Firebird Home Firebird Home Prev: Changes to the Firebird API and ODSFirebird Documentation IndexUp: Changes to the Firebird API and ODSNext: Reserved Words and Changes

Application Programming Interfaces

Interfaces and the New Object-oriented API
Other New APIs
API Improvements

A new public API replaces the legacy one in new applications, especially object-oriented ones. The interface part can be found in the header file Interfaces.h in the directory /include/firebird beneath the installation root directory.

Note

POSIX installations have a symlink pointing to /usr/include/firebird/Interfaces.h

The new public API can be also used inside user-defined routines (UDR, q.v.) for callbacks inside the engine, allowing a UDR to select or modify something in the database, for example.

The main difference between the new API and the legacy one is that UDRs can query and modify data in the same connection or transaction context as the user query that called that UDR. It is now possible to write external triggers and procedures, not just external functions (UDFs).

Interfaces and the New Object-oriented API

Alex Peshkov

Firebird needed a modernised API for a number of compelling reasons.

  • High on the list was the limitation of the 16-bit integer pervading the legacy API, encompassing message size, SQL operator length, BLOB data portions, to name a few examples. While 16-bit was probably adequate when that old API came to life, in today's environments it is costly to work around.

    A trivial solution might be to add new functions that support 32-bit variables. The big downside is the obvious need to retain support for the old API by having pairs of functions with the same functionality but differing integer sizes. In fact, we did something like this to support 64-bit performance counters, for no better reason than being pressed to provide for it without having a more elegant way to implement it.

  • Another important reason, less obvious, derives from the era when Firebird's predecessor, InterBase, did not support SQL. It used a non-standard query language, GDML, to manage databases. Data requests were transported between client and server using messages whose formats were defined at request compilation time in BLR (binary language representation). In SQL, the operator does not contain the description of the message format so the decision was taken to surround each message with a short BLR sequence describing its format.

    The ISC API also has the XSQLDA layer over BLR. The trap with the XSQLDA solution is that it encapsulates both the location of the data and their format, making it possible to change location or format (or both) between fetch calls. Hence, the need for the BLR wrapping in every fetch call—notwithstanding, this potential capability to change the data format between fetches was broken in the network layer before Firebird existed.

    But to support the XSQLDA layer that rides on top of the message-based API that lower level API also has support sending format BLR at every turn.

    This system involving calls processing data through multiple layers is hard to extend and wastes performance; the SQLDA is not simple to use; the desire to fix it was strong.

  • Other reasons—numerous but perhaps less demanding—for changing the API included enhancing the status vector and optimizing dynamic library loading. Interfaces also make it so much easier and more comfortable to use the messages API.

The Non-COM Choice

The new interfaces are not compatible with COM, deliberately, and the reasons have to do with future performance enhancement.

At the centre of the Providers architecture in Firebird 3.0 is the y-valve, which is directed at dispatching API calls to the correct provider. Amongst the potential providers are older ones with potentially older interfaces. If we used COM, we would have to call the method IUnknown for each call (including record fetch), just to ensure that the provider really had some newer API method. Along with that comes the likelihood of future additions to the catalogue of API calls to optimize performance. A COM-based solution does not play well with that.

Firebird interfaces, unlike COM, support multiple versions. The interface version is determined by the total number of virtual functions it encompasses and is stored as a pointer-size integer at the beginning of the virtual functions table. This makes it possible for very fast checking of the interface version, since it requires no virtual call. That is to say, the pointer check has no overhead, unlike COM.

The Hierarchy of Interfaces

A detailed discussion of all the functions presented by all the interfaces is outside the scope of this overview. The general schematic looks like this:

The base of the structure is IVersioned. It is the interface that enables a version upgrade. A lot of interfaces not requiring additional lifetime control are based directly on IVersioned. IMaster is one example already mentioned. Others include a number of callback interfaces whose lifetimes must match the lifetimes of the objects from which they were to be used for callback.

Two interfaces deal with lifetime control: IDisposable and IRefCounted. The latter is especially active in the creation of other interfaces: IPlugin is reference counted, as are many other interfaces that are used by plug-ins. These include the interfaces that describe database attachment, transaction management and SQL statements.

Not everything needs the extra overhead of a reference-counted interface. For example, IMaster, the main interface that calls functions available to the rest of the API, has unlimited lifetime by definition. For others, the API is defined strictly by the lifetime of a parent interface; the IStatus interface is non-threaded. For interfaces with limited lifetimes it is of benefit to have a simple way to destroy them, that is, a dispose() function.

Each plug-in has one and only one main interface—IPlugin—which is responsible for basic plug-in functionality. In fact, a lot of plugins have only that interface, although that is not a requirement.

Finally, there is IProvider, a kind of main plug-in in the Firebird API. IProvider is derived from IPlugin and must be implemented by every provider. If you want to write your own provider you must implement IProvider. It is implemented also by the y-valve: it is the y-valve implementation that is returned to the user when the getDispatcher() function from the master interface is called.

IProvider contains functions enabling creation of an attachment to a database (attach and create) or to the Services Manager.

Interfaces Q & A

Q. We access new API using IMaster. But how to get access to IMaster itself?

  • A. This is done using just the one new API function fb_get_master_interface(). It is exported by the fbclient library. Also IMaster is passed as a parameter to each plug-in during its registration in the system.

Q. The non-use of COM-based interfaces was said to be to avoid working with IUnknown methods and that this is done due to performance issues. Instead you have to check the interface version. Why is that faster than using IUnknown?

  • A. As was already mentioned we do not need to execute virtual calls when checking the interface version. Taking into an account that each virtual call means a reset of the CPU cache, it is an important difference, especially for the very small calls like getting specific metadata properties from IMetadata.

Other New APIs

Other new APIs support various plug-ins by declaring the interfaces between the engine and the plug-in. Besides pluggable authentication and pluggable encryption, Firebird 3 supports external engines, bridges between the engine and the execution environments that can run UDRs: native code, Java and others. By and large they are intended for use by third-party solution providers, rather than for client application development.

For creating custom plug-ins and bridges, the relevant interface (API) needs to be implemented in the plug-in code.

API Improvements

The following improvements to the API should be noted.

Some SQL Size Limits Removed Using New API

Dmitry Yemanov

If and only if the new API is being used.-

  • The size of the body of a stored procedure or a trigger can exceed the traditional limit of 32 KB. The theoretical limit provided by the new API is 4GB. At the moment, as a security measure, a hard-coded limit of 10MB is imposed. The same limit of 10MB also applies to any user-defined DSQL query.

  • The total size of all input or output parameters for a stored procedure or a user-defined DSQL query is no longer limited to the traditional size of (64KB minus overhead). The theoretical limit provided by the new API is 4GB.

Legacy API

Improvements to the legacy API include.-

Scrollable Cursor Support

Dmitry Yemanov

In PSQL, a scrollable cursor can be operated on directly to navigate flexibly from the current row to any another row either forwards or backwards. API support is available to make scrollable cursors available to DSQL applications.

Scrollable Cursor Usage

The result set must be opened with the flag IStatement::CURSOR_TYPE_SCROLLABLE explicitly specified.

Fetch Methods

The following fetch methods of the IResultSet interface are available:

int fetchNext(IStatus* status, void* message);
// equivalent to FETCH NEXT FROM <cursor name>
              

Moves the cursor's current position to the next row and returns it. If the cursor is empty or already positioned at the last row, the condition NO_DATA is returned.

int fetchPrior(IStatus* status, void* message);
// equivalent to FETCH PRIOR FROM <cursor name>
              

Moves the cursor's current position to the prior row and returns it. If the cursor is empty or already positioned at the first row, the condition NO_DATA is returned.

int fetchFirst(IStatus* status, void* message);
// equivalent to FETCH FIRST FROM <cursor name>
              

Moves the cursor's current position to the first row and returns it. If the cursor is empty, the condition NO_DATA is returned.

int fetchLast(IStatus* status, void* message);
// equivalent to FETCH LAST FROM <cursor name>
              

Moves the cursor's current position to the last row and returns it. If the cursor is empty, the condition NO_DATA is returned.

int fetchAbsolute(IStatus* status, int position, void* message);
// equivalent to FETCH ABSOLUTE <position> FROM <cursor name>
              

Moves the cursor's current position to the specified <position> and returns the located row. If <position> is beyond the cursor's boundaries, the condition NO_DATA is returned.

int fetchRelative(IStatus* status, int offset, void* message);
// equivalent to FETCH RELATIVE <offset> FROM <cursor name>
              

Moves the cursor's current position backward or forward by the specified <offset> and returns the located row. If the calculated position is beyond the cursor's boundaries, the condition NO_DATA is returned.

Notes

  1. When a scrolling option is omitted, NO SCROLL is implied (i.e., the cursor is opened as forward-only). This means that only the fetchNext() API call can be used. Other fetch methods will return an error.

  2. Scrollable cursors are internally materialized as a temporary record set, thus consuming memory/disk resources, so this feature should be used only when really necessary.

SPB Support for New Statistics Feature in gbak Output

Vlad Khorsun

A new, much requested feature was added to gbak verbose output: optional run-time statistics. Read about it here. The feature is fully supported in the Services API with a new item in the SPB (Services Parameter Block),

#define isc_spb_bkp_stat 15
              

along with its synonym

#define isc_spb_res_stat isc_spb_bkp_stat
              

Usage

isc_spb_bkp_stat, <len>, <string>
isc_spb_res_stat, <len>, <string>
              

where <len> (2 bytes) indicates the length of the following string parameter, and <string> (1-4 bytes) is a string consisting of one character per statistics item.

The fbsvcmgr utility also supports the new SPB tags.

Better Error Reports for String Overflows

Alex Peshkov

Include expected and actual string length in the error message for string overflows (SQLCODE -802).

More Detail in Wrong Page Type Error Reports

Alex Peshkov

More details in the error message "wrong page type", i.e., identifying expected and encountered page types by name instead of numerical type.

New Item for isc_database_info() Call

Vlad Khorsun

An option was added to the API function isc_database_info() to return the number of free pages in a database. See CORE-1538.

Compression and Encryption Status Info

Alex Peshkov

Added in Firebird 3.0.3, see CORE-5601.

Compression details and encryption status of a connection (fb_info_conn_flags) have been added to the getInfo() API call. The data stored in the information block are of type integer in network format, accessible as isc_vax_integer.

Currently only 2 bits are meaningful:

  #define isc_dpb_addr_flag_conn_compressed   0x01
  #define isc_dpb_addr_flag_conn_encrypted    0x02
              

New Services Tag for Overriding LINGER

Alex Peshkov

The Services API now includes the tag isc_spb_prp_nolinger, for example (in one line):

 fbsvcmgr host:service_mgr user sysdba password xxx
        action_properties dbname employee prp_nolinger
            

For information regarding LINGER, see the write-up in the DDL chapter.

Enable Use of ***_auth_plugin_list Item from Application

Firebird 3.0.4: To specify the names of authentication plug-ins, an application must create a config string with "AuthClient = <plugin-list>", along with lines for other options. A specific DPB/SPB item actually exists, item isc_dpb_auth_plugin_list/isc_spb_auth_plugin_list that the client itself uses to communicate the plug-in list to the server. However if an application populated that same item, it was ignored by the client.

This improvement allows an application to use the item item isc_dpb_auth_plugin_list/isc_spb_auth_plugin_list in lieu of creating and passing a config string for it.

New Connection Formats for Local Superserver Clients on Windows

In previous Firebird versions, a serverless protocol known as Windows Local was available to local clients connecting to Superserver on a Windows platform, using the XNET subsystem. A typical connection string looked like this:

  c:\Program Files\Firebird_2_5\examples\empbuild\employee.fdb
          

Under the new unified server, that form of connection attempts to load an embedded server. It is no longer valid for a serverless client connection to Superserver. If you try, you will get a refusal message to the effect File is in use by another process. This is not a bug. Since Superserver clients share resources, another server (in this case, an embedded server) cannot attach a client to the same database that Superserver has any clients attached to.

However, all is not lost. The XNET subsystem can still do local client sessions for Superserver. You just need a more elaborate connection string now. You have a few choices:

  • this one is the former Windows local, using the XNET subsystem and shared memory for a (nominally) serverless connection:

      xnet://alias-or-path-to-database
                    

    So, for our connection to the employee database:

      xnet://c:\Program Files\Firebird_3_0\examples\empbuild\employee.fdb
                    

    or using an alias:

      xnet://employee
                    
  • Connection to host/port via TCP:

      inet://host:port/alias-or-path-to-database
                    
  • Connection to host/port via TCP, restricting the name search to IPv4:

      inet4://host:port/alias-or-path-to-database
                    

    This option was added in v.3.0.1. See also the configuration parameter IPv6V6Only.

  • Connection to host/port via Named Pipes (aka NetBEUI):

      wnet://host:port/alias-or-path-to-database
                    
  • Connection to localhost via TCP:

      inet://alias-or-path-to-database
                    
  • Connection to localhost via named pipes (aka NetBEUI):

      wnet://alias-or-path-to-database
                    

Perform Some Validation Services On-line

Vlad Khorsun

This feature was ported forward from Firebird 2.5.4.

Database validation enables low-level checks of the consistency of on-disk structures and even to fix some minor corruptions. The recommended procedure for any valuable database is for the DBA to validate a database periodically to ensure it is healthy.

Exclusive access to the database is required: any kind of concurrent access is forbidden during validation. Sometimes, blocking user access could be a major hold-up, especially if the database is large and complex.

Online validation is a new feature that allows some consistency checks to be performed without exclusive access.

What Online Validation Can Do
  • validate some (or all) user tables in a database.

    System tables are not validated.

  • validate some (or all) indices

Other ODS checks, such as Header\PIP\TIP\Generators pages, are not performed.

Protection During Online Validation

While a table (and\or its index) is undergoing validation, user attachments are allowed to read this table. Any attempt to change data (INSERT\UPDATE\DELETE) will wait until validation finishes or, depending on the lock timeout of the user transaction, will return a lock timeout error.

Any kind of garbage collection on the table or its indexes is disabled whilst it is undergoing validation:

  • background and cooperative garbage collection will just skip this table

  • sweep will be terminated with an error

When online validation starts to check a table, it acquires a couple of locks to prevent concurrent modifications of its data:

  • a relation lock in PR (protected read) mode

  • (NEW) a garbage collection lock in PW (protected write) mode

Both locks are acquired using a user-specified lock timeout. An error is reported for any lock request that fails and that table is skipped.

Once the locks are acquired, the table and its indexes are validated in the same way as a full validation does it. The locks are released when it completes and the whole procedure is repeated for the next table.

The New Services API action: isc_action_svc_validate

Online validation is implemented as a Firebird service and is accessed through the Services API. Thus, it cannot be run from the gfix utility.

The call involves the following elements:

Action:
	isc_action_svc_validate

Parameters:
	isc_spb_dbname :
		database file name, string, mandatory

	isc_spb_val_tab_incl, isc_spb_val_tab_excl,
	isc_spb_val_idx_incl, isc_spb_val_idx_excl :
		patterns for tables\indices names, string, optional

	isc_spb_val_lock_timeout :
		lock timeout, integer, optional

Output:
	text messages with progress of online validation process
             
Using isc_action_svc_validate Interactively

The fbsvcmgr utility has full support for the new service. The syntax is:

fbsvcmgr [host:]service_mgr [user <...>] [password <...>]
	action_validate dbname <filename>
	[val_tab_incl <pattern>]
	[val_tab_excl <pattern>]
	[val_idx_incl <pattern>]
	[val_idx_excl <pattern>]
	[val_lock_timeout <number>]
             

where

val_tab_incl pattern for table names to include in validation run
val_tab_excl pattern for table names to exclude from validation run
val_idx_incl pattern for index names to include in validation run, by default %, i.e. all indexes
val_idx_excl pattern for index names to exclude from validation run
val_lock_timeout lock timeout, used to acquire locks for table to validate, in seconds, default is 10 secs. 0 is no-wait, -1 is infinite wait

Usage Notes

  • Patterns are regular expressions, processed by the same rules as SIMILAR TO expressions.

  • All patterns are case-sensitive, regardless of database dialect.

  • If the pattern for tables is omitted then all user tables will be validated.

  • If the pattern for indexes is omitted then all indexes of the appointed tables will be validated.

  • System tables are not validated.

  • To specify a list of tables or indexes:

    1. Separate names with the pipe character '|'

    2. Do not add spaces: TAB1 | TAB2 is wrong

    3. Enclose the whole list in double quotes to avoid confusing the command interpreter

Examples

  1. Validate all tables in database 'c:\db.fdb' with names starting with 'A'. Indexes are not validated. Lock wait is not performed.

    fbsvcmgr.exe service_mgr user SYSDBA password masterkey
    		action_validate dbname c:\db.fdb
    		val_tab_incl A%
    		val_idx_excl %
    		val_lock_timeout 0
                     
  2. Validate tables TAB1 and TAB2 and all their indexes. Lock wait timeout is 10 seconds (the default):

    fbsvcmgr.exe service_mgr user SYSDBA password masterkey
    		action_validate dbname c:\db.fdb
    		val_tab_incl "TAB1|TAB2"
                     
  3. Default behavior of val_XXX options: validate all user tables and their indexes in database 'c:\db.fdb', lock wait is the default 10 seconds:

    fbsvcmgr.exe service_mgr user SYSDBA password masterkey
                   action_validate dbname c:\db.fdb
                     

Code Improvement

Alex Peshkov

(CORE-4387)   The functions IStatement::execute() and IAttachment::execute() now return an error pointer to the old transaction interface.

Prev: Changes to the Firebird API and ODSFirebird Documentation IndexUp: Changes to the Firebird API and ODSNext: Reserved Words and Changes
Firebird Documentation IndexFirebird 3.0.6 Release NotesChanges to the Firebird API and ODS → Application Programming Interfaces