v.8.7.7

• claviz-node-firebird upgraded to 2.0.5.
• Firebird processor now handles backpressure while streaming rows - when stream pauses it waits to resume before fetching more (works only when useClavizNodeFirebird option is set to true).

v.8.7.4

• If a delimited file has the same column name twice, numbers like (1), (2) will be added so each column maps correctly to obj.

v.8.7.0

• A new postLoadQuery option has been added to the MysqlDestination, allowing you to retrieve query results after a batch load. This is particularly useful when you need to load data and execute a query such as SELECT LAST_INSERT_ID() within single connection:

const destination = new MySqlDestination({
    connection,
    table: 'test',
    batchSize: 1,
    postLoadQuery: `SELECT LAST_INSERT_ID()`,
});
job.on('loadedBatch', async (destinationIndex, data, result) => {
    const lastInsertId = result[0][0]['LAST_INSERT_ID()'];
});

• Add useSourceColumns option to MysqlDestination - this option allows you to configure the MysqlDestination to only insert columns present in the source data.
• Improved HttpProcessor to include the response body in the error message:

job.onAny(async (event, data) => {
    if (event === 'processingError') {
        console.error(data.message); // e.g., 'Request failed with status code 500. Response: {"success":false}'
    }
});

v.8.5.0

You can now provide nested value of sourceField for authorizationRequest using dot notation:

const processor = new HttpProcessor({
    // ...
    authorizationRequest: {
        // ...
        sourceField: 'nested.auth_token',
    }
});

v.8.4.0

Added the useClavizNodeFirebird option to the FirebirdProcessor connection settings. If this option is set to true, data coming from Firebird will be decoded using iso-8859-13 encoding under the hood.

v.8.3.0

Added Firebird database processor.

v.8.2.0

Added encoding option for DelimitedProcessor and HttpProcessor (if dataFormat is delimited):

const processor = new DelimitedProcessor({
    path: './',
    files: [filePath],
    rowSeparator: '\n', 
    encoding: 'windows-1257', 
});

const processor = new HttpProcessor({
    dataFormat: 'delimited',
    rowSeparator: '\n', 
    connection: {
        method: 'GET',
        url: 'http://localhost:3000/windows-1257-encoded',
    },
    encoding: 'windows-1257', 
});

v.8.1.0

Added MySqlProcessor and MySqlDestination.

v.8.0.0

BREAKING.
msnodesqlv8 is now optional dependancy and needs to be installed separately - npm install msnodesqlv8.

driver option usage for msnodesqlv8 is changed:

BEFORE:

const connection: IMssqlDbConnection = {
  user: 'user',
  password: 'password',
  server: 'server',
  database: 'database',
  driver: 'msnodesqlv8'
};
const source = new MssqlProcessor({ connection, query: 'select * from orders' });

AFTER:

const nativeDriver: ITdsDriver = await import('mssql/msnodesqlV8');
const connection: IMssqlDbConnection = {
  user: 'user',
  password: 'password',
  server: 'server',
  database: 'database',
  driver: nativeDriver
};
const source = new MssqlProcessor({ connection, query: 'select * from orders' });

If you were using default driver and haven't specified tedious in connection options, nothing needs to be changed.

Thanks to @teleological for this change.

v.7.0.0

• BREAKING. Event listeners added by reporters are now prioritised. This means that these event listeners will now be executed before those added later.