UD 5.0.0.4

For me, the release of the Universal Driver 5.0 for use with OpenInsight 9.4 was one of the best enhancements in recent times for people running OpenInsight.  It had new features that were sought after by many VARs and end users, and now it just got better with the release of 5.0.0.4.

Now available from www.revelation.com, this new version replaces all previous versions of the Universal Driver 5 and it is available at no cost to anyone who has purchased the Universal Driver 5.  Don’t feel left behind if you have an older version of the UD because we have an upgrade path for anyone running the UD and a new purchase option if this is your first time using the UD to enhance your deployed system.

So what’s all the excitement about?

The Universal Driver 5.0.0.4 resolves a potential memory leak /overrun issue when running in ‘debug output’ or logging mode.

It has also been enhanced to eliminate ‘hanging’ when shutting down the service.  This should allow sites to terminate and restart the Universal Driver without requiring a server reboot.

The new version addresses an issue that recently came to light where the record count and alpha values in the file header can get corrupted when several incomplete latent select / readnext are followed by a complete latent_Select/readnext.  This corruption has the side effect to cause the LK to expand rapidly.  A later complete select/readnext can repair the header and cause the UD to resize the table to normal size.  From the outside it looks like the service is bogging down for no reason.  In extreme cases the resize can crash the service and leave a huge file on the server.  It appears that this is a long-standing issue which may explain some of the issues witnessed by large sites.

The Volume Shadow Copy Service (VSS) feature now pauses the resizing thread and flushes all buffers.  Before 5.0.0.4 there was the possibility of a file being resized during the snapshot and of some new header record counts / alpha values not being written to disk.

Revelation have implemented the ability to pause the resizing thread.  The idea is to let the UD continue to queue files for resizing, pause the resize thread during the day so it does not slow the server, then release the thread in the evening to let the files catch up.

Revelation has also implemented a set-sizelock call inside the UD.  Prior to 5.0.0.4 the fix_lh program must OSBWrite the new sizelock into the header, which is not compatible with using a sharename to isolate the rev files.  The next UD Manager will support this feature.  The goal is to provide fine-grained control of resizing.  In theory you could pause the resizing thread, then view which files are going to resize, and control the order of the resizing by locking or releasing individual tables.

A “feature” where the UD will clear a sizelock of 1 has also been identified and removed in the new version.  Revelation has always stated that you presize a file, set sizelock to 1 and load it with data.  But, the UD clears the 1 on any select / readnext, making the feature less useful.  They have therefore removed this feature from the UD so that it is easier to pre-size tables and to leave them set to expand only.

The Universal Driver 5.0.0.4 software can be downloaded HERE.

UD5 vs UD4.7

I don’t look at the Knowledge Base on www.revelation.com enough and I’m often surprised at the information that resides there.  Today was no exception and just proves how long it was since I took a look and how important it is to check there fore time to time.

I have been talking to a few people about the UD5 and the upgrade process from earlier versions of the Universal Driver and the benefits of the upgrade and performance ‘always’ comes up.  I was therefore pleasantly surprised to see that Revelation have recently published some bench test results which compare the Universal Driver 4.7 to the Universal Driver 5.0, running both with and without encryption.

There are four tests which were completed using all three configurations (UD4.7 without encryption, UD 5.0 without encryption and UD 5.0 with encryption).  The results are nicely summarised in a graphic and there are also detailed screen shots of the results screens and the parameters used for each of the tests – totally transparent reporting.

I’ll leave you to review the knowledge base article, but it was good to see that the UD 5.0 is consistently faster than the UD4.7 when running without encryption. 

Running with encryption sees a decrease in performance of around 2 to 2.5 times the time taken in the UD 4.7 without encryption.Which is expected for obvious reasons.

Please see www.revsoft.co.uk for more details about the Universal Driver or contact your local Revelation representative or application author to discuss your upgrade to the UD5.

Securing data over the web within an O4W solution

There are three main areas of consideration when securing Linear Hash data over the web as part of an O4W based solution.  Those are the data that resides on the web server (or data server), the data whilst in transmission and then the data displayed within the browser.

Whilst these notes are written with an O4W solution in mind, it should be recognised that the provision for securing your data actually resides outside of the O4W system itself.

Liner Hash
It is worth mentioning that most hackers are familiar with relational data of the SQL type and understandably most people would prefer to work with data that they understand.  With MultiValue database driven solutions (like OpenInsight and O4W’s linear Hash) the data structure is usually unfamiliar to many hackers and therefore a lesser target in itself.

However, if you are running a high profile system, such as a bank with client financial details, a clinical system with patient data and the like, then you will still want to consider the security of your data and to do everything that you can to avoid those nightmare news headlines.

Data at rest
One of the first considerations that you will be faced with is where exactly should you house your data.  Locating it on the web server is usually fairly easy, it’s local, there are no networking (path) issues to consider and you can easily backup the whole system.  However, locating the data on the web server itself comprises a potential security risk in itself.

It is therefore recommended that a separate Data Server be used to locate the application’s data.  This server can then be hidden behind one or  more firewalls, so if your web server is compromised in anyway, the hacker will have more than one large hurdle to overcome.

Furthermore, with Revelation Software, you can utilise a Universal Driver between the web server and the data server and configure the system so that only access to the data is permitted through the Universal Driver.  Now that provides any external or internal hacker with yet another problem to overcome.

For those who need their data secure whilst it is residing on the data server, industry standard encryption tools can be used to encrypt the data to 128-bit encryption levels, for example.  A Google search for 128-bit encryption will provide you with plenty of information and solutions for  Advanced Encryption Standard (AES) and Data Encryption Standard (DES).

In addition, from OpenInsight 9.3, the toolset itself will support Data Encryption At Rest, making it even easier for Revelation developers to secure their data in single use, LAN, WAN and Web based solutions.

Data in transit
One of the weakest points before the data reaches the browser is during transmission.  Your precious data contained within one or more data packets is passed through countless networks, computers, hubs, etc. and at anytime these packets could be intercepted and interrogated.  If you have decrypted the data within the O4W application ready for it to be read by the user, then anyone could hijack it.

Fortunately, Hyper Text Transfer Protocol Secure (HTTPS) is your friend here.  HTTPS has been around for many years and it is well known as the industry standard for secure transmission of data over the web.  It has been supported since Internet Explorer 2 was around, so that really highlights that we are talking about a technology that is well tried and very well tested.

HTTPS is a secure version of the Hyper Text Transfer Protocol (HTTP) which allows for secure ecommerce transactions, such as online banking.  The technology effectively brings together HTTP and the SSL/TLS protocol to provide encrypted communication and secure identification of a network web server.

It is usually the System Administrator (or whoever has overall responsibility for the Web server) who configures the server to use HTTPS.  This is usually simply as case of acquiring a public key certificate for the web server from a trusted certificate authority.  This certificate must be signed by that authority for the web browser to accept and use the certificate.

It is also possible for the system to be configured for client authorisation.  Configuring a system in this way can limit access to a web server and thereby only permit authorised users.  This is achieved by the System Administrator creating a certificate for each user and this certificate is then loaded into the user’s browser during the session.  The certificate usually contains some identifying information (e.g. name and email address) and this is automatically verified by the server on each reconnect.

Web browsers such as Internet Explorer (IE) and Firefox display a padlock icon to indicate to the user that the website is secure.  In addition, the web address (URL) begins with https:\\, rather than http:\\.  When a user connects to a website via HTTPS, the site encrypts the session with a

digital certificate before any data is transmitted to the user’s browser.  O4W supports HTTPS, so transmitting your data using 128-bit encryption levels is super easy.  System Administrators can find a lot more information about the acquisition and use of public key certificates by running a Google search.

Data in the browser
So, we have our data encrypted at rest on our data server and we are using HTTPS to secure the data during transmission.  What about data in the browser?

Securing data in the browser is a tough one, mainly because the user needs to view the data in human readable format on the screen.  Or do they?

And herein lays the answer.

If the user needs to view the sensitive data in the browser, then you will have to revert to good old education.  Don’t leave your screen showing the data when you leave your desk, run your password protected screen saver before leaving your desk and, if you are near a customer facing position, make sure no unauthorised person can see your screen.

However, if the data does not need to be humanly read in the browser, why display it in the first place?  Leave the data on the data server and use some other form of key to interact with that sensitive data on the server, behind your firewall.  Many systems use this approach for user IDs.  The system maintains a cookie in the user’s browser and this contains some form of unique identifier – it could simply be a random number;  1234657684346846 (a session token of some kind).  It is this non-descript information that is then passed backwards and forwards over the web to identify the user.  When this key is received by your web application, you have code that takes the key, reads it and matches it to the user’s record.  Your program code can then use the sensitive login data to run the application on the web server as required.

To summarise
As an O4W web developer, you’ll want to concern yourself with protecting sensitive data within your specific application.  For example, credit card details; you will need to both protect a credit card field and audit who has access to the field and when it is accessed.  This is usually achieved by the developer including access controls to protect fields and the display of sensitive information.  As a conscientious developer, you may want to encrypt each record individually so that it can only be decrypted with a pin number that only the user knows and you’ll want to run a check to verify that user is legitimate.

This level of security in the application is good because it prevents an administrator from potentially stealing information from the system.  However, this does present new obstacles because the developer now has to code around the in ability to freely access all of the fields. How do you report on fields you can’t decrypt? What if the user forgets their pin, can the data be reset and re-encrypted or will the loss of data be irreversible?

The developer has to choose a balance in the application’s design between ease of access to data and the level of security. More security inevitably creates complexity so it’s important consider the risks and benefits of exposing the data.

The System Administrator should be responsible to ensure the underlying environment that the application runs on will be secure. If an attacker can gain administrator access in the operating system then the entire applications security protections could be rendered useless.  By protecting the environment (server, network, and client) at the operating system level the system administrator will be protecting the application.

Developers and administrators handle different levels of security that complement each other. The developer should be responsible for the application level access to the data while an administrator should be responsible for system level access.

But O4W helps . . .
Whilst you ‘could’ leave the question of data security up to someone else, effectively passing the problem on, there are ways that O4W can help.

From OpenInsight version 9.3, Revelation Software plan to introduce data encryption at rest.  This will be delivered through a new encryption service’ that system administrators can deploy to encrypt OpenInsight (OI) and O4W data on a field by field level.  The data will naturally be encrypted using industry standard encryption routines (DES, AES, TRIPLEDES, etc.).  This encryption service will use the Windows Communication Foundation (WCF) to communicate, thus ensuring that the data is also encrypted in transit.

In terms of protecting data in O4W ‘specifically’, O4W (in 9.2.1 and above) encrypts the user passwords when stored in OpenInsight, and never transmits any password information between the browser and the server.  When validating user information, O4W encrypts the user-entered password in the browser itself, and then returns only an impenetrable hash which the server then uses in its own calculations to determine if this is a valid logon.

In addition, the O4W programming paradigm is to ‘never’ send sensitive data to the browser if it can be helped.  Instead, temporary, unique records are created with the sensitive information, and stored on the server, and only the unique identifiers for these records are transmitted back and forth to the browser.

Of course, with the 9.3 release, the encryption service can apply encryption to any fields in the OpenInsight database, including those that are used by O4W, so by using https:// and the encryption service ‘any’ record and ‘any’ field can be protected all the way from “at rest” in the server,  through the engine server, through the web server, and to the browser.

My thanks to Bryan Shumsky (Revelation Software, Inc.) and Andrew McAuley (Sprezzatura) for their help and advice without which this posting would not have been completed.