Things just got a little quicker.


benchtestresultsThis will be my last posting of the day with regards to the new Universal Driver (UD) releases, I promise, no more interruptions to your day from me😉

Everyone keeps talking to me about performance these days and I wanted to share with you that I have heard a whisper that a test using the UD 4.7.2 (and I believe the UD 5.0.0.4 because that has the same enhancements) is performing better than previous versions during a mammoth testing project.

Testing is on going and loads are increasing, but I’m hearing about 17,000 reads being completed in around 3 seconds.  Obviously different systems will perform differently, but if performance is important to you, then maybe now is the time to take a look at what OpenInsight 9.4 and the Universal Driver 4.7.2 or the 5.0.0.4 can do for you.

UD 5.0.0.4


benchtestresultsFor me, the release of the Universal Driver 5.0 for use with OpenInsight 9.4 was one of the best enhancements in recent times for people running OpenInsight.  It had new features that were sought after by many VARs and end users, and now it just got better with the release of 5.0.0.4.

Now available from www.revelation.com, this new version replaces all previous versions of the Universal Driver 5 and it is available at no cost to anyone who has purchased the Universal Driver 5.  Don’t feel left behind if you have an older version of the UD because we have an upgrade path for anyone running the UD and a new purchase option if this is your first time using the UD to enhance your deployed system.

So what’s all the excitement about?

The Universal Driver 5.0.0.4 resolves a potential memory leak /overrun issue when running in ‘debug output’ or logging mode.

It has also been enhanced to eliminate ‘hanging’ when shutting down the service.  This should allow sites to terminate and restart the Universal Driver without requiring a server reboot.

The new version addresses an issue that recently came to light where the record count and alpha values in the file header can get corrupted when several incomplete latent select / readnext are followed by a complete latent_Select/readnext.  This corruption has the side effect to cause the LK to expand rapidly.  A later complete select/readnext can repair the header and cause the UD to resize the table to normal size.  From the outside it looks like the service is bogging down for no reason.  In extreme cases the resize can crash the service and leave a huge file on the server.  It appears that this is a long-standing issue which may explain some of the issues witnessed by large sites.

The Volume Shadow Copy Service (VSS) feature now pauses the resizing thread and flushes all buffers.  Before 5.0.0.4 there was the possibility of a file being resized during the snapshot and of some new header record counts / alpha values not being written to disk.

Revelation have implemented the ability to pause the resizing thread.  The idea is to let the UD continue to queue files for resizing, pause the resize thread during the day so it does not slow the server, then release the thread in the evening to let the files catch up.

Revelation has also implemented a set-sizelock call inside the UD.  Prior to 5.0.0.4 the fix_lh program must OSBWrite the new sizelock into the header, which is not compatible with using a sharename to isolate the rev files.  The next UD Manager will support this feature.  The goal is to provide fine-grained control of resizing.  In theory you could pause the resizing thread, then view which files are going to resize, and control the order of the resizing by locking or releasing individual tables.

A “feature” where the UD will clear a sizelock of 1 has also been identified and removed in the new version.  Revelation has always stated that you presize a file, set sizelock to 1 and load it with data.  But, the UD clears the 1 on any select / readnext, making the feature less useful.  They have therefore removed this feature from the UD so that it is easier to pre-size tables and to leave them set to expand only.

The Universal Driver 5.0.0.4 software can be downloaded HERE.

UD 4.7.2


benchtestresultsFollowing several weeks of intensive research, working with one of our key users here in the UK and reviewing many thousands of lines of code, Revelation have now released the Universal Driver 4.7.2.

So why should a new point release be of such interest?

The UD 4.7.2 is a minor upgrade from UD 4.7 but it resolves a significant issue with corrupted table header counts and sizes, a potential memory leak/overrun issue when running in ‘debug output’ or logging mode, and eliminates “hanging” when shutting down the service.  This should allow sites to terminate and restart the UD without requiring a server reboot.

The UD 4.7.2 has been released for both the “NT” and “NU” versions.  This replaces all previous versions of the Universal Driver 4.7 and 4.7.1 and it is available at no cost to anyone who already has the Universal Driver 4.x.

The Universal Driver 4.7.2 fixes an issue where the record count and alpha values in the file header can get corrupted when several incomplete latent select / readnext process are followed by a complete latent_Select/readnext process.  This corruption has the side effect to cause the LK to expand rapidly.  A later complete select/readnext can repair the header and cause the UD to resize the table to normal size.  From the outside it looks like the service is bogging down for no reason.  In extreme cases the resize can crash the service and leave a huge file on the server.  It appears that this is a long standing feature which may explain some of the issues witnessed by large sites.

The Universal Driver 4.7.2 also now enables system administrators to more gracefully close down the Universal Driver in the event of a system crash. In such a situation the UD appears to have simply hung and most people are finding it necessary to aggressively terminate the Universal Driver before restarting the system.  This new version addresses this issue.

Please contact your system author (VAR) or your usual Revelation Representative to obtain your UD 4.7.2 upgrade or to discuss upgrade options if you are running an older Universal Driver (1.5 2.x or 3.x).

 

Linear Hash Sizelock Administration Tool


sizelocktool

Before I get into the changes to the Universal Driver 4.x and 5.x, I’d like to touch on some of the great work that RevUS have been doing with regards to helping people to get more performance out of their systems.  Most of us use indexing and we try to optimise our code as much as possible, but Revelation are also tinkering with things and they have recently released some tools to help developers to get the most from their systems.

For those of you that have made the move to OpenInsight 9.4, (and those of you that have not yet made the move, now would be an ideal time) Revelation have a Linear Hash Sizelock Administration Tool for you.  This has been key to helping some of our larger clients with large systems to address some intermittent, but nonetheless, troubling issues.

In conjunction with the Linear Hash Reporting and Resiaing Tool released in March 2016, Revelation have published a sizelock administration tool called RTI_SET_SIZELOCK.  This tool will enable OpenInsight system administrators to view linear hash statisitcs within an application and identity and modify the file sizelock parameter.

Why is this important and useful?

The sizelock parameter is used to control the automatic resizing of the primary address space.  A sizelock of “0” indicates that a table should be allowed to expand as more rows are entered, and become smaller as rows are deleted.  A sizelock of “1” indicates that a table should be allowed to expand, but should not become smaller.  This is useful in cases of a presized table, when a preliminary expectation is for the table to hold 1000 rows, but more rows may be added.  If a sizelock is set to “2” or more, the table is not allowed to expand or contract.  A sizelock of “2” is very important during a select operation executed on a network, when one user is searching a table for information while another user is entering new data.

OpenInsight increments the sizelock by 2 for the duration of the select operation.  If another user is adding information, it is possible that the system could determine that a table needs to be resized.  If this were to occur during a select operation, the process might yield unpredictable results: rows that have not yet been evaluated might be skipped, or rows might be processed more than once.  Because the sizelock is “2”, no resizing takes place.  Once the process is finished, the sizelock is decremented by 2, restoring it to the table’s original sizelock value.  While the table might now be slightly out of size, the addition or deletion of another row will result in correct resizing.

The RTI_SET_SIZELOCK tool will allow an OpenInsight system administrator to globally set the sizlelock parameter on all or select tables.  For instance if a sizelock parameter is set to “2” or greater the table will not resize.  You can use this to disable resizing during busy periods and then re-enable resizing during quiet periods.

You can use this tool on any table, including ! and DICT files, as well as system files such as SYSLISTS and SYSOBJ.

RTI_SET_SIZELOCK is available at no cost to all current Works members and can be downloaded from the Works Download section of www.revelation.com.

 

Important News about the UD


benchtestresultsIf you are running, or have clients running, multi-user systems based on OpenInsight, there are a couple of postings in the WORKS discussion forum that you will have seen this morning relating to some big changes to the Universal Driver line up.

Because some of my readers are not on WORKS and because these new versions of the Universal Driver have some important fixes and new features that I know most end users will want to take advantage of I plan to write a few separate postings on each piece of news.

In the meantime, I would like to personally thank Bob Carten and Bryan Shumsky and one of my key UK clients who have worked extremely hard in identifying the cause of and correcting an issue in the Universal Driver.  Also, thank you to the team at Revelation who have added some much sought after new features in the UD5 and who have rolled those back for those of you running on OpenInsight version 9.4.

Watch for more coming from me over the next hour or so……

OI10 Update


didyouknowoi10As you would expect the recent Revelation Software conference was alive with everything OpenInsight 10 and, along with all of the new functionality, I was blown away with all of the new thinking and redesign of the toolset that is bound to make every developers workflow so much easier and faster.

I could fill this blog with month’s worth of articles exploring the new version of OpenInsight, but there is absolutely no point in reinventing the wheel and there are technical people doing a much better job than me.  Furthermore, the technical guys are a little more pragmatic and they don;t get caught up in the sales side of the topic.  I’ll therefore defer to:

Carl’s blog is made up of a number of articles that explore his part of developing the interface and the code changes that he has implemented.  This is a rich resource that is s must read for all OpenInsight developers.

Don at SRP has been blogging consistently since the conference and he is tackling specific subjects in some detail.

Andrew’s conference presentation was extremely well received, not least because it gave his audience a real world OpenInsight developers perspective of working with the Alpha that everyone went away with.  If you are planning on looking at the Alpha, then this is a must read blog article.

So what about the Alpha itself.  I had originally thought that it would only be available to conference attendees but following the way that it has been received, Revelation have now made it available to all OI WORKS subscribers via the WORKS area on www.revelation.com.

Armed with the Alpha and the OI10 resources linked above, this is the perfect time to take a look at what is coming in the beta in a few months time.  There is still a lot to be done, but now that Revelation are building OI10 in OI10, things are rushing forward at quite a pace and the official beta is now in sight.

I am still trying to find some time to take a slow walk through the Alpha but please drop me an email if you are not a WORKS Subscriber and would like to see the Alpha and to talk about what is coming.