Understanding the Linear Hash SIZELOCK parameter and Best Practice Usage


The sizelock parameter is used to control the automatic resizing of the primary address space.  A sizelock of “0” indicates that a table should be allowed to expand as more rows are entered, and become smaller as rows are deleted.  A sizelock of “1” indicates that a table should be allowed to expand, but should not become smaller.  This is useful in cases of a presized table, when a preliminary expectation is for the table to hold 1000 rows, but more rows may be added.  If a sizelock is set to “2” or more, the table is not allowed to expand or contract.  A sizelock of “2” is very important during a select operation executed on a network, when one user is searching a table for information while another user is entering new data.

Read On...

Linear Hash Sizelock Administration Tool for OI v9.4 released


In conjunction with the Linear Hash Reporting and Resizing Tool released in March 2016, Revelation Software are publishing a sizelock administration tool called RTI_SET_SIZELOCK.  This tool will enable OpenInsight system administrators to view linear hash statistics within an application and identify and modify the file sizelock parameter.

The sizelock parameter is used to control the automatic resizing of the primary address space. A sizelock of “0” indicates that a table should be allowed to expand as more rows are entered, and become smaller as rows are deleted. A sizelock of “1” indicates that a table should be allowed to expand, but should not become smaller. This is useful in cases of a presized table, when a preliminary expectation is for the table to hold 1000 rows, but more rows may be added. If a sizelock is set to “2” or more, the table is not allowed to expand or contract. A sizelock of “2” is very important during a select operation executed on a network, when one user is searching a table for information while another user is entering new data.

OpenInsight increments the sizelock by 2 for the duration of the select operation. If another user is adding information, it is possible that the system could determine that a table needs to be resized. If this were to occur during a select operation, the process might yield unpredictable results: rows that have not yet been evaluated might be skipped, or rows might be processed more than once. Because the sizelock is “2”, no resizing takes place. Once the process is finished, the sizelock is decremented by 2, restoring it to the table’s original sizelock value. While the table might now be slightly out of size, the addition or deletion of another row will result in correct resizing.

The RTI_SET_SIZELOCK tool will allow an OpenInsight system administrator to globally set the sizlelock parameter on all or select tables.  For instance if a sizelock parameter is set to “2” or greater the table will not resize.  You can use this to disable resizing during busy periods and then re-enable resizing during quiet periods.

You can use this tool on any table, including ! and DICT files, as well as system files such as SYSLISTS and SYSOBJ.

RTI_SET_SIZELOCK is available at no cost to all current Works members and can be downloaded from the Works Download section of our web site.

UD 5.0.0.4


benchtestresultsFor me, the release of the Universal Driver 5.0 for use with OpenInsight 9.4 was one of the best enhancements in recent times for people running OpenInsight.  It had new features that were sought after by many VARs and end users, and now it just got better with the release of 5.0.0.4.

Now available from www.revelation.com, this new version replaces all previous versions of the Universal Driver 5 and it is available at no cost to anyone who has purchased the Universal Driver 5.  Don’t feel left behind if you have an older version of the UD because we have an upgrade path for anyone running the UD and a new purchase option if this is your first time using the UD to enhance your deployed system.

So what’s all the excitement about?

The Universal Driver 5.0.0.4 resolves a potential memory leak /overrun issue when running in ‘debug output’ or logging mode.

It has also been enhanced to eliminate ‘hanging’ when shutting down the service.  This should allow sites to terminate and restart the Universal Driver without requiring a server reboot.

The new version addresses an issue that recently came to light where the record count and alpha values in the file header can get corrupted when several incomplete latent select / readnext are followed by a complete latent_Select/readnext.  This corruption has the side effect to cause the LK to expand rapidly.  A later complete select/readnext can repair the header and cause the UD to resize the table to normal size.  From the outside it looks like the service is bogging down for no reason.  In extreme cases the resize can crash the service and leave a huge file on the server.  It appears that this is a long-standing issue which may explain some of the issues witnessed by large sites.

The Volume Shadow Copy Service (VSS) feature now pauses the resizing thread and flushes all buffers.  Before 5.0.0.4 there was the possibility of a file being resized during the snapshot and of some new header record counts / alpha values not being written to disk.

Revelation have implemented the ability to pause the resizing thread.  The idea is to let the UD continue to queue files for resizing, pause the resize thread during the day so it does not slow the server, then release the thread in the evening to let the files catch up.

Revelation has also implemented a set-sizelock call inside the UD.  Prior to 5.0.0.4 the fix_lh program must OSBWrite the new sizelock into the header, which is not compatible with using a sharename to isolate the rev files.  The next UD Manager will support this feature.  The goal is to provide fine-grained control of resizing.  In theory you could pause the resizing thread, then view which files are going to resize, and control the order of the resizing by locking or releasing individual tables.

A “feature” where the UD will clear a sizelock of 1 has also been identified and removed in the new version.  Revelation has always stated that you presize a file, set sizelock to 1 and load it with data.  But, the UD clears the 1 on any select / readnext, making the feature less useful.  They have therefore removed this feature from the UD so that it is easier to pre-size tables and to leave them set to expand only.

The Universal Driver 5.0.0.4 software can be downloaded HERE.

Linear Hash Sizelock Administration Tool


sizelocktool

Before I get into the changes to the Universal Driver 4.x and 5.x, I’d like to touch on some of the great work that RevUS have been doing with regards to helping people to get more performance out of their systems.  Most of us use indexing and we try to optimise our code as much as possible, but Revelation are also tinkering with things and they have recently released some tools to help developers to get the most from their systems.

For those of you that have made the move to OpenInsight 9.4, (and those of you that have not yet made the move, now would be an ideal time) Revelation have a Linear Hash Sizelock Administration Tool for you.  This has been key to helping some of our larger clients with large systems to address some intermittent, but nonetheless, troubling issues.

In conjunction with the Linear Hash Reporting and Resiaing Tool released in March 2016, Revelation have published a sizelock administration tool called RTI_SET_SIZELOCK.  This tool will enable OpenInsight system administrators to view linear hash statisitcs within an application and identity and modify the file sizelock parameter.

Why is this important and useful?

The sizelock parameter is used to control the automatic resizing of the primary address space.  A sizelock of “0” indicates that a table should be allowed to expand as more rows are entered, and become smaller as rows are deleted.  A sizelock of “1” indicates that a table should be allowed to expand, but should not become smaller.  This is useful in cases of a presized table, when a preliminary expectation is for the table to hold 1000 rows, but more rows may be added.  If a sizelock is set to “2” or more, the table is not allowed to expand or contract.  A sizelock of “2” is very important during a select operation executed on a network, when one user is searching a table for information while another user is entering new data.

OpenInsight increments the sizelock by 2 for the duration of the select operation.  If another user is adding information, it is possible that the system could determine that a table needs to be resized.  If this were to occur during a select operation, the process might yield unpredictable results: rows that have not yet been evaluated might be skipped, or rows might be processed more than once.  Because the sizelock is “2”, no resizing takes place.  Once the process is finished, the sizelock is decremented by 2, restoring it to the table’s original sizelock value.  While the table might now be slightly out of size, the addition or deletion of another row will result in correct resizing.

The RTI_SET_SIZELOCK tool will allow an OpenInsight system administrator to globally set the sizlelock parameter on all or select tables.  For instance if a sizelock parameter is set to “2” or greater the table will not resize.  You can use this to disable resizing during busy periods and then re-enable resizing during quiet periods.

You can use this tool on any table, including ! and DICT files, as well as system files such as SYSLISTS and SYSOBJ.

RTI_SET_SIZELOCK is available at no cost to all current Works members and can be downloaded from the Works Download section of www.revelation.com.