UniVerse 11.3.2 supports replicating of Type 1 and 19 files via sequential I/O
UniVerse Type 1/19 files have been broadly used in UniVerse applications. In the older versions of UniVerse, writing a record into a Type 1/19 file via a standard WRITE statement or ED will be replicated. However, the updates done via sequential I/O capabilities of BASIC would not be replicated to the subscriber. This has been a major pain point for some of our MV customers because they need to handle this scenario by themselves but still face the risk of missing data on subscriber systems.
Now, in the recently released UniVerse 11.3.2, U2 Replication will support replicating updates done to records in UniVerse Type 1 and 19 files via sequential I/O (i.e. CREATE, WRITESEQ, WRITESEQF, WRITEBLK etc.).
Operations to be replicated
Logically any BASIC statements that make updates to a sequential file will be replicated to the subscriber, including:
Replication of CREATE
The BASIC CREATE statement creates a sequential file. If the parent directory, Type 1/19 file, is a replication object (please refer to the end of this blog post entitled Best practice – Configuration), the activity will be replicated to subscribers.
The ownership of the newly created sequential file follows the same rule as CREATE.FILE, that is:
- If RW_UID is configured in repconfig, the sequential file’s owner will be set to RW_UID;
- Otherwise, if the UID passed from the publisher is a valid user of the subscriber, then it is the owner of the sequential file to be created;
- Otherwise, the owner of the parent directory will be the owner of the sequential file to be created.
Replication of WRITESEQF, and FLUSH
A WRITESEQF statement consists of two steps: a WRITESEQ followed by a FLUSH.
For details regarding WRITESEQ, please refer to section “3. Replication of WRITESEQ, and WRITEBLK”. The FLUSH statement triggers sending all data present in the sequential buffer to the subscriber immediately.
Replication of WRITESEQ, and WRITEBLK
Both statements write data to sequential files. The difference is that WRITESEQ will append an extra newline character to the end of the data.
From a performance perspective, data written by WRITESEQ and WRITEBLK will be buffered by default and will not be immediately copied to the subscriber until one of the following three conditions is met.
- Buffer is full;
- One of the following operations is executed: – FLUSH, CLOSESEQ, WRITESEQF, WEOFSEQ, NOBUF;
- An executed operation affects the offset – SEEK, READSEQ, READBLK.
All these methods will ensure that data in the buffer is transferred to subscribers before the execution of the next statement. However, these actions will inevitably impact performance because of the extra I/O and network activities.
With that, you need to strike a balance between data consistency and performance. If data consistency is more important than performance in your business context, then you can use NOBUF at the very beginning of opening a sequential file, as shown below. This will close the buffer mechanism. Or, you can use one of the statements mentioned above in bullet point b and c when you think it’s time to forcibly send the data in the buffer to subscribers.
This statement truncates the sequential file at its current offset and then immediately informs the subscribers about the operation.
Limitations & Notes
- U2 Replication does not replicate operations on devices.
For example, the statement SEND writes a block of data to a device that has been opened for I/O using OPENDEV or OPENSEQ. This kind of operation will not be replicated to subscribers.
- U2 Replication doesn’t convert newline characters.
Your publisher and subscriber may have different newline characters, for example, CR+LF(‘\r\n’) for Linux, LF(‘\n’) for Windows. U2 Replication just transfers exactly what’s generated on the publisher to subscribers and doesn’t convert newline characters.
- Different behavior from previous versions
For users who have an account group, once you upgrade to UniVerse 11.3.2, the Type 1/19 files under this account will be replicated to subscribers if your application uses Sequential I/O. This behavior is different from previous versions. If you do not want to replicate these files, you’ll need to exclude them.
The configuration is very straightforward, you simply need to make sure the parent directory of the target sequential files is a replicate-able object. It can be defined using the FILE phrase in repconfig, or it can be implicitly defined in an account level group.
- Give it a shot!
Use the simple BASIC program below to quickly take advantage of this feature. All you need to do is run it and then verify that the data is the same between publisher and subscriber.
Here’s another example that represents the buffer mechanism of this new feature. The buffer size on Linux is 4096 bytes.
The feature mentioned in this blog post is available in UniVerse 11.3.2. Check out this blog post for more information on UniVerse 11.3.2.