Qlik Replicate how to find the correct timestamp to resume a task

How to restart logstream task from timestamp

How do you find which timestamp to use to restart your tasks without missing data? This article will give insight into how to find the timestamp at which Replicate was last reading from at the source database.

Starting task from correct timestamp

When a Qlik Replicate task is stopped, an entry is created in the task logs. For example:

2023-10-19T14:32:51 [SORTER          ]I:  Final saved task state. Stream position 00000037.6ad8b1ed.00000001.0000.00.0000:244870.867634.16, Source id 684257314, next Target id 2193264361, confirmed Target id 2193264357, last source timestamp 1697686214000000 

The “last source timestamp” number “1697686214000000” can be converted into a readable format by going to https://www.epochconverter.com/

This timestamp represents the time at which this Qlik Replicate task was processing/reading from the source database’s transaction/redo logs when it was stopped. Also notice that the timestamp on the left in the logs “2023-10-19T14:32:51” differs from the source timestamp “2:30 PM”. This log timestamp should not be used as it does not indicate the source timestamp. Using it can result in data loss. Please use the source timestamp.

Once you have converted the source timestamp into a readable format, you can start the task by timestamp. You can choose to go back some time from this point to ensure that there are no missing 

changes, then clean up the duplicates at the target. In this case we go back one hour.

When the task is started from timestamp, the start of task log file should show the timestamp at which the task was started from in UTC time. This is because Qlik Replicate automatically converts server time to UTC in the logs. Example below:

2023-10-19T14:36:01 [TASK_MANAGER    ]I:  Task 'LogStreamTask' running CDC only in fresh start mode, starting from log position: 'timestamp:2023-10-19T02:30:14' (UTC).