Section 11 Data Management

Section 11 Data Management

Data Management

Data Management allows administrators to watch sessions and keep data synchronized with changes to the configuration by rebuilding data at various levels.

Section 11.1    Data Management Monitoring Levels

Data Management is organized by Levels in a Workspace. In a Workspace there are Project, Collections, Sessions, Records, Images and lastly attributes that make up the index of a record. Data Management allows administrators to watch uploads at the collection, session, and box level. Administrators can update record indices at the Collection and Session Level.

Section 11.1.1                   Monitor Upload Status

Project Managers, Team Leads, Administrators, and if given access, End-Users may want to monitor the Upload and Photo Pipeline status. Data Management allows users with access to monitor the process in real time.

Section 11.1.1.1              Collection Level Statistics

At the collection, the total sessions and record count along with the project name and deployed taxonomy are displayed in a grid. Select a collection to view the sessions uploaded to the collection.

Section 11.1.1.2             Session Level Statistics

At the session level, the session ID, session note, status and stats for the session are displayed in a grid. The Session statistics include expected record count, received record total, records that have completed the upload process, total image count, errors, and fatal errors. Errors can be inspected at the Record Level.

Section 11.1.1.3             Photo Pipeline Processes

The Photo Pipeline Process is the proprietary handling of images from the Capture Program on the WIB™ Unit to WIB™ Review. The Photo Pipeline is where the images are transmitted to WIB™ Review and prepared for end users. The following explains each step in the Photo Pipeline.


Section 11.1.1.3.1                 Upload

If there is an internet connection, records are transmitted to the photo pipeline as they are completed either on the WIB Unit or in the Mobile Application. Records are zipped prior to transmittal. A record either completely succeeds or completely fails during the transfer process because they are in zip files. A partial or part of a record cannot be uploaded. The WIB Unit and Mobile Application will make successive attempts to upload records until all are successful.

Section 11.1.1.3.2                  Virus Scan
The Photo Pipeline performs Hash Reputation based Malware detection by comparing hash values of newly uploaded files against the hash value of known viruses, trojans, spyware, and ransomware. For more information, please contact support@radixdata.com.
Section 11.1.1.3.3                 Unzip

When a complete record is successfully received it is then unzipped; images, session log and notes file are extracted from the zip file.

Section 11.1.1.3.4                 OCR

The extracted images are run through the OCR Engine.

Section 11.1.1.3.5                 Barcodes

The Photo Pipeline Process reads all barcodes from the images. The barcodes are then available to view in the Barcodes widget. See Section 6.2.5 Barcodes

Section 11.1.1.3.6                 Notes

The extracted notes file is processed, and metadata is stored in the system attributes for Notes.

Section 11.1.1.3.6.1     Session Note

Session notes appear after the session ID in the navigation tree and in the session description under data administration in WIB Review. Notes are displayed in the scanner notes field of third-party applications (see 3rd Party Application Scanner Notes for more details). The user who captured the note along with the date/time stamp the note was created are captured as metadata in system fields.

Section 11.1.1.3.6.2     Box Record Note

The record note information is stored in the system attributes for notes with a box record type. See below the System Attribute Label and the value stored in the attribute.

Section 11.1.1.3.6.3     Image Note

The image note information is stored in the system attributes for notes with an image record type. See below the System Attribute Label and the value stored in the attribute.

System Attribute Label

Note Value Stored

Value Format

Notes  

Note Text

string

Note User

User ID               

string

Created

Note Date Created

Date/Time mm:dd:yyyy hh:mm:ss AM/PM

 

Section 11.1.1.3.7                 Zip File Audit
The Photo Pipeline performs and audits the zip file content once unzipped. This is done by checking each image in the zip file against the capture log file to ensure that all images written to the log file are present in the zip file
Section 11.1.1.3.8                 Search Database

The complete OCR text is stored in the search database for future search queries.

Section 11.1.1.3.9                Review Package

If the processed images and metadata are loaded into a third-party application a Review Package is created for import into the application. If a third-party application is not used, this step is skipped in the photo pipeline.

Section 11.1.1.3.10                 Review Platform

WIB Review checks the import process for delivering the records to a third-party application. Upon completion, the stepper will show the number of records in the session are available in the third-party application. This process is skipped if a third-party application is not used.

Section 11.1.1.3.11                Session Audit

The session log is audited to ensure all records in the session have successfully processed through each step of the photo pipeline.

Section 11.1.1.4             Record Level Status

A process stepper shows how many records have successfully completed each step in the upload and photo pipeline process. At the record level status, each record is listed with a status.

Section 11.1.1.4.1                 Green Status

Signifies successful upload and processing through the photo pipeline without errors.

Section 11.1.1.4.2                Yellow status

Signifies successful upload and processing through the photo pipeline with errors.

Section 11.1.1.4.3                 Red status

Signifies a record with a fatal error during the upload or photo pipeline process. These records require further inspection and may require re-processing through one or more stages of the photo pipeline.

Section 11.1.1.5             Photo Pipeline Error Handling

Should there be an error during the photo pipeline process, an administrator has the ability to perform steps to correct the error. In the event the following steps do not clear the error, please contact Radix Data support at support@radixdata.com or open a support ticket using the support widget.


Section 11.1.1.5.1                  Re-Process

Selecting Re-Process will process the session and/or box(es) through the photo pipeline. Navigate to the sessions grid by selecting the Collection, then the session(s), and then Re-process. These are the same steps and administrator will take to update the records after a Taxonomy change. To Re-process a specific box or boxes continue the process and select a session then the box or boxes within the session and then Re-Process.

Section 11.1.1.5.2                 Session/Box Audit

There may be an instance where the session or box audit needs to be run after re-processing and session or box. To run the audit process on a session, select the session from the Data Management session grid and then select Session Audit. To run the Session Audit for a box that was re-processed, select the box or boxes and then using the drop down change the Audit to Box and run the audit on the box(es).

Section 11.1.2                  Update Indices

There are times that the attributes (indices) will require an update. If a regular expression, list, or the taxonomy are changed, if an attribute is added to the project, etc. There are different levels that indices can be updated. Indices can be updated at the Collection and/or Session level and in some instances for third-party applications. Contact support@radixdata.com to inquire if a third-party application is connected through an API to the update feature.

Section 11.1.2.1             Collection Level Update in WIB Review

To update all records at a collection level from the Collection Level Stats Grid, select the collection and rebuild attributes.

Section 11.1.2.2            Collection Level Update in Third-Party Application

To update all records stored in a third-party application, first rebuild the attributes (earlier step) then overlay the new values in the third-party application. This assumes there is an API for the third-party application. If the values do not update in the third-party application, please contact support@radixdata.com to make sure the third-party application update has been configured.

Section 11.1.2.3            Session Level Update in WIB Review

To update all records at a session level, select the collection from the Collection Level Status Grid, then select a session from the Session Level Stats Grid, last select rebuild attributes.

Section 11.1.2.4            Session Level Update in Third-Party Application

To update all records stored in a third-party application, first rebuild the attributes (earlier step) at the session level then overlay the new values in the third-party application. This assumes there is an API for the third-party application. If the values do not update in the third-party application, please contact support@radixdata.com to make sure the third-party application update has been configured.

Section 11.1.3                   Update OCR

Modifications to the Line Recognition configuration will require the reprocessing of OCR for the images. This feature groups the text into lines and paragraphs and is displayed in a more human readable format and improves the Regular Expression Data Extraction. To update the OCR select the Build Grouped Text from the Data Management Collections Grid.

To delete a box the uploaded zip file must also be deleted. Select Delete Zip file and provide a note as to the reason for deleting the box. A deletion note is required. You must also confirm that you want to delete the box.









    • Related Articles

    • Section 12 Security Management

      Security Management Security Management allows Administrators to set up users, groups, and roles which combined become a Security Policy. Policies are vital when it comes to keeping proper access rights to your most sensitive data. The ability to ...
    • Section 10 Configuration Management

      Configuration Management Configuration Manager allows you to create attributes, setup the automation and workflow for a project. Configuration Management allows you to configure the systems settings to reach product optimization and because changes ...
    • Section 11.1.2 Data Management Update Attributes

      Section 11.1.2 Update Indices There are times that the attributes (indices) will require an update. If a regular expression, list, or the taxonomy are changed, if an attribute is added to the project, etc. There are different levels that indices can ...
    • Section 9 Project Management

      Project Management A Project holds the overall design of the system. This area allows the Workspace Administrator to manage Projects and their parts (entities). A Project has the following entities: collections, attributes, automation, and workflow. ...
    • Section 9.1 Project Management Overview

      Section 9.1 Project Management Overview The overview simply informs the user about the specific area of the website. The overview also has a link to a wizard and quick links to other entities that need to be configured before creating a Project.