Big Data has been a topical issue for a number of years and as the need for integration increases so too does the need for adequate data storage and analytics. Data is a key driver within the Insurance industry as it forms the base on which key underwriting, claim and pricing decisions are made – whether it is new product development, product enhancements, product continuity or premium adjustments. As such, data integrity and accuracy is imperative.
In an industry marred by legacy issues, disparate storage and ineffective system integration the need for data cleansing and analytics is becoming vital to ensure future competitiveness.
But where does this process start?
Data cleansing and data issues
Current legislation is being driven towards an empowered consumer. This consumer has a variety of choices and options, for which insurance companies must cater. As a result, data cleansing is essential, but it is vital to ensure that cleansed data conforms to a set standard.
The data cleansing process typically involves identifying, correcting and removing corrupt or inaccurate data from a database. In this process, the data to be cleansed needs to be identified. Following this, the necessary data firstly needs to be obtained and archived, and then replaced with current or up-to-date data. The updated data then needs to be standardised. Revisiting validation rules, which is the criteria used in the processing of data, is a necessity.
There have been significant advances in this regard. System solution providers are using validation rules for data capturing of information associated with products and policyholders, using structured processes. High value exists in this unstructured and semi-structured information, as it is an untapped resource for insurance companies.
Avoiding future issues
Contributing to the existing challenge is the fact that, historically, there was no set of pre-existing industry rules. In order to rectify this, industry standards will need to be defined, agreed, formalized and complied with in the future.
In future, data cleansing may not be a major task if validation rules for applications are robust and comprehensive. From the beginning, data cleansing must result in data that can be imported or exported in a standard format. This idea for conformity will be further supported by the incoming Protection of Personal Information Act (POPI) legislation.
Why it is so important
The significance of data cleansing cannot be ignored. The overall process may involve a degree of manual intervention to obtain the latest and most up-to-date information from consumers. But the value from this process translates into immediate benefits, such as improved and professional Customer Relationship Management (CRM), accurate premium structures and efficient claims processes.
The effective incorporation of data cleansing and validation rules can be achieved through the establishment of an industry committee. However, the organisations involved in this committee must be open to collaboration with competitors and business-to-business (B2B) partners. In the end, the aim will be the agreement by the industry to data standards at a database level, which will enable data to be secure, with no compromise in data integrity and ultimately a more effective use of data.