Learn about... Data Integration
The new CRM system allows for numerous new opportunities, including most importantly, having a single, holistic view of student information from entrance to the University through post graduate interactions. In order to make this work, the data providing these opportunities must be timely, accurate, and defined by business processes. This is ultimately handled by migrating existing data and creating synchronization points between systems like Banner and Salesforce. In doing so, we must ensure that the data is of high quality and free from redundancy.
Understanding the Data:
One of the first steps needed in creating a successful data integration is creating a thorough understanding of how the data will work. This includes understanding criteria such as:
- How frequently does this data need to be refreshed in the CRM?
- Will the data elements change over time, and do these changes need to be reflected in downstream systems?
- Do the fields between source and destination map or is there additional logic needed in moving data?
These questions are often addressed at the beginning of the development cycle, in partnership with the Business Analysts in the Process Evaluation and Improvement team. One of the steps taken to provide an understanding of the data, is to map out the flow of data from source to final destination. Below is an example of one such data flow diagram.
Building Out the Integration:
Once the data fields are mapped out, and a solid understanding of how the data is to function is established, the Data Management team builds out ETL (extract, transform and load) data jobs using our data integration middleware platform to move data between systems. It is also during this phase that we apply any logic such as calculations or string manipulation (for example, merging a first name and last name into a full name field) into the data integration job.
Finalizing the Job:
Once a data migration job is complete, the Data Management team works closely with Business Analysts, Project Managers and Functional areas to validate that the data is flowing to the appropriate locations properly, looks correct, and is free from errors. Once this is confirmed, the data is pushed into production.