We looked at the business impacts of the build process and how every marketer needs to worry about it. Here’s a technical dimension to managing the campaign build process effectively, though as a solution it is challenging to implement unless you own the marketing platform lock, stock and barrel.

Solution: Move the Build & Merge process outside the operational Email Marketing database.

Benefits:  

• Reduced DB Locking: Crucial DB Objects like Contact Lists and Attribute tables are freed up.
• Reduced Latch / Lock Manager contention and fewer dead locks.
• Accelerated Data Processing– Customer Data Imports are accelerated --new customers get into the system faster.
• Increased Online Subscriptions -Database is free to manage customer preferences, subscriptions , registrations
• Improved Event Data Exports:  Event Data exports get faster due to availability of crucial DB resources.

Implementation:

Moving the build process to an external repository is a challenge necessitating changes in Platform Architecture, Design and the overall Campaign Delivery Process. It could involve building new connectors, asynchronous call back and queuing mechanisms in order to make the external repository integrate with the current architecture. Some of the considerations in choosing an external repository are listed below.

Table:

The modified process architecture looks like the Figure 1 depicted below.

The Campaign Build Process was artificially replicated under two testing environments 

• A traditional RDBMS hosting the customer data and managing the build process internally.
• RDBMS Hosting the data and the build process running out of a NOSQL – In Memory DB.

The actual test conditions and results from the “Externalization” of the Campaign Build process are beyond the scope of this paper but some of the Positive outcomes of the experiment was a clear drop in memory footprint, significant performance improvement across the segmentation, personalization and content merging processes by over 100% for a sample size of a million customers.

The benefits while using the NOSQL In-Memory database can be attributed to the following reasons.

• Horizontal Scalability of the IN-Memory database.
• Improved Processing Efficiency since DISK IO was minimized.
• Reduced memory foot print and contention for system resources.
• Parallel Processing through concurrent Worker threads.

The following comparison depicts the kind of performance gains observed when executing the campaign build process using an In Memory NOSQL database ecosystem. The tests covered 3 categories of systems – RDBMS, In Memory NOSQL and In Memory NOSQL with three Parallel Job workers. All outputs were measured against the same CRM Database hosting 1 million customer records across 10 tables of personalization and supplemental information.

No SQL for the Build process works – make no bones about that – it is simply a question of who bells the cat and who takes the initiative to make the change.