Tagged: load test

Installing the Sitecore Digital Marketing System

The Sitecore Digital Marketing System (DMS) slots in alongside the Sitecore Content Management System (CMS), offering an array of additional features for your website such as multivariate testing, profiling and personalizing the customer journey, launching campaigns and setting business goals. The combination of the CMS and DMS is referred to as the Sitecore Customer Engagement Platform (CEP).

This article highlights some of the technical considerations we encountered in getting the DMS enabled on a high traffic, high availability website, so that we could start utilising its marketing tools.


First off, since Sitecore are currently binding new releases of the DMS with the CMS, we had to upgrade our version of the CMS before taking advantage of the latest DMS features.

Our existing version of the CMS was:

  • Sitecore 6.5.0 Update-3 (rev. 111230), DMS 2.0.0 (rev. 111230)

And the upgrade options we considered for the CMS were:

  • Sitecore CMS and DMS 6.6.0 rev. 130529 (6.6.0 Update-6)
  • Sitecore CMS and DMS 7.0 rev. 130424 (7.0 Initial Release)

We decided to upgrade to Sitecore 6.6 (update 6) as it gave us the DMS features we desired (such as improvements to the Executive Insight Dashboard, support for a separate reporting database, plus a number of fixes) as well as improvements to the CMS (incl. performance gains – more on that below).

We decided to hold off upgrading to Sitecore 7 for now (which offers, amongst other improvements, the use of item buckets to store vast amounts of child content without it having to display in the content tree, along with searching and indexing features) as it requires a shift to .NET Framework 4.5 and Visual Studio 2012, which we were yet to roll out due to other factors in our organisation.

Note: regarding the DMS fixes, we had previously attempted to enable DMS 2.0.0 rev. 111230 on our site – however, we encountered issues during load testing regarding database locking and long-running database queries, so took the decision not to proceed.

During the CMS upgrade we made use of the following resources:

Installation guide: http://sdn.sitecore.net/upload/sitecore6/66/installation_guide_sc66-a4.pdf

Release history (Sitecore login required): http://sdn.sitecore.net/Products/Sitecore%20V5/Sitecore%20CMS%206/ReleaseNotes/ChangeLog/Release%20History%20SC66.aspx

We deployed new Sitecore databases (core, master, web – adopting the Sitecore version number into our database naming convention) into each environment alongside our existing Sitecore databases (which were earmarked for decommissioning once upgrade complete). A content package was created from our existing Sitecore Editorial for population into these new databases (with our Sitecore editors informed of a content freeze whilst the upgrade took place). Our Sitecore web application was updated to target the new Sitecore binaries, as well as the various configuration changes highlighted in the release notes. A fresh client install was carried out on each of our environment web servers (uninstalling existing client and re-hardening security on the new client).

Note: we had investigated Sitecore’s ‘upgrade a previous Sitecore CMS 6 version to this release’ path (rather than implementing fresh installs of database/client) – however, with our existing client already security-hardened we were unable to use its upgrade tools.

We found a minor issue on upgrade: single-text fields would render html tags as text. Sitecore were aware of the issue and provided dll fix ‘Sitecore.Support.381846’ – however, we chose to update the fields to rich-text instead, to resolve.

During load testing, we found that our upgraded version of Sitecore was using roughly half as much CPU compared to the existing version. The performance benefits were tangible – our page download times had significantly improved.

Typical metrics from load testing show CPU level and test duration down on upgrade of Sitecore, confirming that our servers are less worked and our web pages are delivered to the customer faster:

Sitecore version Test settings Test Duration CPU level
6.5.0 Update-3 20 constant users, 5000 test iterations 18:00 min:sec 65.8%
6.6.0 Update-6 20 constant users, 5000 test iterations 12:21 min:sec 29.5%


With our CMS upgraded, we were now in a position to integrate our desired version of the DMS.

We installed the DMS Analytics database, along with a second copy which we would use for reporting (splitting the DMS into two databases allows for analysis of data using reporting tools without affecting site performance). The DMS also provides a SSIS (SQL Server Integration Services) package for replication of data between the Analytics and Reporting databases, plus a sql script for refreshing the Reporting database (note: the refresh can alternatively be configured in the Sitecore.Analytics.config file) – resources can be found at http://sdn.sitecore.net/SDN5/Reference/Sitecore%206/DMS%20Documentation.aspx

Our Sitecore web application required the following DMS config considerations:

Web.config: add attribute enableAnalytics=”true” to applicable <sites> (note: for our Editorial server we set value to false, as we did not want to track our editors activity)

Sitecore.Analytics.config: analytics sample rate could be throttled as desired (for example, setting the percentage value to 50 would mean only half of our website sessions would participate in DMS management)

  • <setting name=”Analytics.Sampling.Percentage” value=”100″ />

Sitecore.Analytics.config: the DNS lookup feature could be disabled if required

  • <setting name=”Analytics.PerformLookup” value=”false” />

ConnectionStrings.config: separate connection strings were utilized for the Analytics and Reporting databases

Layouts: our site layouts adopted a tag for robot detection

  • <sc:VisitorIdentification runat=”server”/>

We conducted load testing of our CMS before and after integration with the DMS, utilising the sample rate config to determine the effect on our website as more and more users participated in DMS management. We also load tested the effect on the site whilst DMS database truncation and replication were executed. As with previous load testing, we monitored the logs for errors and growth, profiled the database, monitored memory and performance counters on the servers – as well as analysing the test outputs under various conditions (number of constant users, test iterations, stress test, soak test). A record count of visits, page hits, etc confirmed that the Analytics database was storing data as expected. Integrating the DMS did not have any significant effect on performance, log file size or memory usage, although we did note that CPU increased by up to a quarter.

Typical metrics from load testing show CPU level and test duration slightly up on integration with DMS, confirming that although our servers are being worked a little harder our web pages are still being delivered to the customer in a timely manner:

Sitecore version Test settings Test Duration CPU level
6.6.0 Update-6(without DMS) 100 constant users, 3000 test iterations 17:40 min:sec 70%
6.6.0 Update-6 (with DMS 100% sample) 100 constant users, 3000 test iterations 18:11 min:sec 78%

Visual Studio 2010 was utilized for load testing. Several Web Performance tests were created, mimicking common customer website journeys (note these tests can be configured with variables to populate web forms and run under varying user accounts). Load tests were then created, allowing for a mix of Web Performance tests to be executed against a set number of constant users during a set number of iterations. For example, in the above typical metrics, we maintained a flow of 100 users against our site for a mix of 3000 customer journeys. To stress test we increased the number of constant users. To soak test we increased the number of iterations (so that it effectively ran all night).

Go-live strategy

Before switching on the DMS in our production environment, we put together an estimate of the record growth we would expect to see by calculating our current site traffic against the record counts we were experiencing in load test – this allowed us to provision the required database disc space.

We also opted for a soft-launch, going live with a sample rate of 10% and shifting to a full 100% rate once we were satisfied that the DMS integration had not raised any concerns (a rollback strategy was in place, utilising the config setting <setting name=”Analytics.Enabled” value=”false” />).

Our data replication between the Analytics and Reporting databases was also increased from daily to hourly after soft-launch, allowing for more frequent reporting.


The integration of the Sitecore Digital Marketing System (DMS) into the Sitecore Content Management System (CMS) offers a range of features for managing a customer’s experience with your website. Hopefully this article has gone some way to highlight the technical considerations in bringing the DMS on-board in a way which is thorough and sympathetic to the requirements of a high availability, high traffic environment.