Sunday, October 21, 2018

Unit testing Kentico with xUnit

With Kentico EMS 12 almost ready I noticed that support for MSTest has been dropped from the CMS.Test assembly. I think that’s a good thing given the (slow but steady) move towards .NET Core, where MSTest is no longer the default unit testing framework. I was sort of hoping for a bit more bold move to drop the dependency on a specific test framework all together, or to factor that dependency out to a separate Nuget package.
This is mostly because I really prefer xUnit over NUnit. It’s low ceremony approach to unit testing results in simple and clean code.
Luckily, there’s nothing stopping us from using xUnit with Kentico and at TrueLime we’ve been doing that for over 2 years now. I’ll get to the code shortly but first, it’s probably good to touch on the differences between NUnit and xUnit.
pexels-photo-1331975

Differences between NUnit and xUnit

In a nutshell, xUnit lacks most of the ceremony of older frameworks like NUnit:
  • There’s no Setup or TearDown – use constructor and IDisposable instead
  • Testclasses are not fixtures, Fixtures are in seperate classes to promote reuse
  • Each test runs in it’s own instance of the test class to improve isolation
  • Tests either pass or fail, there is no intermediate state
The net result is that xUnit tests mostly look like the rest of your code. This encourages developers to treat the test code with the same hygene as the application code (refactor, clean up etc.). It also makes it more natural to write clean tests. All the code involved in the test should go into the test, not into setup and teardown at any level.
If you do need to handle some sort of context around your test, like running Kentico, there’s always standard C# constructors and you can implement IDisposable to clean stuff up.’

Enough talk, time for code

Kentico provides support for working with it’s data APIs in unit tests, which is pretty cool. There are some caveats (see next section), but once you’re past those it works quite well and fast.
Unfortunately, since Kentico is based on NUnit we do need to handle some ceremony but we can tuck that away into a base class and keep it out of our test code.
public abstract class KenticoUnitTest : CMS.Tests.UnitTests, IDisposable
{
    protected KenticoUnitTest()
    {
        // Initialize Kentico test infrastructure
        InitFixtureBase();
        InitBase();
        UnitTestsSetUp(); // enable Kentico object faking
    }

    void IDisposable.Dispose()
    {
 // Cleanup Kentico Test infra
        CleanUpTestClass();
        ResetAllFakes();
        try
        {
            CleanUpBase();
            CleanUpFixtureBase();
        }
        catch( System.IO.PathTooLongException )
        {
            // this fails under VS Live testing but that is not critical
        }
    }
}

Kentico Unit Testing Caveats

  • Always use .WithData with Fake if you’re going to query that data. If not, you’re in for some very nasty and hard to decypher stack traces. For example:
    Fake<SettingsKeyInfo,SettingsKeyInfoProvider>().WithData();
  • If you do run into nasty stack traces, especially the ones that end in a failing DB connection, carefully read the first calls in the stack trace and try to figure out what entity is being used so you can fake it.
  • Be careful with VS Live unit testing. We’ve seen some tests failing due to errors unrelated to the test itself.
  • When using nCrunch for continuous testing,make sure you configure the project to copy in referenced assemblies. This is due  Kentico dynamically loading lots of assemblies while scanning for extensions like modules and custom data classes.
  • Custom data classes and other CMS extensions will only be available if the containing assembly is marked with the assembly discoverable attribute:
  • [assembly:CMS.AssemblyDiscoverable]

References

Wednesday, April 18, 2018

Kentico EMS - Enable bulk delete of form data

With GDPR right around the corner many of our clients are reviewing what data they have in their Kentico installation. One of the primary areas of concern is data collected through online forms.

GDPR and Kentico Online Forms

Kentico has a quite powerful online forms, making it easy to ask for input from your visitors. Unfortunately, privacy regulations including GDPR and it's predecessors state clearly that you cannot keep that data around after you're done processing it.

In addition to that, the best way to ensure data doesn't get stolen or leaked is not to have it in the first place.

Kentico does not make it particularly easy to manage high volumes of form input though. One of the glaring omissions is support for bulk deletion of form data.

The power of the UniGrid

Funny enough though, Kentico's data grids and data layer do support bulk delete out of the box. It's a matter of tuning the grid that shows the form data. The Kentico data grid is backed by the UniGrid control. In the case of form data, this control is defined in

CMS\CMSModules\BizForms\Controls\BizFormEditData.ascx

All that is needed to enable the mass delete action:

<cms:unigrid runat="server" id="gridData" islivesite="false" >
    ...
    <GridOptions DisplayFilter="true" />
    <%-- insert the following tag --%>
    <GridMassActions>
        <ug:MassAction Name="#delete" Caption="$General.Delete$" Behavior="openmodal" />
    </GridMassActions>
</cms:unigrid>

After this change, the form data grid will show a selection field as the first column and the delete action is available at the bottom of the grid.

Further reading

If you want to know more about the powers of the Kentico UniGrid, check the docs.

Thursday, March 15, 2018

Kentico EMS - Timeout in inactive contact cleanup

On high traffic sites Kentico's EMS feature is a treasure trove of marketing information but the large volume of data can also become your site's tombstone. As the Kentico guidance documentation rightfully points out, it makes no sense to keep older data around forever so you should setup a strategy to clear out inactive contacts from the start.

En garde!

Even when you properly setup the inactive contact cleanup though, the volume of data can get to a point where the contact cleanup tasks start to time out and nothing gets cleaned up any more. As it turns out, there is quite a bit you can do to prevent from getting into this situation. One of the keys is to keep an eye on your event log and your database. Especially long running queries and timeouts should raise a red flag.

Tuning the database

One of the key queries used by Kentico inactive contact cleanup is this one:

SELECT (COUNT(*)) AS [Count]
FROM (
  SELECT *
     FROM OM_Contact
     WHERE (([ContactEmail] = N'' OR [ContactEmail] IS NULL) AND (EXISTS (
          SELECT TOP 1 [ActivityContactID]
          FROM OM_Activity
          WHERE [ActivityContactID] = [ContactID]
          GROUP BY ActivityContactID
          HAVING MAX(ActivityCreated) <= '1/14/2018 2:00:20 AM'
    ) 
    OR ([ContactCreated] < '1/14/2018 2:00:20 AM' AND NOT EXISTS (
         SELECT TOP 1 [ActivityContactID]
         FROM OM_Activity
         WHERE [ActivityContactID] = [ContactID]
   ))))
) AS SubData

Given this query and the knowledge that there can be millions of rows in the OM_Activity table (17 million in this case) the middle part of the query really stands out as being risky:

SELECT TOP 1 [ActivityContactID]
FROM OM_Activity
WHERE [ActivityContactID] = @ContactId
GROUP BY ActivityContactID
HAVING MAX(ActivityCreated) <= '1/14/2018 2:00:20 AM'

It will effectively need to search through the entire OM_Activity table to figure out what contact has recent activities. Unfortunately, the default Kentico setup does not provide a covering index for this. This will force SQL Server to process the entire table, which is quite costly. In this particular case it took an Azure SQL S2 instance well over 40 minutes to execute this.

The covering index is pretty straight foreward and looks like this:

CREATE NONCLUSTERED INDEX [IX_ActivityByContactAndCreated]
ON [dbo].[OM_Activity] ([ActivityContactID])
INCLUDE ([ActivityCreated])

It takes SQL server a bit of time to build up this index so make sure you do that during off-peak hours. After applying it though, Kentico is able to determine the number of rows up for deletion in just over 1 minute, well within the configured timeouts.