Hacking Sitecore and Life one pipeline at a time!

Wheels on the Rebus Go Queue, Queue, Queue

Exploring Sitecore’s Rebus Implementation

What an amazing event the SUGCON EU 2019 was! I am humbled and grateful to have been offered an opportunity to speak in London last week! Enjoyed seeing, meeting, and hanging out with the Sitecore Community and learning from each other the incredible talent that we bring to this community.

For SUGCON EU 2019, I chose to get real technical. Talking about the exploration of the Rebus.NET implementation that Sitecore included in the platform starting in version 9.0.1. This session, we learned what a message queue and service bus is, and I took the audience on an inclusive experience turning the audience into a real live working demonstration of a service bus. Showcasing how using a message queue and service bus pattern, scale becomes a non-factor over the direct connection alternative.

Wheels on the Rebus Presentation

London buses taken with tilt and shift to give narrow depth of field.

Example Code

Below is the example code that I walked through during the Demo portion of the presentation. This code is provided as is and is not guaranteed to work as shown in your implementations. It is provided to demonstrate how you would go about implementing your own queue.

[
{
"campaign_id": "1234",
"contact_identifier": "1234",
"email": "pnavarra65456@connectivedx.com",
"email_address_history_entry_id": "1234",
"event": "bounce",
"instance_id": "1234",
"ip": "167.89.106.58",
"message_id": "1234",
"reason": "550 5.4.1 [pnavarra65456@connectivedx.com]: Recipient address rejected: Access denied [BY2NAM05FT005.eop-nam05.prod.protection.outlook.com]",
"sg_event_id": "K-8dsTHFRqKipipvI40C3g",
"sg_message_id": "cJzCzytgTeiyRns9uybNaQ.filter0071p3iad2-27823-5C320D7B-2A.0",
"smtp-id": "<cJzCzytgTeiyRns9uybNaQ@ismtpd0051p1iad1.sendgrid.net>",
"status": "5.4.1",
"target_language": "1234",
"test_value_index": "1234",
"timestamp": 1546784125,
"tls": 1,
"type": "bounce"
}
]
view raw 0.bounce.json hosted with ❤ by GitHub
namespace SitecoreHacker.London.Foundation.EmailCampaign.EDS.Messaging.Buses
{
public sealed class EmailBounceMessageBus
{
}
}
using System;
using Newtonsoft.Json;
namespace SitecoreHacker.London.Foundation.EmailCampaign.EDS.Messaging.Model
{
[Serializable]
public class EmailBounceMessage : EventMessageBase
{
[JsonProperty("ip")]
public string IPAddress { get; set; }
[JsonProperty("reason")]
public string BounceReason { get; set; }
[JsonProperty("status")]
public string BounceStatus { get; set; }
[JsonProperty("type")]
public string BounceType { get; set; }
}
}
using System;
using System.Threading.Tasks;
using System.Web.Http;
using Newtonsoft.Json;
using Newtonsoft.Json.Converters;
using Newtonsoft.Json.Linq;
using Sitecore.Framework.Messaging;
using Sitecore.Modules.EmailCampaign.Core.Contacts;
using Sitecore.Modules.EmailCampaign.Core.Pipelines.HandleMessageEventBase;
using SitecoreHacker.London.Foundation.EmailCampaign.EDS.Messaging.Buses;
using SitecoreHacker.London.Foundation.EmailCampaign.EDS.Messaging.Model;
using SitecoreHacker.London.Foundation.EmailCampaign.Providers.SendGrid.Models;
namespace SitecoreHacker.London.Foundation.EmailCampaign.EDS.Events
{
public class WebhookController : ApiController
{
private readonly IMessageBus<EmailBounceMessageBus> _emailBounceMessagesBus;
private readonly IEventDataService _eventDataService;
public const string Endpoint = "eds/events";
public const string Keyname = "apikey";
public WebhookController(IMessageBus<EmailBounceMessageBus> emailBounceMessagesBus, IEventDataService eventDataService)
{
_emailBounceMessagesBus = emailBounceMessagesBus;
_eventDataService = eventDataService;
}
[HttpPost]
[ActionName("DefaultAction")]
[Route(Endpoint)]
public async Task<IHttpActionResult> ProcessEvent(string apikey)
{
string result = await Request.Content.ReadAsStringAsync();
var content = JsonConvert.DeserializeObject<ISendGridHook[]>(result, new SendGridHookConverter());
try
{
foreach (var hookMessage in content)
{
switch (hookMessage)
{
case Bounce b:
var bounceMessge = new EmailBounceMessage();
bounceMessge.BounceReason = b.reason;
bounceMessge.BounceStatus = b.status;
bounceMessge.BounceType = b.type;
bounceMessge.ContactIdentifier = _eventDataService.Decrypt(b.contact_identifier)?.ToContactIdentifier();
bounceMessge.MessageId = Guid.TryParse(_eventDataService.Decrypt(b.message_id), out var messageid) ? messageid : Guid.Empty;
bounceMessge.CampaignId = Guid.TryParse(b.campaign_id, out var campaignGuid) ? campaignGuid : Guid.Empty;
bounceMessge.IPAddress = b.ip;
bounceMessge.EmailAddressHistoryEntryId = b.email_address_history_entry_id;
bounceMessge.InstanceId = Guid.TryParse(_eventDataService.Decrypt(b.instance_id), out var instanceId) ? instanceId : Guid.Empty;
bounceMessge.TargetLanguage = b.target_language;
bounceMessge.TestValueIndex = byte.TryParse(b.test_value_index.ToString(),out var byteNum) ? byteNum : byte.MaxValue;
DateTimeOffset dateTimeOffset = DateTimeOffset.FromUnixTimeSeconds(b.timestamp);
bounceMessge.TimeStamp = dateTimeOffset.DateTime;
await _emailBounceMessagesBus.SendAsync(bounceMessge);
break;
case Complaint c:
break;
}
}
return Ok(content.Length);
}
catch (Exception e)
{
return BadRequest(e.Message);
}
}
}
}
using System;
using System.Threading.Tasks;
using Sitecore.EmailCampaign.Cm.Pipelines.HandleBounce;
using Sitecore.EmailCampaign.Model.Messaging;
using Sitecore.EmailCampaign.Model.XConnect.Events;
using Sitecore.ExM.Framework.Diagnostics;
using Sitecore.Framework.Messaging;
using Sitecore.Framework.Messaging.DeferStrategies;
using Sitecore.Modules.EmailCampaign.Core;
using Sitecore.Modules.EmailCampaign.Core.Contacts;
using SitecoreHacker.London.Foundation.EmailCampaign.EDS.Messaging.Buses;
using SitecoreHacker.London.Foundation.EmailCampaign.EDS.Messaging.Model;
namespace SitecoreHacker.London.Foundation.EmailCampaign.EDS.Messaging.Handlers
{
public class EmailBounceMessageHandler : IMessageHandler<EmailBounceMessage>
{
private readonly ILogger _logger;
private readonly PipelineHelper _pipelineHelper;
private readonly IDeferStrategy<DeferDetectionByResultBase<HandlerResult>> _deferStrategy;
private readonly IMessageBus<EmailBounceMessageBus> _bus;
public EmailBounceMessageHandler(ILogger logger, PipelineHelper pipelineHelper, IDeferStrategy<DeferDetectionByResultBase<HandlerResult>> deferStrategy, IMessageBus<EmailBounceMessageBus> bus)
{
_logger = logger;
_pipelineHelper = pipelineHelper;
_deferStrategy = deferStrategy;
_bus = bus;
}
public async Task Handle(EmailBounceMessage message, IMessageReceiveContext receiveContext, IMessageReplyContext replyContext)
{
var result = await _deferStrategy
.ExecuteAsync(_bus, message, receiveContext, () => RegisterBounce(message));
_logger.LogInfo(
result.Deferred
? $"[EmailBounceMessageHandler] defered message \'{message.MessageId}\' to \'{message.ContactIdentifier.ToLogFile()}\'."
: $"[EmailBounceMessageHandler] processed message \'{message.MessageId}\' to \'{message.ContactIdentifier.ToLogFile()}\'");
}
private HandlerResult RegisterBounce(EmailBounceMessage message)
{
try
{
var contactIdentifier = message.ContactIdentifier;
var emailOpenedEvent = new BounceEvent(message.TimeStamp);
var messageId = message.MessageId;
emailOpenedEvent.MessageId = messageId;
var instanceId = message.InstanceId;
emailOpenedEvent.InstanceId = instanceId;
var targetLanguage = message.TargetLanguage;
emailOpenedEvent.MessageLanguage = targetLanguage;
var testValueIndex = message.TestValueIndex;
emailOpenedEvent.TestValueIndex = testValueIndex;
var addressHistoryEntryId = message.EmailAddressHistoryEntryId;
emailOpenedEvent.EmailAddressHistoryEntryId = addressHistoryEntryId;
_pipelineHelper.RunPipeline("handleUndeliveredMessage", new HandleBounceArgs(new EventData(contactIdentifier, emailOpenedEvent), true));
}
catch (Exception ex)
{
_logger.LogError("Failed to process an email opened task", ex);
return new HandlerResult(HandlerResultType.Error);
}
return new HandlerResult(HandlerResultType.Successful);
}
}
}
using System;
using Sitecore.Framework.Messaging;
using Sitecore.Pipelines;
using SitecoreHacker.London.Foundation.EmailCampaign.EDS.Messaging.Buses;
namespace SitecoreHacker.London.Foundation.EmailCampaign.EDS.Messaging.Pipelines.Initialize
{
public class InitializeMessageBus
{
private readonly IServiceProvider _serviceProvider;
public InitializeMessageBus(IServiceProvider serviceProvider)
{
_serviceProvider = serviceProvider;
}
public void Process(PipelineArgs args)
{
_serviceProvider.StartMessageBus<EmailBounceMessageBus>();
}
}
}
using Microsoft.Extensions.DependencyInjection;
using Sitecore.DependencyInjection;
using Sitecore.Framework.Messaging;
using SitecoreHacker.London.Foundation.EmailCampaign.EDS.Messaging.Handlers;
using SitecoreHacker.London.Foundation.EmailCampaign.EDS.Messaging.Model;
namespace SitecoreHacker.London.Foundation.DependencyInjection
{
public class WebhookConfigurator : IServicesConfigurator
{
public void Configure(IServiceCollection serviceCollection)
{
// configurator per project? Use this:
serviceCollection.AddMvcControllersInCurrentAssembly();
serviceCollection.AddTransient<IMessageHandler<EmailBounceMessage>, EmailBounceMessageHandler>();
}
}
}
<?xml version="1.0" encoding="utf-8" ?>
<configuration xmlns:x="http://www.sitecore.net/xmlconfig/" xmlns:role="http://www.sitecore.net/xmlconfig/role/" xmlns:eds="http://www.sitecore.net/xmlconfig/eds/" xmlns:exmEnabled="http://www.sitecore.net/xmlconfig/exmEnabled/">
<sitecore exmEnabled:require="yes">
<pipelines>
<initialize>
<processor type="SitecoreHacker.London.Foundation.DependencyInjection.Initialize.RegisterWebhookRoute, SitecoreHacker.London" />
<processor type="SitecoreHacker.London.Foundation.EmailCampaign.EDS.Messaging.Pipelines.Initialize.InitializeMessageBus, SitecoreHacker.London" resolve="true" />
</initialize>
</pipelines>
<services>
<configurator type="SitecoreHacker.London.Foundation.DependencyInjection.WebhookConfigurator, SitecoreHacker.London" />
</services>
<Messaging>
<Rebus>
<SitecoreHacker.London.Foundation.EmailCampaign.EDS.Messaging.Buses.EmailBounceMessageBus>
<Transport>
<SqlServer>
<OneWay role:require="(Standalone or ContentManagement) and !ContentDelivery">false</OneWay>
<OneWay role:require="ContentDelivery">true</OneWay>
<ConnectionStringOrName>messaging</ConnectionStringOrName>
<TableName>Sitecore_Transport</TableName>
<InputQueueName>EmailBounceMessageQueue</InputQueueName>
</SqlServer>
</Transport>
<Routing>
<TypeBasedMappings>
<TypeMappings>
<EmailBounceMapping>
<Type>SitecoreHacker.London.Foundation.EmailCampaign.EDS.Messaging.Model.EmailBounceMessage, SitecoreHacker.London</Type>
<DestinationQueue>EmailBounceMessageQueue</DestinationQueue>
</EmailBounceMapping>
</TypeMappings>
</TypeBasedMappings>
</Routing>
<Options role:require="Standalone or ContentManagement">
<SetNumberOfWorkers>1</SetNumberOfWorkers>
<SimpleRetryStrategy>
<ErrorQueueAddress>Error</ErrorQueueAddress>
<MaxDeliveryAttempts>1</MaxDeliveryAttempts>
<SecondLevelRetriesEnabled>false</SecondLevelRetriesEnabled>
</SimpleRetryStrategy>
</Options>
<Logging Type="Sitecore.Messaging.SitecoreLoggerFactory,Sitecore.Messaging"/>
</SitecoreHacker.London.Foundation.EmailCampaign.EDS.Messaging.Buses.EmailBounceMessageBus>
</Rebus>
</Messaging>
</sitecore>
</configuration>

Summary

As I explained during the session, the biggest notion to take home is that while we can utilize this pattern in any solution, transport and how Sitecore has implemented Rebus could very well change in future releases, near and or far.

I hope that anyone attending enjoyed my silly little presentation. If even one person learned something, this was a successful session. If you did attend, THANK YOU for your participation!

Tagged as: , ,

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 1,004 other followers

Blog Stats

  • 108,031 hits
Follow Sitecore Hacker on WordPress.com
Sitecore® and Own the Experience® are registered trademarks of Sitecore Corporation A/S in the U.S. and other countries.  This website is independent of Sitecore Corporation, and is not affiliated with or sponsored by Sitecore Corporation.
%d bloggers like this: