Exploring Sitecore’s Rebus Implementation
What an amazing event the SUGCON EU 2019 was! I am humbled and grateful to have been offered an opportunity to speak in London last week! Enjoyed seeing, meeting, and hanging out with the Sitecore Community and learning from each other the incredible talent that we bring to this community.
For SUGCON EU 2019, I chose to get real technical. Talking about the exploration of the Rebus.NET implementation that Sitecore included in the platform starting in version 9.0.1. This session, we learned what a message queue and service bus is, and I took the audience on an inclusive experience turning the audience into a real live working demonstration of a service bus. Showcasing how using a message queue and service bus pattern, scale becomes a non-factor over the direct connection alternative.
Wheels on the Rebus Presentation
Example Code
Below is the example code that I walked through during the Demo portion of the presentation. This code is provided as is and is not guaranteed to work as shown in your implementations. It is provided to demonstrate how you would go about implementing your own queue.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| [ | |
| { | |
| "campaign_id": "1234", | |
| "contact_identifier": "1234", | |
| "email": "pnavarra65456@connectivedx.com", | |
| "email_address_history_entry_id": "1234", | |
| "event": "bounce", | |
| "instance_id": "1234", | |
| "ip": "167.89.106.58", | |
| "message_id": "1234", | |
| "reason": "550 5.4.1 [pnavarra65456@connectivedx.com]: Recipient address rejected: Access denied [BY2NAM05FT005.eop-nam05.prod.protection.outlook.com]", | |
| "sg_event_id": "K-8dsTHFRqKipipvI40C3g", | |
| "sg_message_id": "cJzCzytgTeiyRns9uybNaQ.filter0071p3iad2-27823-5C320D7B-2A.0", | |
| "smtp-id": "<cJzCzytgTeiyRns9uybNaQ@ismtpd0051p1iad1.sendgrid.net>", | |
| "status": "5.4.1", | |
| "target_language": "1234", | |
| "test_value_index": "1234", | |
| "timestamp": 1546784125, | |
| "tls": 1, | |
| "type": "bounce" | |
| } | |
| ] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| namespace SitecoreHacker.London.Foundation.EmailCampaign.EDS.Messaging.Buses | |
| { | |
| public sealed class EmailBounceMessageBus | |
| { | |
| } | |
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| using System; | |
| using Newtonsoft.Json; | |
| namespace SitecoreHacker.London.Foundation.EmailCampaign.EDS.Messaging.Model | |
| { | |
| [Serializable] | |
| public class EmailBounceMessage : EventMessageBase | |
| { | |
| [JsonProperty("ip")] | |
| public string IPAddress { get; set; } | |
| [JsonProperty("reason")] | |
| public string BounceReason { get; set; } | |
| [JsonProperty("status")] | |
| public string BounceStatus { get; set; } | |
| [JsonProperty("type")] | |
| public string BounceType { get; set; } | |
| } | |
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| using System; | |
| using System.Threading.Tasks; | |
| using System.Web.Http; | |
| using Newtonsoft.Json; | |
| using Newtonsoft.Json.Converters; | |
| using Newtonsoft.Json.Linq; | |
| using Sitecore.Framework.Messaging; | |
| using Sitecore.Modules.EmailCampaign.Core.Contacts; | |
| using Sitecore.Modules.EmailCampaign.Core.Pipelines.HandleMessageEventBase; | |
| using SitecoreHacker.London.Foundation.EmailCampaign.EDS.Messaging.Buses; | |
| using SitecoreHacker.London.Foundation.EmailCampaign.EDS.Messaging.Model; | |
| using SitecoreHacker.London.Foundation.EmailCampaign.Providers.SendGrid.Models; | |
| namespace SitecoreHacker.London.Foundation.EmailCampaign.EDS.Events | |
| { | |
| public class WebhookController : ApiController | |
| { | |
| private readonly IMessageBus<EmailBounceMessageBus> _emailBounceMessagesBus; | |
| private readonly IEventDataService _eventDataService; | |
| public const string Endpoint = "eds/events"; | |
| public const string Keyname = "apikey"; | |
| public WebhookController(IMessageBus<EmailBounceMessageBus> emailBounceMessagesBus, IEventDataService eventDataService) | |
| { | |
| _emailBounceMessagesBus = emailBounceMessagesBus; | |
| _eventDataService = eventDataService; | |
| } | |
| [HttpPost] | |
| [ActionName("DefaultAction")] | |
| [Route(Endpoint)] | |
| public async Task<IHttpActionResult> ProcessEvent(string apikey) | |
| { | |
| string result = await Request.Content.ReadAsStringAsync(); | |
| var content = JsonConvert.DeserializeObject<ISendGridHook[]>(result, new SendGridHookConverter()); | |
| try | |
| { | |
| foreach (var hookMessage in content) | |
| { | |
| switch (hookMessage) | |
| { | |
| case Bounce b: | |
| var bounceMessge = new EmailBounceMessage(); | |
| bounceMessge.BounceReason = b.reason; | |
| bounceMessge.BounceStatus = b.status; | |
| bounceMessge.BounceType = b.type; | |
| bounceMessge.ContactIdentifier = _eventDataService.Decrypt(b.contact_identifier)?.ToContactIdentifier(); | |
| bounceMessge.MessageId = Guid.TryParse(_eventDataService.Decrypt(b.message_id), out var messageid) ? messageid : Guid.Empty; | |
| bounceMessge.CampaignId = Guid.TryParse(b.campaign_id, out var campaignGuid) ? campaignGuid : Guid.Empty; | |
| bounceMessge.IPAddress = b.ip; | |
| bounceMessge.EmailAddressHistoryEntryId = b.email_address_history_entry_id; | |
| bounceMessge.InstanceId = Guid.TryParse(_eventDataService.Decrypt(b.instance_id), out var instanceId) ? instanceId : Guid.Empty; | |
| bounceMessge.TargetLanguage = b.target_language; | |
| bounceMessge.TestValueIndex = byte.TryParse(b.test_value_index.ToString(),out var byteNum) ? byteNum : byte.MaxValue; | |
| DateTimeOffset dateTimeOffset = DateTimeOffset.FromUnixTimeSeconds(b.timestamp); | |
| bounceMessge.TimeStamp = dateTimeOffset.DateTime; | |
| await _emailBounceMessagesBus.SendAsync(bounceMessge); | |
| break; | |
| case Complaint c: | |
| break; | |
| } | |
| } | |
| return Ok(content.Length); | |
| } | |
| catch (Exception e) | |
| { | |
| return BadRequest(e.Message); | |
| } | |
| } | |
| } | |
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| using System; | |
| using System.Threading.Tasks; | |
| using Sitecore.EmailCampaign.Cm.Pipelines.HandleBounce; | |
| using Sitecore.EmailCampaign.Model.Messaging; | |
| using Sitecore.EmailCampaign.Model.XConnect.Events; | |
| using Sitecore.ExM.Framework.Diagnostics; | |
| using Sitecore.Framework.Messaging; | |
| using Sitecore.Framework.Messaging.DeferStrategies; | |
| using Sitecore.Modules.EmailCampaign.Core; | |
| using Sitecore.Modules.EmailCampaign.Core.Contacts; | |
| using SitecoreHacker.London.Foundation.EmailCampaign.EDS.Messaging.Buses; | |
| using SitecoreHacker.London.Foundation.EmailCampaign.EDS.Messaging.Model; | |
| namespace SitecoreHacker.London.Foundation.EmailCampaign.EDS.Messaging.Handlers | |
| { | |
| public class EmailBounceMessageHandler : IMessageHandler<EmailBounceMessage> | |
| { | |
| private readonly ILogger _logger; | |
| private readonly PipelineHelper _pipelineHelper; | |
| private readonly IDeferStrategy<DeferDetectionByResultBase<HandlerResult>> _deferStrategy; | |
| private readonly IMessageBus<EmailBounceMessageBus> _bus; | |
| public EmailBounceMessageHandler(ILogger logger, PipelineHelper pipelineHelper, IDeferStrategy<DeferDetectionByResultBase<HandlerResult>> deferStrategy, IMessageBus<EmailBounceMessageBus> bus) | |
| { | |
| _logger = logger; | |
| _pipelineHelper = pipelineHelper; | |
| _deferStrategy = deferStrategy; | |
| _bus = bus; | |
| } | |
| public async Task Handle(EmailBounceMessage message, IMessageReceiveContext receiveContext, IMessageReplyContext replyContext) | |
| { | |
| var result = await _deferStrategy | |
| .ExecuteAsync(_bus, message, receiveContext, () => RegisterBounce(message)); | |
| _logger.LogInfo( | |
| result.Deferred | |
| ? $"[EmailBounceMessageHandler] defered message \'{message.MessageId}\' to \'{message.ContactIdentifier.ToLogFile()}\'." | |
| : $"[EmailBounceMessageHandler] processed message \'{message.MessageId}\' to \'{message.ContactIdentifier.ToLogFile()}\'"); | |
| } | |
| private HandlerResult RegisterBounce(EmailBounceMessage message) | |
| { | |
| try | |
| { | |
| var contactIdentifier = message.ContactIdentifier; | |
| var emailOpenedEvent = new BounceEvent(message.TimeStamp); | |
| var messageId = message.MessageId; | |
| emailOpenedEvent.MessageId = messageId; | |
| var instanceId = message.InstanceId; | |
| emailOpenedEvent.InstanceId = instanceId; | |
| var targetLanguage = message.TargetLanguage; | |
| emailOpenedEvent.MessageLanguage = targetLanguage; | |
| var testValueIndex = message.TestValueIndex; | |
| emailOpenedEvent.TestValueIndex = testValueIndex; | |
| var addressHistoryEntryId = message.EmailAddressHistoryEntryId; | |
| emailOpenedEvent.EmailAddressHistoryEntryId = addressHistoryEntryId; | |
| _pipelineHelper.RunPipeline("handleUndeliveredMessage", new HandleBounceArgs(new EventData(contactIdentifier, emailOpenedEvent), true)); | |
| } | |
| catch (Exception ex) | |
| { | |
| _logger.LogError("Failed to process an email opened task", ex); | |
| return new HandlerResult(HandlerResultType.Error); | |
| } | |
| return new HandlerResult(HandlerResultType.Successful); | |
| } | |
| } | |
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| using System; | |
| using Sitecore.Framework.Messaging; | |
| using Sitecore.Pipelines; | |
| using SitecoreHacker.London.Foundation.EmailCampaign.EDS.Messaging.Buses; | |
| namespace SitecoreHacker.London.Foundation.EmailCampaign.EDS.Messaging.Pipelines.Initialize | |
| { | |
| public class InitializeMessageBus | |
| { | |
| private readonly IServiceProvider _serviceProvider; | |
| public InitializeMessageBus(IServiceProvider serviceProvider) | |
| { | |
| _serviceProvider = serviceProvider; | |
| } | |
| public void Process(PipelineArgs args) | |
| { | |
| _serviceProvider.StartMessageBus<EmailBounceMessageBus>(); | |
| } | |
| } | |
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| using Microsoft.Extensions.DependencyInjection; | |
| using Sitecore.DependencyInjection; | |
| using Sitecore.Framework.Messaging; | |
| using SitecoreHacker.London.Foundation.EmailCampaign.EDS.Messaging.Handlers; | |
| using SitecoreHacker.London.Foundation.EmailCampaign.EDS.Messaging.Model; | |
| namespace SitecoreHacker.London.Foundation.DependencyInjection | |
| { | |
| public class WebhookConfigurator : IServicesConfigurator | |
| { | |
| public void Configure(IServiceCollection serviceCollection) | |
| { | |
| // configurator per project? Use this: | |
| serviceCollection.AddMvcControllersInCurrentAssembly(); | |
| serviceCollection.AddTransient<IMessageHandler<EmailBounceMessage>, EmailBounceMessageHandler>(); | |
| } | |
| } | |
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| <?xml version="1.0" encoding="utf-8" ?> | |
| <configuration xmlns:x="http://www.sitecore.net/xmlconfig/" xmlns:role="http://www.sitecore.net/xmlconfig/role/" xmlns:eds="http://www.sitecore.net/xmlconfig/eds/" xmlns:exmEnabled="http://www.sitecore.net/xmlconfig/exmEnabled/"> | |
| <sitecore exmEnabled:require="yes"> | |
| <pipelines> | |
| <initialize> | |
| <processor type="SitecoreHacker.London.Foundation.DependencyInjection.Initialize.RegisterWebhookRoute, SitecoreHacker.London" /> | |
| <processor type="SitecoreHacker.London.Foundation.EmailCampaign.EDS.Messaging.Pipelines.Initialize.InitializeMessageBus, SitecoreHacker.London" resolve="true" /> | |
| </initialize> | |
| </pipelines> | |
| <services> | |
| <configurator type="SitecoreHacker.London.Foundation.DependencyInjection.WebhookConfigurator, SitecoreHacker.London" /> | |
| </services> | |
| <Messaging> | |
| <Rebus> | |
| <SitecoreHacker.London.Foundation.EmailCampaign.EDS.Messaging.Buses.EmailBounceMessageBus> | |
| <Transport> | |
| <SqlServer> | |
| <OneWay role:require="(Standalone or ContentManagement) and !ContentDelivery">false</OneWay> | |
| <OneWay role:require="ContentDelivery">true</OneWay> | |
| <ConnectionStringOrName>messaging</ConnectionStringOrName> | |
| <TableName>Sitecore_Transport</TableName> | |
| <InputQueueName>EmailBounceMessageQueue</InputQueueName> | |
| </SqlServer> | |
| </Transport> | |
| <Routing> | |
| <TypeBasedMappings> | |
| <TypeMappings> | |
| <EmailBounceMapping> | |
| <Type>SitecoreHacker.London.Foundation.EmailCampaign.EDS.Messaging.Model.EmailBounceMessage, SitecoreHacker.London</Type> | |
| <DestinationQueue>EmailBounceMessageQueue</DestinationQueue> | |
| </EmailBounceMapping> | |
| </TypeMappings> | |
| </TypeBasedMappings> | |
| </Routing> | |
| <Options role:require="Standalone or ContentManagement"> | |
| <SetNumberOfWorkers>1</SetNumberOfWorkers> | |
| <SimpleRetryStrategy> | |
| <ErrorQueueAddress>Error</ErrorQueueAddress> | |
| <MaxDeliveryAttempts>1</MaxDeliveryAttempts> | |
| <SecondLevelRetriesEnabled>false</SecondLevelRetriesEnabled> | |
| </SimpleRetryStrategy> | |
| </Options> | |
| <Logging Type="Sitecore.Messaging.SitecoreLoggerFactory,Sitecore.Messaging"/> | |
| </SitecoreHacker.London.Foundation.EmailCampaign.EDS.Messaging.Buses.EmailBounceMessageBus> | |
| </Rebus> | |
| </Messaging> | |
| </sitecore> | |
| </configuration> |
Summary
As I explained during the session, the biggest notion to take home is that while we can utilize this pattern in any solution, transport and how Sitecore has implemented Rebus could very well change in future releases, near and or far.
I hope that anyone attending enjoyed my silly little presentation. If even one person learned something, this was a successful session. If you did attend, THANK YOU for your participation!
Sitecore Hacker 
Leave a Reply