Distributed Systems

Building NServiceBus Metrics with OpenTelemetry and System.Diagnostics.Metrics

The release of System.Diagnostics.DiagnosticSource version 6.0 a few months back brought something entirely new to the library - support for OpenTelemetry Metrics. Since this package releases out-of-band from the .NET 6 SDK, it also means you can use these new metrics APIs in any application targeting .NET

Azure

Local Development with Azure Service Bus

For teams new to Azure Service Bus, one of the first questions you have to answer is "how do I develop against this?" And it turns out the answer isn't that straightforward - because it's currently impossible to run Azure Service Bus outside of Azure. There's no install. There's no

Distributed Systems

OpenTelemetry 1.0 Extensions Released

With the release of OpenTelemetry tracing specification reaching 1.0, and the subsequent release of the 1.0 release of the core components of .NET, I've pushed updates to my OpenTelemetry packages for: * NServiceBus.Extensions.Diagnostics.OpenTelemetry * MongoDB.Driver.Core.Extensions.OpenTelemetry While those packages didn't really change much, one

Distributed Systems

Diagnostics and Instrumentation Packages Targeting Open Telemetry Beta for MongoDB and NServiceBus Published

That's...a long title. I've published release packages for System.Diagnostics support for NServiceBus and MongoDB: * NServiceBus.Extensions.Diagnostics * MongoDB.Driver.Core.Extensions.DiagnosticSources And published new beta packages targeting the beta release of OpenTelemetry: * NServiceBus.Extensions.Diagnostics.OpenTelemetry * MongoDB.Driver.Core.Extensions.OpenTelemetry The Diagnostics packages are separate from

Distributed Systems

Diagnostics and Instrumentation Packages for MongoDB and NServiceBus Published

As part of the end-to-end diagnostics and tracing blog series, I had an end goal of eventually publishing out NuGet packages for diagnostics (Activity and DiagnosticSource). I'm happy to announce that I've released 4 packages to NuGet: * NServiceBus.Extensions.Diagnostics * NServiceBus.Extensions.Diagnostics.OpenTelemetry * MongoDB.Driver.Core.Extensions.DiagnosticSources * MongoDB.

Microservices

Building End-to-End Diagnostics and Tracing: An Intro

Posts in this series: * An Intro * Trace Context * Diagnostic Events * OpenTelemetry Integration * Activity and Span Correlation * Visualization with Exporters * User-Defined Context with Correlation Context * ActivitySource and OpenTelemetry 1.0 Source Code As microservices introduced (forced) highly complex distributed systems into organizations, the tools required to operate these architectures needed to

Microservices

Life Beyond Distributed Transactions: An Apostate's Implementation - Conclusion

Posts in this series: * A Primer * Document Coordination * Document Example * Dispatching Example * Failures and Retries * Failure Recovery * Sagas * Relational Resources * Conclusion We started out with a common, but nearly always overlooked problem: how do we reliably coordinate activities between different transactional resources? The question we need to ask first is

Microservices

Life Beyond Distributed Transactions: An Apostate's Implementation - Relational Resources

Posts in this series: * A Primer * Document Coordination * Document Example * Dispatching Example * Failures and Retries * Failure Recovery * Sagas * Relational Resources * Conclusion Sample code from this series So far in this series we've mainly concerned ourselves with a single resource that can't support distributed (or multi-entity) transactions. While that is becoming

Microservices

Life Beyond Distributed Transactions: An Apostate's Implementation - Sagas

Posts in this series: * A Primer * Document Coordination * Document Example * Dispatching Example * Failures and Retries * Failure Recovery * Sagas * Relational Resources * Conclusion Sample code from this series So far in this series, we've looked at the ins and outs of moving beyond distributed transactions using persisted messages as a means of

Microservices

Life Beyond Distributed Transactions: An Apostate's Implementation - Dispatcher Failure Recovery

Posts in this series: * A Primer * Document Coordination * Document Example * Dispatching Example * Failures and Retries * Failure Recovery * Sagas * Relational Resources * Conclusion Sample code from this series In the last post, we looked at how we can recover from exceptions from inside our code handling messages. We perform some action in

Architecture

Life Beyond Distributed Transactions: An Apostate's Implementation - Failures and Retries

Posts in this series: * A Primer * Document Coordination * Document Example * Dispatching Example * Failures and Retries * Failure Recovery * Sagas * Relational Resources * Conclusion Sample code from this series In the last post, we looked at an example of dispatching document messages to other documents using a central dispatcher. Our example worked well

Microservices

Life Beyond Distributed Transactions: An Apostate's Implementation - Dispatching Example

Posts in this series: * A Primer * Document Coordination * Document Example * Dispatching Example * Failures and Retries * Failure Recovery * Sagas * Relational Resources * Conclusion Sample code from this series In the last post, we looked at refactoring our documents to use messaging to communicate changes. We're still missing something, however - the dispatcher:

Microservices

Life Beyond Distributed Transactions: An Apostate's Implementation - Document Example

Posts in this series: * A Primer * Document Coordination * Document Example * Dispatching Example * Failures and Retries * Failure Recovery * Sagas * Relational Resources * Conclusion Sample code from this series In the last post, I walked through the "happy path" scenario of coordinated communication/activities between multiple resources that otherwise can't participate in a

Distributed Systems

Life Beyond Distributed Transactions: An Apostate's Implementation - Document Coordination

Posts in this series: * A Primer * Document Coordination * Document Example * Dispatching Example * Failures and Retries * Failure Recovery * Sagas * Relational Resources * Conclusion Sample code from this series Quick note - I've updated this post to use the more accurate term "Document" than the less-accurate, DDD-specific term "Aggregate". In the images, it

Distributed Systems

Life Beyond Distributed Transactions: An Apostate's Implementation - A Primer

Posts in this series: * A Primer * Document Coordination * Document Example * Dispatching Example * Failures and Retries * Failure Recovery * Sagas * Relational Resources * Conclusion Sample code from this series For those working with SQL databases, working with transactions is more or less a given. The most we may need to worry about is:

Distributed Systems

Dealing With Optimistic Concurrency Control Collisions

Optimistic Concurrency Control (OCC) is a well-established solution for a rather old problem - handling two (or more) concurrent writes to a single object/resource/entity without losing writes. OCC works (typically) by including a timestamp as part of the record, and during a write, we read the timestamp: 1.

REST

Swagger, the REST Kryptonite

Swagger, a tool to help design, build, document, and consume RESTful APIs is ironically kryptonite for building actual RESTful APIs. The battle over the term "REST" is lost, where "RESTful" simply means "an API over HTTP" but these days is 99% of the time referring to "RPC over HTTP". In

Architecture

Refactoring Towards Resilience: Process Manager Solution

Other posts in this series: * A primer * Evaluating Stripe Options * Evaluating SendGrid Options * Evaluating RabbitMQ Options * Evaluating Coupling * Async Workflow Options * Process Manager Solution In the last post, we examined all of our coordination options as well as our process coupling to design a solution for our order processor. In

Architecture

Refactoring Towards Resilience: Async Workflow Options

Other posts in this series: * A primer * Evaluating Stripe Options * Evaluating SendGrid Options * Evaluating RabbitMQ Options * Evaluating Coupling * Async Workflow Options * Process Manager Solution In the last post, we looked at coupling options in our 3rd-party resources we use as part of "button-click" place order, and whether or not we

Architecture

Refactoring Towards Resilience: Evaluating Coupling

Other posts in this series: * A primer * Evaluating Stripe Options * Evaluating SendGrid Options * Evaluating RabbitMQ Options * Evaluating Coupling * Async Workflow Options * Process Manager Solution So far, we've been looking at our options on how to coordinate various services, using Hohpe as our guide: * Ignore * Retry * Undo * Coordinate These options, valid

Architecture

Refactoring Towards Resilience: Evaluating Stripe Options

Other posts in this series: * A primer * Evaluating Stripe Options * Evaluating SendGrid Options * Evaluating RabbitMQ Options * Evaluating Coupling * Async Workflow Options * Process Manager Solution In the last post, I looked at a common problem in web applications - accepting payments via 3rd party APIs (in this case, Stripe). We saw

Architecture

Refactoring Towards Resilience: A Primer

Other posts in this series: * A primer * Evaluating Stripe Options * Evaluating SendGrid Options * Evaluating RabbitMQ Options * Evaluating Coupling * Async Workflow Options * Process Manager Solution Recently, I sat down to help a team to put in some resiliency in a payment page. This payment page used Stripe as its payment gateway.