Skip to main content

Circuit Breaker


Overview

The Circuit Breaker module protects your upstream applications by rejecting traffic to your endpoints when the upstreams become overloaded, allowing them time to recover to a steady operational state.

When your upstream application's 5XX responses exceed a percentage threshold of the total, the circuit breaker 'opens' or 'breaks' the circuit. When the circuit is open, the ngrok edge rejects requests to your endpoint with a 503 error response and does not forward them through to your upstream server.

When the circuit is open, ngrok will allow a small set of requests through to the upstream server to determine whether it has recovered. If those requests are successful, it will close the circuit permitting traffic to flow to your upstream application once again.

Quickstart

Agent CLI

ngrok http 80 --circuit-breaker 0.5

Agent Configuration File

tunnels:
example:
proto: http
addr: 80
circuit_breaker: 0.5

SSH

ssh -R 443:localhost:80 connect.ngrok-agent.com http --circuit-breaker 0.5

Go SDK

import (
"context"
"net"

"golang.ngrok.com/ngrok"
"golang.ngrok.com/ngrok/config"
)

func listenCircuitBreaker(ctx context.Context) net.Listener {
listener, _ := ngrok.Listen(ctx,
config.HTTPEndpoint(
config.WithCircuitBreaker(0.5),
),
ngrok.WithAuthtokenFromEnv(),
)

return listener
}

Rust SDK

use ngrok::prelude::*;

async fn start_tunnel() -> anyhow::Result<impl Tunnel> {
let sess = ngrok::Session::builder()
.authtoken_from_env()
.connect()
.await?;

let tun = sess
.http_endpoint()
.circuit_breaker(0.5)
.listen()
.await?;

println!("Listening on URL: {:?}", tun.url());

Ok(tun)
}

Kubernetes Ingress Controller

kind: NgrokModuleSet
apiVersion: ingress.k8s.ngrok.com/v1alpha1
metadata:
name: ngrok-module-set
modules:
circuit_breaker:
errorThresholdPercentage: "0.50"
trippedDuration: 10s
rollingWindow: 10s
numBuckets: 10
volumeThreshold: 10
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: example-ingress
annotations:
k8s.ngrok.com/modules: ngrok-module-set
spec:
ingressClassName: ngrok
rules:
- host: your-domain.ngrok.app
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: example-service
port:
number: 80

Edges

Circuit Breaker is a supported Edge module which can be applied to routes.

Behavior

The Circuit Breaker module is an implementation of Netflix's Hystrix circuit breaker specification.

If the upstream server responds with more than the threshold percentage of requests with 5XX status codes, the circuit breaker preemptively reject all subsequent requests at the ngrok edge with a 503 until the upstream server's error rate drops below the threshold percentage.

Circuit breaker state is tracked on each ngrok edge server individually. There are many ngrok edge servers which means that your upstream server may observe requests even after you would expect the circuit breaker to open. All of ngrok's edge servers will eventually open their circuits to protect an failing upstream application but the behavior you observe may not exactly match the parameters you've set because circuit breaker state is tracked individually on each of ngrok's edge servers.

Reference

Configuration

The Agent and Agent SDKs do not support configuration of some parameters of the Circuit Breaker.

ParameterDefaultDescription
Error ThresholdThe threshold percentage of upstream requests that must fail before the circuit is opened expressed a decimal value between 0 and 1. ngrok defines any HTTP response with a status code greater than or equal to 500 as an error response that counts toward the circuit breaker threshold.
Tripped Duration10The number of seconds to reject requests, once the circuit is opened, before rechecking if the circuit should again be closed.
Rolling Window10The window of time we keep metrics for the circuit breaker, the error threshold only considers successes and errors that fall within this window.
Number of Buckets10The number of discrete time intervals the rolling window duration is divided into. Along with the rolling window duration, this defines the granularity at which requests expire out of the rolling window. Max 128.
Volume Threshold20The minimum number of requests required in a rolling window that will trip the circuit.

Upstream Headers

This module does not add any upstream headers.

Errors

CodeHTTP StatusError
ERR_NGROK_3202503This error is returned if the circuit breaker is open because the upstream service is failing.

Events

When the Circuit Breaker module is enabled, it populates the following fields in http_request_complete.v0 events.

Fields
circuit_breaker.decision

Limits

This module is available on all plans.

Try it out

This short Go program below that demonstrates Circuit Breaker behavior. Create the file example.go with the Go code below and then run the following commands.

export NGROK_AUTHTOKEN="<your authtoken>"
go mod init example.com/ngrok-circuit-breaker
go get golang.ngrok.com/ngrok
go run example.go
example.go
package main

import (
"context"
"fmt"
"log"
"net"
"net/http"
"net/url"
"os"

"golang.ngrok.com/ngrok"
"golang.ngrok.com/ngrok/config"
)

func main() {
l, err := ngrok.Listen(context.Background(),
config.HTTPEndpoint(
config.WithCircuitBreaker(0.1),
),
ngrok.WithAuthtokenFromEnv(),
)
if err != nil {
log.Fatal(err)
}
fmt.Println("Running at", l.URL())
go makeRequests(l.URL())
http.Serve(l, http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/500" {
w.WriteHeader(500)
fmt.Fprintln(w, "Hello error!")
} else {
w.WriteHeader(200)
fmt.Fprintln(w, "Hello world!")
}
}))
}

func makeRequests(appURL string) {
// make sure we always dial the same IP addresss for testing purposes because
// circuit breaker state is applied on each ngrok edge server individually
u, _ := url.Parse(appURL)
addrs, err := net.LookupHost(u.Host)
if err != nil {
log.Fatal(err)
}
httpClient := http.Client{Transport: &http.Transport{
Dial: func(network, _ string) (net.Conn, error) {
return net.Dial(network, addrs[0]+":443")
},
}}

// make requests that return a 500 until the circuit opens
for {
resp, err := httpClient.Get(appURL + "/500")
if err != nil {
log.Fatal(err)
}
fmt.Printf("Status Code %d\n", resp.StatusCode)
if resp.StatusCode == 503 {
fmt.Println("Circuit opened")
break
}
}

// make requests that will eventually return a 200 which will close the circuit
for {
resp, err := httpClient.Get(appURL)
if err != nil {
log.Fatal(err)
}
fmt.Printf("Status Code %d\n", resp.StatusCode)
if resp.StatusCode != 503 {
fmt.Println("Circuit closed")
os.Exit(0)
}
}
}