strong-tie/inbound-calls

1<h1 align="center">Fastify</h1>2 3## Recommendations4 5This document contains a set of recommendations when using Fastify.6 7- [Use A Reverse Proxy](#use-a-reverse-proxy)8  - [HAProxy](#haproxy)9  - [Nginx](#nginx)10- [Kubernetes](#kubernetes)11- [Capacity Planning For Production](#capacity)12- [Running Multiple Instances](#multiple)13 14## Use A Reverse Proxy15<a id="reverseproxy"></a>16 17Node.js is an early adopter of frameworks shipping with an easy-to-use web18server within the standard library. Previously, with languages like PHP or19Python, one would need either a web server with specific support for the20language or the ability to set up some sort of [CGI gateway][cgi] that works21with the language. With Node.js, one can write an application that _directly_22handles HTTP requests. As a result, the temptation is to write applications that23handle requests for multiple domains, listen on multiple ports (i.e. HTTP _and_24HTTPS), and then expose these applications directly to the Internet to handle25requests.26 27The Fastify team **strongly** considers this to be an anti-pattern and extremely28bad practice:29 301. It adds unnecessary complexity to the application by diluting its focus.312. It prevents [horizontal scalability][scale-horiz].32 33See [Why should I use a Reverse Proxy if Node.js is Production Ready?][why-use]34for a more thorough discussion of why one should opt to use a reverse proxy.35 36For a concrete example, consider the situation where:37 381. The app needs multiple instances to handle load.391. The app needs TLS termination.401. The app needs to redirect HTTP requests to HTTPS.411. The app needs to serve multiple domains.421. The app needs to serve static resources, e.g. jpeg files.43 44There are many reverse proxy solutions available, and your environment may45dictate the solution to use, e.g. AWS or GCP. Given the above, we could use46[HAProxy][haproxy] or [Nginx][nginx] to solve these requirements:47 48### HAProxy49 50```conf51# The global section defines base HAProxy (engine) instance configuration.52global53  log /dev/log syslog54  maxconn 409655  chroot /var/lib/haproxy56  user haproxy57  group haproxy58 59  # Set some baseline TLS options.60  tune.ssl.default-dh-param 204861  ssl-default-bind-options no-sslv3 no-tlsv10 no-tlsv1162  ssl-default-bind-ciphers ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:RSA+AESGCM:RSA+AES:!aNULL:!MD5:!DSS63  ssl-default-server-options no-sslv3 no-tlsv10 no-tlsv1164  ssl-default-server-ciphers ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:RSA+AESGCM:RSA+AES:!aNULL:!MD5:!DSS65 66# Each defaults section defines options that will apply to each subsequent67# subsection until another defaults section is encountered.68defaults69  log   global70  mode  http71  option        httplog72  option        dontlognull73  retries       374  option redispatch75  # The following option makes haproxy close connections to backend servers76  # instead of keeping them open. This can alleviate unexpected connection77  # reset errors in the Node process.78  option http-server-close79  maxconn       200080  timeout connect 500081  timeout client 5000082  timeout server 5000083 84  # Enable content compression for specific content types.85  compression algo gzip86  compression type text/html text/plain text/css application/javascript87 88# A "frontend" section defines a public listener, i.e. an "http server"89# as far as clients are concerned.90frontend proxy91  # The IP address here would be the _public_ IP address of the server.92  # Here, we use a private address as an example.93  bind 10.0.0.10:8094  # This redirect rule will redirect all traffic that is not TLS traffic95  # to the same incoming request URL on the HTTPS port.96  redirect scheme https code 308 if !{ ssl_fc }97  # Technically this use_backend directive is useless since we are simply98  # redirecting all traffic to this frontend to the HTTPS frontend. It is99  # merely included here for completeness sake.100  use_backend default-server101 102# This frontend defines our primary, TLS only, listener. It is here where103# we will define the TLS certificates to expose and how to direct incoming104# requests.105frontend proxy-ssl106  # The `/etc/haproxy/certs` directory in this example contains a set of107  # certificate PEM files that are named for the domains the certificates are108  # issued for. When HAProxy starts, it will read this directory, load all of109  # the certificates it finds here, and use SNI matching to apply the correct110  # certificate to the connection.111  bind 10.0.0.10:443 ssl crt /etc/haproxy/certs112 113  # Here we define rule pairs to handle static resources. Any incoming request114  # that has a path starting with `/static`, e.g.115  # `https://one.example.com/static/foo.jpeg`, will be redirected to the116  # static resources server.117  acl is_static path -i -m beg /static118  use_backend static-backend if is_static119 120  # Here we define rule pairs to direct requests to appropriate Node.js121  # servers based on the requested domain. The `acl` line is used to match122  # the incoming hostname and define a boolean indicating if it is a match.123  # The `use_backend` line is used to direct the traffic if the boolean is124  # true.125  acl example1 hdr_sub(Host) one.example.com126  use_backend example1-backend if example1127 128  acl example2 hdr_sub(Host) two.example.com129  use_backend example2-backend if example2130 131  # Finally, we have a fallback redirect if none of the requested hosts132  # match the above rules.133  default_backend default-server134 135# A "backend" is used to tell HAProxy where to request information for the136# proxied request. These sections are where we will define where our Node.js137# apps live and any other servers for things like static assets.138backend default-server139  # In this example we are defaulting unmatched domain requests to a single140  # backend server for all requests. Notice that the backend server does not141  # have to be serving TLS requests. This is called "TLS termination": the TLS142  # connection is "terminated" at the reverse proxy.143  # It is possible to also proxy to backend servers that are themselves serving144  # requests over TLS, but that is outside the scope of this example.145  server server1 10.10.10.2:80146 147# This backend configuration will serve requests for `https://one.example.com`148# by proxying requests to three backend servers in a round-robin manner.149backend example1-backend150  server example1-1 10.10.11.2:80151  server example1-2 10.10.11.2:80152  server example2-2 10.10.11.3:80153 154# This one serves requests for `https://two.example.com`155backend example2-backend156  server example2-1 10.10.12.2:80157  server example2-2 10.10.12.2:80158  server example2-3 10.10.12.3:80159 160# This backend handles the static resources requests.161backend static-backend162  server static-server1 10.10.9.2:80163```164 165[cgi]: https://en.wikipedia.org/wiki/Common_Gateway_Interface166[scale-horiz]: https://en.wikipedia.org/wiki/Scalability#Horizontal167[why-use]: https://web.archive.org/web/20190821102906/https://medium.com/intrinsic/why-should-i-use-a-reverse-proxy-if-node-js-is-production-ready-5a079408b2ca168[haproxy]: https://www.haproxy.org/169 170### Nginx171 172```nginx173# This upstream block groups 3 servers into one named backend fastify_app174# with 2 primary servers distributed via round-robin175# and one backup which is used when the first 2 are not reachable176# This also assumes your fastify servers are listening on port 80.177# more info: https://nginx.org/en/docs/http/ngx_http_upstream_module.html178upstream fastify_app {179  server 10.10.11.1:80;180  server 10.10.11.2:80;181  server 10.10.11.3:80 backup;182}183 184# This server block asks NGINX to respond with a redirect when185# an incoming request from port 80 (typically plain HTTP), to186# the same request URL but with HTTPS as protocol.187# This block is optional, and usually used if you are handling188# SSL termination in NGINX, like in the example here.189server {190  # default server is a special parameter to ask NGINX191  # to set this server block to the default for this address/port192  # which in this case is any address and port 80193  listen 80 default_server;194  listen [::]:80 default_server;195 196  # With a server_name directive you can also ask NGINX to197  # use this server block only with matching server name(s)198  # listen 80;199  # listen [::]:80;200  # server_name example.tld;201 202  # This matches all paths from the request and responds with203  # the redirect mentioned above.204  location / {205    return 301 https://$host$request_uri;206  }207}208 209# This server block asks NGINX to respond to requests from210# port 443 with SSL enabled and accept HTTP/2 connections.211# This is where the request is then proxied to the fastify_app212# server group via port 3000.213server {214  # This listen directive asks NGINX to accept requests215  # coming to any address, port 443, with SSL.216  listen 443 ssl default_server;217  listen [::]:443 ssl default_server;218 219  # With a server_name directive you can also ask NGINX to220  # use this server block only with matching server name(s)221  # listen 443 ssl;222  # listen [::]:443 ssl;223  # server_name example.tld;224 225  # Enable HTTP/2 support226  http2 on;227 228  # Your SSL/TLS certificate (chain) and secret key in the PEM format229  ssl_certificate /path/to/fullchain.pem;230  ssl_certificate_key /path/to/private.pem;231 232  # A generic best practice baseline for based233  # on https://ssl-config.mozilla.org/234  ssl_session_timeout 1d;235  ssl_session_cache shared:FastifyApp:10m;236  ssl_session_tickets off;237 238  # This tells NGINX to only accept TLS 1.3, which should be fine239  # with most modern browsers including IE 11 with certain updates.240  # If you want to support older browsers you might need to add241  # additional fallback protocols.242  ssl_protocols TLSv1.3;243  ssl_prefer_server_ciphers off;244 245  # This adds a header that tells browsers to only ever use HTTPS246  # with this server.247  add_header Strict-Transport-Security "max-age=63072000" always;248 249  # The following directives are only necessary if you want to250  # enable OCSP Stapling.251  ssl_stapling on;252  ssl_stapling_verify on;253  ssl_trusted_certificate /path/to/chain.pem;254 255  # Custom nameserver to resolve upstream server names256  # resolver 127.0.0.1;257 258  # This section matches all paths and proxies it to the backend server259  # group specified above. Note the additional headers that forward260  # information about the original request. You might want to set261  # trustProxy to the address of your NGINX server so the X-Forwarded262  # fields are used by fastify.263  location / {264    # more info: https://nginx.org/en/docs/http/ngx_http_proxy_module.html265    proxy_http_version 1.1;266    proxy_cache_bypass $http_upgrade;267    proxy_set_header Upgrade $http_upgrade;268    proxy_set_header Connection 'upgrade';269    proxy_set_header Host $host;270    proxy_set_header X-Real-IP $remote_addr;271    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;272    proxy_set_header X-Forwarded-Proto $scheme;273 274    # This is the directive that proxies requests to the specified server.275    # If you are using an upstream group, then you do not need to specify a port.276    # If you are directly proxying to a server e.g.277    # proxy_pass http://127.0.0.1:3000 then specify a port.278    proxy_pass http://fastify_app;279  }280}281```282 283[nginx]: https://nginx.org/284 285## Kubernetes286<a id="kubernetes"></a>287 288The `readinessProbe` uses [(by289default](https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/#configure-probes))290the pod IP as the hostname. Fastify listens on `127.0.0.1` by default. The probe291will not be able to reach the application in this case. To make it work,292the application must listen on `0.0.0.0` or specify a custom hostname in293the `readinessProbe.httpGet` spec, as per the following example:294 295```yaml296readinessProbe:297    httpGet:298        path: /health299        port: 4000300    initialDelaySeconds: 30301    periodSeconds: 30302    timeoutSeconds: 3303    successThreshold: 1304    failureThreshold: 5305```306 307## Capacity Planning For Production308<a id="capacity"></a>309 310In order to rightsize the production environment for your Fastify application,311it is highly recommended that you perform your own measurements against312different configurations of the environment, which may313use real CPU cores, virtual CPU cores (vCPU), or even fractional314vCPU cores. We will use the term vCPU throughout this315recommendation to represent any CPU type.316 317Tools such as [k6](https://github.com/grafana/k6)318or [autocannon](https://github.com/mcollina/autocannon) can be used for319conducting the necessary performance tests.320 321That said, you may also consider the following as a rule of thumb:322 323* To have the lowest possible latency, 2 vCPU are recommended per app324instance (e.g., a k8s pod). The second vCPU will mostly be used by the325garbage collector (GC) and libuv threadpool. This will minimize the latency326for your users, as well as the memory usage, as the GC will be run more327frequently. Also, the main thread won't have to stop to let the GC run.328 329* To optimize for throughput (handling the largest possible amount of330requests per second per vCPU available), consider using a smaller amount of vCPUs331per app instance. It is totally fine to run Node.js applications with 1 vCPU.332 333* You may experiment with an even smaller amount of vCPU, which may provide334even better throughput in certain use-cases. There are reports of API gateway335solutions working well with 100m-200m vCPU in Kubernetes.336 337See [Node's Event Loop From the Inside Out ](https://www.youtube.com/watch?v=P9csgxBgaZ8)338to understand the workings of Node.js in greater detail and make a339better determination about what your specific application needs.340 341## Running Multiple Instances342<a id="multiple"></a>343 344There are several use-cases where running multiple Fastify345apps on the same server might be considered. A common example346would be exposing metrics endpoints on a separate port,347to prevent public access, when using a reverse proxy or an ingress348firewall is not an option.349 350It is perfectly fine to spin up several Fastify instances within the same351Node.js process and run them concurrently, even in high load systems.352Each Fastify instance only generates as much load as the traffic it receives,353plus the memory used for that Fastify instance.354