Squeezing all the juice from HTTP/2

HTTP/2Since we are well into the 21st century and HTTP/1.1 is no longer enough for most of us, let’s discuss some ways we can speed up our websites to run them faster, more securely, etc.

Many people have heard of HTTP/2 protocol, but not everyone knows how to use all its features.

 

Lets take a look at some 🙂

One of the goals of HTTP/2 is:

Decrease latency to improve page load speed in web browsers by considering:

Pretty great, right?

A quick note on compression:  In 1992, “gzip” compression was released. It is still used by all browsers, but are there alternatives after 25 years? The answer is “YES!” Brotli is supported by all modern browsers and has a better compression rate then “gzip”.

Now let’s get back to improving page load time in HTTP/2 protocol.

  • – Multiplexing
  • – HTTP/2 server push

Here are some illustrations to help show what happens:
http1.1 vs http2
http2 push

How it works:

You open an initial connection to get “page.html”. That html can have (or more likely will always have) some javascript and css files.

But servers can also send back to a browser “pushed” resources to be used by the browser after it parses “page.html”. Other resources like fonts, images, etc., could be loaded in the same TCP session!

Resources can be pushed if “page.html” has an additional header “Link” like:

link:</style.css>; rel=preload; as=style;
or for JS
link:</script.js>; rel=preload; as=script;
 

Pretty cool? NO! we can do more 🙂

What if we’re tired of TCP with its “Syn-Sent, Syn-Received, Established, Closed, and so on”?

Lets use UDP! Meet QUIC!

QUIC supports a set of multiplexed connections between two endpoints over UDP. And UDP is faster then TCP!

And how can we use all of that?

Lets start!

First meet Caddy!
It is a powerful HTTP server with QUIC support and HTTP/2 by default. And it also can HTTP/2 push. Yes, that’s all we need!

And now the fun part. Real examples!

I am using the following structure:

Caddy(as webserver in proxy mode)->Varnish(as cache server)->Nginx(as backend)->PHP app(as PHP app)

Caddy:

/etc/caddy/Caddyfile:

my.website:443 {
push #we want push to be enabled.
gzip { #we still need some compression for those who cant decode brotli
level 9
}
log /var/log/access.log
errors /var/log/error.log
proxy / 127.0.0.1:80 { #that will be our Varnish
header_upstream Host "my.website" #need to send some headers to it
header_upstream X-Forwarded-Proto "https"
}
tls /etc/caddy/mywebsite.crt /etc/caddy/mywebsite.key #that are our SSL cerfificate and Key files
root /var/www/html
}

Starting caddy as: /usr/bin/caddy -quic -agree=true -pidfile=/var/run/caddy.pid -log=/var/log/caddy.log -conf=/etc/caddy/Caddyfile

Seems not so complicated.

Varnish:

I’m using Varnish5, some of you can use Varnish4. As you wish.

/etc/varnish/default.vcl:

vcl 4.0;
import std;
import directors;
import header;
backend default {
    .host = "127.0.0.1";
    .port = "8080"; #that will be nginx
}
sub vcl_init {
}
sub vcl_hash {
    if(req.http.X-brotli == "true") {
            hash_data("brotli"); #need to cache brotli separately
    }else{hash_data(req.url); }
}
sub vcl_recv {
    if(req.http.Accept-Encoding ~ "br" && req.url !~
                "\.(jpg|png|gif|gz|mp3|mov|avi|mpg|mp4|swf|wmf)$") {
            set req.http.X-brotli = "true"; #need to send nginx request header .Accept-Encoding="br" from browser
    }
//you can add some rules depends on your app
    return (hash);
}
sub vcl_backend_fetch{
    if(bereq.http.X-brotli == "true") {
        set bereq.http.Accept-Encoding = "br"; #sending request header .Accept-Encoding="br" to backend
        unset bereq.http.X-brotli;
    }
}
sub vcl_backend_response {
if (!bereq.http.Accept-Encoding ~ "br"){
    if (bereq.url ~ "\.js$" || beresp.http.content-type ~ "text") {
        set beresp.do_gzip = true; #only use gzip if no Brotli support enabled
    }
}
//you can add some cache rules depends on your app
if (bereq.url ~ "(/$|\.php)"){
  header.append(beresp.http.Link, "</script.js>; rel=preload; as=script;");
  header.append(beresp.http.Link, "</style.css>; rel=preload; as=style;");
}
    return (deliver);
}
sub vcl_deliver {
    unset resp.http.X-Powered-By;
    unset resp.http.Server;
    unset resp.http.Via; #i would like to hide some headers from backend.
}

Starting varnish as: /usr/bin/varnishd -P /var/run/varnish.pid -a :80 -f /etc/varnish/default.vcl -T 0.0.0.0:6082 -p thread_pool_min=20 -p thread_pool_max=500 -p connect_timeout=600 -s malloc,1G

Next. Nginx:

Natively nginx does not support Brotli compression. But we can use ngx_brotli

After nginx recompile we can add following to our my.website.conf:

gzip on;
gzip_disable "msie6";
gzip_comp_level 9;
gzip_min_length 1100;
gzip_buffers 16 8k;
gzip_proxied any;
gzip_types
    text/plain
    text/css
    text/js
    text/xml
    text/javascript
    application/javascript
    application/x-javascript
    application/json
    application/xml
    application/xml+rss
    image/svg+xml;
brotli on;
brotli_static on;
brotli_types
    text/plain
    text/css
    text/js
    text/xml
    text/javascript
    application/javascript
    application/x-javascript
    application/json
    application/xml
    application/xml+rss
    image/svg+xml;
brotli_comp_level 11;

Yes we still need gzip, but it depends on the Accept-Encoding request header. Nginx will compress on-the-fly to gzip or brotli methods.

You can use any of app behind nginx. Doesn’t matter.

 

And whats next?

Have fun!

Lets see what we will get.
HTTP2 Push
 

Our real js and css files were pushed while “index” file is still downloading!

I got ~17% file size on brotli compression and ~30% speed time using HTTP/2 and HTTP/2 push option!

That is pretty cool!

Good luck in speeding up your websites!

Leave a Reply

Your email address will not be published. Required fields are marked *

By submitting this form, you accept the Mollom privacy policy.