caddy/caddyhttp/gzip/gzip.go

169 lines
4.8 KiB
Go
Raw Normal View History

Rewrote Caddy from the ground up; initial commit of 0.9 branch These changes span work from the last ~4 months in an effort to make Caddy more extensible, reduce the coupling between its components, and lay a more robust foundation of code going forward into 1.0. A bunch of new features have been added, too, with even higher future potential. The most significant design change is an overall inversion of dependencies. Instead of the caddy package knowing about the server and the notion of middleware and config, the caddy package exposes an interface that other components plug into. This does introduce more indirection when reading the code, but every piece is very modular and pluggable. Even the HTTP server is pluggable. The caddy package has been moved to the top level, and main has been pushed into a subfolder called caddy. The actual logic of the main file has been pushed even further into caddy/caddymain/run.go so that custom builds of Caddy can be 'go get'able. The HTTPS logic was surgically separated into two parts to divide the TLS-specific code and the HTTPS-specific code. The caddytls package can now be used by any type of server that needs TLS, not just HTTP. I also added the ability to customize nearly every aspect of TLS at the site level rather than all sites sharing the same TLS configuration. Not all of this flexibility is exposed in the Caddyfile yet, but it may be in the future. Caddy can also generate self-signed certificates in memory for the convenience of a developer working on localhost who wants HTTPS. And Caddy now supports the DNS challenge, assuming at least one DNS provider is plugged in. Dozens, if not hundreds, of other minor changes swept through the code base as I literally started from an empty main function, copying over functions or files as needed, then adjusting them to fit in the new design. Most tests have been restored and adapted to the new API, but more work is needed there. A lot of what was "impossible" before is now possible, or can be made possible with minimal disruption of the code. For example, it's fairly easy to make plugins hook into another part of the code via callbacks. Plugins can do more than just be directives; we now have plugins that customize how the Caddyfile is loaded (useful when you need to get your configuration from a remote store). Site addresses no longer need be just a host and port. They can have a path, allowing you to scope a configuration to a specific path. There is no inheretance, however; each site configuration is distinct. Thanks to amazing work by Lucas Clemente, this commit adds experimental QUIC support. Turn it on using the -quic flag; your browser may have to be configured to enable it. Almost everything is here, but you will notice that most of the middle- ware are missing. After those are transferred over, we'll be ready for beta tests. I'm very excited to get this out. Thanks for everyone's help and patience these last few months. I hope you like it!!
2016-06-05 07:00:29 +08:00
// Package gzip provides a middleware layer that performs
// gzip compression on the response.
package gzip
import (
"bufio"
"compress/gzip"
"fmt"
"io"
"io/ioutil"
"net"
"net/http"
"strings"
Rewrote Caddy from the ground up; initial commit of 0.9 branch These changes span work from the last ~4 months in an effort to make Caddy more extensible, reduce the coupling between its components, and lay a more robust foundation of code going forward into 1.0. A bunch of new features have been added, too, with even higher future potential. The most significant design change is an overall inversion of dependencies. Instead of the caddy package knowing about the server and the notion of middleware and config, the caddy package exposes an interface that other components plug into. This does introduce more indirection when reading the code, but every piece is very modular and pluggable. Even the HTTP server is pluggable. The caddy package has been moved to the top level, and main has been pushed into a subfolder called caddy. The actual logic of the main file has been pushed even further into caddy/caddymain/run.go so that custom builds of Caddy can be 'go get'able. The HTTPS logic was surgically separated into two parts to divide the TLS-specific code and the HTTPS-specific code. The caddytls package can now be used by any type of server that needs TLS, not just HTTP. I also added the ability to customize nearly every aspect of TLS at the site level rather than all sites sharing the same TLS configuration. Not all of this flexibility is exposed in the Caddyfile yet, but it may be in the future. Caddy can also generate self-signed certificates in memory for the convenience of a developer working on localhost who wants HTTPS. And Caddy now supports the DNS challenge, assuming at least one DNS provider is plugged in. Dozens, if not hundreds, of other minor changes swept through the code base as I literally started from an empty main function, copying over functions or files as needed, then adjusting them to fit in the new design. Most tests have been restored and adapted to the new API, but more work is needed there. A lot of what was "impossible" before is now possible, or can be made possible with minimal disruption of the code. For example, it's fairly easy to make plugins hook into another part of the code via callbacks. Plugins can do more than just be directives; we now have plugins that customize how the Caddyfile is loaded (useful when you need to get your configuration from a remote store). Site addresses no longer need be just a host and port. They can have a path, allowing you to scope a configuration to a specific path. There is no inheretance, however; each site configuration is distinct. Thanks to amazing work by Lucas Clemente, this commit adds experimental QUIC support. Turn it on using the -quic flag; your browser may have to be configured to enable it. Almost everything is here, but you will notice that most of the middle- ware are missing. After those are transferred over, we'll be ready for beta tests. I'm very excited to get this out. Thanks for everyone's help and patience these last few months. I hope you like it!!
2016-06-05 07:00:29 +08:00
"github.com/mholt/caddy"
"github.com/mholt/caddy/caddyhttp/httpserver"
)
Rewrote Caddy from the ground up; initial commit of 0.9 branch These changes span work from the last ~4 months in an effort to make Caddy more extensible, reduce the coupling between its components, and lay a more robust foundation of code going forward into 1.0. A bunch of new features have been added, too, with even higher future potential. The most significant design change is an overall inversion of dependencies. Instead of the caddy package knowing about the server and the notion of middleware and config, the caddy package exposes an interface that other components plug into. This does introduce more indirection when reading the code, but every piece is very modular and pluggable. Even the HTTP server is pluggable. The caddy package has been moved to the top level, and main has been pushed into a subfolder called caddy. The actual logic of the main file has been pushed even further into caddy/caddymain/run.go so that custom builds of Caddy can be 'go get'able. The HTTPS logic was surgically separated into two parts to divide the TLS-specific code and the HTTPS-specific code. The caddytls package can now be used by any type of server that needs TLS, not just HTTP. I also added the ability to customize nearly every aspect of TLS at the site level rather than all sites sharing the same TLS configuration. Not all of this flexibility is exposed in the Caddyfile yet, but it may be in the future. Caddy can also generate self-signed certificates in memory for the convenience of a developer working on localhost who wants HTTPS. And Caddy now supports the DNS challenge, assuming at least one DNS provider is plugged in. Dozens, if not hundreds, of other minor changes swept through the code base as I literally started from an empty main function, copying over functions or files as needed, then adjusting them to fit in the new design. Most tests have been restored and adapted to the new API, but more work is needed there. A lot of what was "impossible" before is now possible, or can be made possible with minimal disruption of the code. For example, it's fairly easy to make plugins hook into another part of the code via callbacks. Plugins can do more than just be directives; we now have plugins that customize how the Caddyfile is loaded (useful when you need to get your configuration from a remote store). Site addresses no longer need be just a host and port. They can have a path, allowing you to scope a configuration to a specific path. There is no inheretance, however; each site configuration is distinct. Thanks to amazing work by Lucas Clemente, this commit adds experimental QUIC support. Turn it on using the -quic flag; your browser may have to be configured to enable it. Almost everything is here, but you will notice that most of the middle- ware are missing. After those are transferred over, we'll be ready for beta tests. I'm very excited to get this out. Thanks for everyone's help and patience these last few months. I hope you like it!!
2016-06-05 07:00:29 +08:00
func init() {
caddy.RegisterPlugin(caddy.Plugin{
Name: "gzip",
ServerType: "http",
Action: setup,
})
}
// Gzip is a middleware type which gzips HTTP responses. It is
// imperative that any handler which writes to a gzipped response
// specifies the Content-Type, otherwise some clients will assume
// application/x-gzip and try to download a file.
2015-03-21 08:11:54 +08:00
type Gzip struct {
Rewrote Caddy from the ground up; initial commit of 0.9 branch These changes span work from the last ~4 months in an effort to make Caddy more extensible, reduce the coupling between its components, and lay a more robust foundation of code going forward into 1.0. A bunch of new features have been added, too, with even higher future potential. The most significant design change is an overall inversion of dependencies. Instead of the caddy package knowing about the server and the notion of middleware and config, the caddy package exposes an interface that other components plug into. This does introduce more indirection when reading the code, but every piece is very modular and pluggable. Even the HTTP server is pluggable. The caddy package has been moved to the top level, and main has been pushed into a subfolder called caddy. The actual logic of the main file has been pushed even further into caddy/caddymain/run.go so that custom builds of Caddy can be 'go get'able. The HTTPS logic was surgically separated into two parts to divide the TLS-specific code and the HTTPS-specific code. The caddytls package can now be used by any type of server that needs TLS, not just HTTP. I also added the ability to customize nearly every aspect of TLS at the site level rather than all sites sharing the same TLS configuration. Not all of this flexibility is exposed in the Caddyfile yet, but it may be in the future. Caddy can also generate self-signed certificates in memory for the convenience of a developer working on localhost who wants HTTPS. And Caddy now supports the DNS challenge, assuming at least one DNS provider is plugged in. Dozens, if not hundreds, of other minor changes swept through the code base as I literally started from an empty main function, copying over functions or files as needed, then adjusting them to fit in the new design. Most tests have been restored and adapted to the new API, but more work is needed there. A lot of what was "impossible" before is now possible, or can be made possible with minimal disruption of the code. For example, it's fairly easy to make plugins hook into another part of the code via callbacks. Plugins can do more than just be directives; we now have plugins that customize how the Caddyfile is loaded (useful when you need to get your configuration from a remote store). Site addresses no longer need be just a host and port. They can have a path, allowing you to scope a configuration to a specific path. There is no inheretance, however; each site configuration is distinct. Thanks to amazing work by Lucas Clemente, this commit adds experimental QUIC support. Turn it on using the -quic flag; your browser may have to be configured to enable it. Almost everything is here, but you will notice that most of the middle- ware are missing. After those are transferred over, we'll be ready for beta tests. I'm very excited to get this out. Thanks for everyone's help and patience these last few months. I hope you like it!!
2016-06-05 07:00:29 +08:00
Next httpserver.Handler
Configs []Config
}
// Config holds the configuration for Gzip middleware
type Config struct {
2015-12-08 06:17:05 +08:00
RequestFilters []RequestFilter
ResponseFilters []ResponseFilter
Level int // Compression level
2015-03-21 08:11:54 +08:00
}
// ServeHTTP serves a gzipped response if the client supports it.
func (g Gzip) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error) {
if !strings.Contains(r.Header.Get("Accept-Encoding"), "gzip") {
return g.Next.ServeHTTP(w, r)
}
outer:
for _, c := range g.Configs {
2015-12-08 06:17:05 +08:00
// Check request filters to determine if gzipping is permitted for this request
for _, filter := range c.RequestFilters {
if !filter.ShouldCompress(r) {
continue outer
}
}
// Delete this header so gzipping is not repeated later in the chain
r.Header.Del("Accept-Encoding")
2015-12-08 19:01:24 +08:00
// gzipWriter modifies underlying writer at init,
// use a discard writer instead to leave ResponseWriter in
2015-12-08 19:01:24 +08:00
// original form.
gzipWriter, err := newWriter(c, ioutil.Discard)
if err != nil {
// should not happen
return http.StatusInternalServerError, err
}
defer gzipWriter.Close()
gz := &gzipResponseWriter{Writer: gzipWriter, ResponseWriter: w}
2015-12-08 06:17:05 +08:00
var rw http.ResponseWriter
// if no response filter is used
if len(c.ResponseFilters) == 0 {
// replace discard writer with ResponseWriter
2015-12-08 19:01:24 +08:00
gzipWriter.Reset(w)
2015-12-08 06:17:05 +08:00
rw = gz
} else {
// wrap gzip writer with ResponseFilterWriter
rw = NewResponseFilterWriter(c.ResponseFilters, gz)
}
// Any response in forward middleware will now be compressed
2015-12-08 06:17:05 +08:00
status, err := g.Next.ServeHTTP(rw, r)
// If there was an error that remained unhandled, we need
// to send something back before gzipWriter gets closed at
// the return of this method!
if status >= 400 {
Rewrote Caddy from the ground up; initial commit of 0.9 branch These changes span work from the last ~4 months in an effort to make Caddy more extensible, reduce the coupling between its components, and lay a more robust foundation of code going forward into 1.0. A bunch of new features have been added, too, with even higher future potential. The most significant design change is an overall inversion of dependencies. Instead of the caddy package knowing about the server and the notion of middleware and config, the caddy package exposes an interface that other components plug into. This does introduce more indirection when reading the code, but every piece is very modular and pluggable. Even the HTTP server is pluggable. The caddy package has been moved to the top level, and main has been pushed into a subfolder called caddy. The actual logic of the main file has been pushed even further into caddy/caddymain/run.go so that custom builds of Caddy can be 'go get'able. The HTTPS logic was surgically separated into two parts to divide the TLS-specific code and the HTTPS-specific code. The caddytls package can now be used by any type of server that needs TLS, not just HTTP. I also added the ability to customize nearly every aspect of TLS at the site level rather than all sites sharing the same TLS configuration. Not all of this flexibility is exposed in the Caddyfile yet, but it may be in the future. Caddy can also generate self-signed certificates in memory for the convenience of a developer working on localhost who wants HTTPS. And Caddy now supports the DNS challenge, assuming at least one DNS provider is plugged in. Dozens, if not hundreds, of other minor changes swept through the code base as I literally started from an empty main function, copying over functions or files as needed, then adjusting them to fit in the new design. Most tests have been restored and adapted to the new API, but more work is needed there. A lot of what was "impossible" before is now possible, or can be made possible with minimal disruption of the code. For example, it's fairly easy to make plugins hook into another part of the code via callbacks. Plugins can do more than just be directives; we now have plugins that customize how the Caddyfile is loaded (useful when you need to get your configuration from a remote store). Site addresses no longer need be just a host and port. They can have a path, allowing you to scope a configuration to a specific path. There is no inheretance, however; each site configuration is distinct. Thanks to amazing work by Lucas Clemente, this commit adds experimental QUIC support. Turn it on using the -quic flag; your browser may have to be configured to enable it. Almost everything is here, but you will notice that most of the middle- ware are missing. After those are transferred over, we'll be ready for beta tests. I'm very excited to get this out. Thanks for everyone's help and patience these last few months. I hope you like it!!
2016-06-05 07:00:29 +08:00
httpserver.DefaultErrorFunc(w, r, status)
return 0, err
}
return status, err
}
// no matching filter
return g.Next.ServeHTTP(w, r)
}
// newWriter create a new Gzip Writer based on the compression level.
// If the level is valid (i.e. between 1 and 9), it uses the level.
// Otherwise, it uses default compression level.
2015-12-08 19:01:24 +08:00
func newWriter(c Config, w io.Writer) (*gzip.Writer, error) {
if c.Level >= gzip.BestSpeed && c.Level <= gzip.BestCompression {
return gzip.NewWriterLevel(w, c.Level)
}
return gzip.NewWriter(w), nil
}
// gzipResponeWriter wraps the underlying Write method
// with a gzip.Writer to compress the output.
type gzipResponseWriter struct {
io.Writer
http.ResponseWriter
statusCodeWritten bool
}
// WriteHeader wraps the underlying WriteHeader method to prevent
// problems with conflicting headers from proxied backends. For
// example, a backend system that calculates Content-Length would
// be wrong because it doesn't know it's being gzipped.
func (w *gzipResponseWriter) WriteHeader(code int) {
w.Header().Del("Content-Length")
2015-12-08 19:01:24 +08:00
w.Header().Set("Content-Encoding", "gzip")
w.Header().Add("Vary", "Accept-Encoding")
w.ResponseWriter.WriteHeader(code)
w.statusCodeWritten = true
}
// Write wraps the underlying Write method to do compression.
func (w *gzipResponseWriter) Write(b []byte) (int, error) {
if w.Header().Get("Content-Type") == "" {
w.Header().Set("Content-Type", http.DetectContentType(b))
}
if !w.statusCodeWritten {
w.WriteHeader(http.StatusOK)
}
n, err := w.Writer.Write(b)
return n, err
}
// Hijack implements http.Hijacker. It simply wraps the underlying
// ResponseWriter's Hijack method if there is one, or returns an error.
func (w *gzipResponseWriter) Hijack() (net.Conn, *bufio.ReadWriter, error) {
if hj, ok := w.ResponseWriter.(http.Hijacker); ok {
return hj.Hijack()
}
return nil, nil, fmt.Errorf("not a Hijacker")
}
// Flush implements http.Flusher. It simply wraps the underlying
// ResponseWriter's Flush method if there is one, or panics.
func (w *gzipResponseWriter) Flush() {
if f, ok := w.ResponseWriter.(http.Flusher); ok {
f.Flush()
} else {
panic("not a Flusher") // should be recovered at the beginning of middleware stack
}
}
// CloseNotify implements http.CloseNotifier.
// It just inherits the underlying ResponseWriter's CloseNotify method.
func (w *gzipResponseWriter) CloseNotify() <-chan bool {
if cn, ok := w.ResponseWriter.(http.CloseNotifier); ok {
return cn.CloseNotify()
}
panic("not a CloseNotifier")
}