Timeouts are important for mitigating slowloris, yes. But after a number
of complaints and seeing that default timeouts are a sore point of
confusion, we're disabling them now. However, the code that sets
default timeouts remains intact; the defaults are just the zero value.
While Caddy aims to be secure by default, Caddy also aims to serve a
worldwide audience. Even my own internet here in Utah is poor at times,
with bad WiFi signal, causing some connections to take over 10s to
be established. Many use the Internet while commuting on slower
connection speeds. Latency across country borders is another concern.
As such, disabling default timeouts will serve a greater population of
users than enabling them, as slowloris is easy to mitigate and does
not seem to be reported often (I've only seen it once). It's also very
difficult sometimes to distinguish slowloris from genuine slow networks.
That decision is best left to the site owner for now.
* tls: Refactor TLS config innards with a few minor syntax changes
muststaple -> must_staple
"http2 off" -> "alpn" with list of ALPN values
* Fix typo
* Fix QUIC handler
* Inline struct field assignments
* Store name of authenticated user in basicauth for use by upstream middleware such as fastcgi and cgi.
* Use request context to transfer name of authorized user from basicauth to upstream middleware. Test retrieval of name from context.
* Remove development code that was inadvertently left in place
* Use keys of type httpserver.CtxKey to access Context values
* WIP: Implement HTTPS interception detection by Durumeric, et. al.
Special thanks to @FiloSottile for guidance with the custom listener.
* Add {{.IsMITM}} context action and {mitm} placeholder
* Improve MITM detection heuristics for Firefox and Edge
* Add tests for MITM detection heuristics
* Improve Safari heuristics for interception detection
* Read ClientHello during first Read() instead of during Accept()
As far as I can tell, reading the ClientHello during Accept() prevents
new connections from being accepted during the read. Since Read() should
be called in its own goroutine, this keeps Accept() non-blocking.
* Clean up MITM detection handler; make possible to close connection
* Use standard lib cipher suite values when possible
* Improve Edge heuristics and test cases
* Refactor MITM checking logic; add some debug statements for now
* Fix bug in MITM heuristic tests and actual heuristic code
* Fix gofmt
* Remove debug statements; preparing for merge
* fixing panic when root is symlink
checking root path is a symlink before os.Stat which panics
* fixing formatting
* adding test to verify symlink root path check
* fixing typo
* Add request placeholder support for querying request cookies.
This adds the ability to query the request cookies for placeholders
using the syntax "@cookiename".
For example, this would allow rewriting based on a cookie:
rewrite {
if @version is 'dev'
to /dev/index.html
}
* Switch cookie special char from @ to :
* Switch special char for cookies from : to ~
* Fix data race for max connection limiting in proxy directive.
The Conns and Unhealthy fields are updated concurrently across all active
requests. Because of this, they must use atomic operations for reads and
writes.
Prior to this change, Conns was incremented atomically, but read unsafely.
Unhealthly was updated & read unsafely. The new test
TestReverseProxyMaxConnLimit exposes this race when run with -race.
Switching to atomic operations makes the race detector happy.
* oops, remove leftover dead code.
* Use new subdirectives and flatten rolling config
* Set default rotate config
* Set default rolling config (hopefully) errwhere
* Make private
* Flatten errors directive and remove c.IncrNest()
* Don't skip first error log roller subdirective we see
* Remove hadBlock
* Try lumberjack import
* Unname import
The standard lib pprof library doesn't set its own Content-Type header
properly. If pprof is used with gzip, the index endpoint will be
interpreted as a .gz file; so we force its hand and set the header.
* proxy: use a new context for the outgoing request
fix issue #1345
Signed-off-by: Tw <tw19881113@gmail.com>
* proxy: add test for canceling the request
Signed-off-by: Tw <tw19881113@gmail.com>
* Generate meta elements from useful front matters.
Limited to the default template and specific elements.
* Rerun gofmt
* Add "keywords" and remove "language" to/from the list of meta tags.
* Add a simple positive list test for the meta tag generation.
* Move the meta tag list to a var at the begin of the file.
Seperate the Meta tags from the other front matters:
- Don't override user settings with name `meta`
- Cleaner Code.
* Remove the uneccessary `[:]` in the []Bytes to String casting.
@mholt was right ;)
* One minor refinement. Combining two statements.
Because of this commit(6e36811c37399d60cbce587b7c48e611009c5aec) on go tip,
it will probe the request's body to determine whether to use chunked transfer
encoding which trailers depend on it.
So we just offer a non empty body to make trailers work.
fix issue #1359
Signed-off-by: Tw <tw19881113@gmail.com>
* Add {whenISO} to record timestamp in ISO 8601 format in UTC.
ISO 8601 is the standard time format and is easy to parse.
This change assumes users desiring ISO 8016 generally prefer UTC for simplicity.
This results in {whenISO} to be significantly shorter than {when}:
{when} = "02/Jan/2006:15:04:05 +0000"
{whenISO} = "2006-01-02T15:04:12Z"
Add unit test to verify both, as there was no unit test for {when}.
* Rename {whenISO} to {when_iso}
If only one upstream is defined we don't need to buffer the body.
Instead we directly stream the body to the upstream host,
which reduces memory usage as well as latency.
Furthermore this enables different kinds of HTTP streaming
applications like gRPC for instance.
If a site owner protects a path with basicauth, no need
to use the Authorization header elsewhere upstream, especially since it
contains credentials.
If this breaks anyone, it means they're double-dipping. It's usually
good practice to clear out credentials as soon as they're not needed
anymore. (Note that we only clear credentials after they're used,
they stay for any other reason.)
* Added path cleanup functions with masking to preserve certain patterns + unit tests, #1298
* Use custom PathClean function instead of path.Clean to apply masks to preserve protocol separator in the path
* Indentation corrected in the test data map to pass the lint
* Fixing ineffassign of a temporary string variable
* Improved variable naming and documentation
* Improved variable naming
* Added benchmarks and improved variable naming in tests
* Removed unnecessary value capture when iterating over a map for keys
* A typo correction
This issue was caused by connHijackerTransport trying to record HTTP
response headers by "hijacking" the Read() method of the plain net.Conn.
This does not simply work over TLS though since this will record the TLS
handshake and encrypted data instead of the actual content.
This commit fixes the problem by providing an alternative transport.DialTLS
which correctly hijacks the overlying tls.Conn instead.
* Fixed HTTP/2 support for the proxy middleware
http.Transport instances whose TLSClientConfig, Dial, or DialTLS field
is non-nil will be configured without HTTP/2 support by default.
This commit adds the proper calls to http2.ConfigureTransport()
everywhere a http.Transport is created and thus fixes HTTP/2 in the
proxy middleware whenever insecure_skip_verify or keepalive is provided.
* Added HTTP/2 support check to TestReverseProxyInsecureSkipVerify
* Feature #1282 - Support pre-gzipped files
* Fix broken test cases
* Support brotli encoding as well
* Fix for #1276 - support integers and floats as metadata in markdown (#1278)
* Fix for #1276
* Use strconv.Format
* Use map[string]interface{} as variables
* One more file
* Always run all tests before commit
* Get rid of DocFlags
* Fix syntax in caddy.conf
* Update to Go 1.7.4
* Add send_timeout property to fastcgi directive.
* Convert rwc field on FCGIClient from type io.ReadWriteCloser to net.Conn.
* Return HTTP 504 to the client when a timeout occurs.
* In Handler.ServeHTTP(), close the connection before returning an HTTP
502/504.
* Refactor tests and add coverage.
* Return HTTP 504 when FastCGI connect times out.
* test: add unit test for #1283 (#1288)
* After review fixes
* Limit the number of restarts with systemd
* Prevent fd leak
* Prevent fd leak
* Refactor loops
* gofmt
* Convert rwc field on FCGIClient from type io.ReadWriteCloser to net.Conn.
* Return HTTP 504 to the client when a timeout occurs.
* In Handler.ServeHTTP(), close the connection before returning an HTTP
502/504.
* Refactor tests and add coverage.
* Fix for #1276
* Use strconv.Format
* Use map[string]interface{} as variables
* One more file
* Always run all tests before commit
* Get rid of DocFlags
By setting the read deadline in streamReader.Read(), the deadline was
extended by the read timeout on each subsequent call. To avoid this, the
deadline is set in FCGIClient.Request(), before the first read occurs.
See #1094.
* Filter empty headers
Some web servers (e.g. Jetty 9.3) don’t like HTTP headers with empty values. This commit filters header replacements with zero length.
* Extend tests to verify removal of empty headers
* Handle add-header case
* Change - Use short variable assignment
* Make fastcgi load balanceable too
* Address one more corner case - invalid configuration fastcgi /
* After review fixes
* Simplify conditions
* Error message
* New fastcgi syntax
* golint will be happy
* Change syntax
* Add Files action to template context. (#1198)
* Fixes to testFiles().
- Set os.ModePerm on directories created during test.
- Use filepath.Join() to create directory path.
- Use Fatalf, not Fatal.
* Make additional fixes to test cases.
* Fix test cases to use correct path format.
Dir.Open() in net/http requires '/'-separated paths while
filepath.Join() may produce paths with different separator.
* Remove directory created by test at end of loop.
* Close the FileSystem before returning.
* Initialize names slice to the number of entries.
Also, do not call os.RemoveAll() unless the path to the directory
is a valid one.
* Very simple fix for #1153
* Prevent Caddy-Rewrite-Original-URI being added as an HTTP ENV variable passed to FastCGI
part of fix for #1153
* Changes to Markdown to fix travis CI build.
#1955.2
* Revert "Changes to Markdown to fix travis CI build."
This reverts commit 4a01888839.
* fail fast and fmt changes
* Create test for existance of Caddy-Rewrite-Original-URI header value #1153
* updated test comment
* const moved outside function so available to tests
* Remove the eager check in the browse middleware, whether the root directory exists.
Caddy will start and throw a 404-error until the directory will be created.
* Add the complimentary test.
- Tests the startup of the browse middleware if the site root is inexistent and browse is pointing to the site root.
* Some minor stylistic tweaks.
* keep fastcgi connection open
* poor mans serialisation to make up for the lack of demuxing
* pointing includes to echse's repo
* Revert "pointing includes to echse's repo"
This reverts commit 281daad8d4.
* switch for persistent fcgi connections on/off added
* fixing ineffectual assignments
* camel case instead of _
* only activate persistent sockets on windows (and some naming conventions/cleanup)
* gitfm import sorting
* Revert "fixing ineffectual assignments"
This reverts commit 79760344e7.
# Conflicts:
# caddyhttp/staticfiles/fileserver.go
* added another mutex and deleting map entries. thx to mholts QA comments!
* thinking about it, this RW lock was not a good idea here
* thread safety
* I keep learning about mutexs in go
* some cosmetics
* adding persistant fastcgi connections switch to directive
* Support for configurable connection pool.
* ensure positive integer pool size config
* abisofts pool fix + nicer logging for the fastcgi_test
* abisoft wants to have dialer comparison in _test instead of next to struct
* Do not put dead connections back into pool
* Fix fastcgi header error
* Do not put dead connections back into pool
* some code style improvements from the discussion in https://github.com/mholt/caddy/pull/1134
* abisofts naming convention
* keep fastcgi connection open
* poor mans serialisation to make up for the lack of demuxing
* pointing includes to echse's repo
* Revert "pointing includes to echse's repo"
This reverts commit 281daad8d4.
* switch for persistent fcgi connections on/off added
* fixing ineffectual assignments
* camel case instead of _
* only activate persistent sockets on windows (and some naming conventions/cleanup)
* gitfm import sorting
* Revert "fixing ineffectual assignments"
This reverts commit 79760344e7.
# Conflicts:
# caddyhttp/staticfiles/fileserver.go
* added another mutex and deleting map entries. thx to mholts QA comments!
* thinking about it, this RW lock was not a good idea here
* thread safety
* I keep learning about mutexs in go
* some cosmetics
This function should not be used outside of development. It destroys the
absolute ordering and guarantees of correctness. Multiple uses of it
may work fine, but maybe not if they overlap, causing non-deterministic
builds which is bad. However, this can be convenient when developing
a plugin by calling it from an init() function, since you don't have
to modify the Caddy source code just to try your plugin.
Removing quic protocol headers from being persisted during proxy requests.
Not removing them could lead to the client attempting to connect to the wrong port.
This makes the quic headers consistent with other protocol headers.
* * Added support for environment variables to 'templates' module.
* Fixed flaw in test caused by environment variable ordering during testing on CI.
* Updated some local variables to camel-case.
* Reverted changes to replacer as environment variables are processed elsewhere.
* Removed PrintEnv functionality in favour of documenting using template range.
* Fix for stripping of 'Content-Disposition' and other headers from 'X-Accel-Redirect' redirect scripts.
* Added test case for header manipulation of redirect response.
First, great job on the 0.9 release! It seems caddy's path lead into a bright future. Thanks also for including the locale plugin.
Trying it, I've figured out, that there might be a problem with the order of the directives. In the typical use case, the result of the locale detection might be used in the `rewrite` and `log` plugin. If I'm not mistaken, it makes sense to put the `locale` directive before those.
* Allow for UDP servers
Extend the Server interface with ServePacket and ListenPacket - this is
in the same vein as the net package.
Plumb the packetconn through the start and restart phases.
Rename RestartPair to RestartTriple as it now also contains a Packet.
Not that these can now be nil, so we need to check for that when
restarting.
* Update the documentation
If we listen on 127.0.0.1:80 for `localhost` but :80 for everything else,
then a hostname in the hosts file that resolves to 127.0.0.1 will be
served on :80 (unless the bind directive is used) but the OS will use
the socket listening at 127.0.0.1:80, thus giving a "No such site" error
even though the site is there, but it's on the other listener at :80.
Two ways to fix this: 1) Leave as-is and require the user to set "bind
127.0.0.1" in their Caddyfile for all sites that are resolved in the
hosts file, or 2) Take out this special case and let localhost sites
listen on :80 (unless the user changes that with the bind directive, of
course). Having localhost bind to any interface is a little annoying
(unsettling?) but probably best in the long run.
https://forum.caddyserver.com/t/wildcard-virtual-domains-with-wildcard-roots/221/9?u=matt
OnStartup and OnShutdown callbacks now run as part of restarts, too.
The startup and shutdown directives only run their commands NOT as part
of restarts, as before. Some middleware that use OnStartup may need to
switch to OnFirstStartup and implement OnFinalShutdown to do any cleanup
as needed.
* checkpoint
* Added RequestMatcher interface. Extract 'if' condition into a RequestMatcher.
* Added tests for IfMatcher
* Minor refactors
* Refactors
* Use if_op
* conform with new 0.9 beta function changes.
- Server types no longer need to store their own contexts; they are
stored on the caddy.Instance, which means each context will be
properly GC'ed when the instance is stopped. Server types should use
type assertions to convert from caddy.Context to their concrete
context type when they need to use it.
- Pass the entire context into httpserver.GetConfig instead of only the
Key field.
- caddy.NewTestController now requires a server type string so it can
create a controller with the proper concrete context associated with
that server type.
Tests still need more attention so that we can test the proper creation
of startup functions, etc.
