Original feature request in forum:
https://forum.caddyserver.com/t/caddy-with-specific-hosts-but-on-demand-tls/1704?u=matt
Before, Caddy obtained certificates for every name it could at startup.
And it would only obtain certificates during the handshake for sites
defined with a hostname that didn't qualify at startup (like
"*.example.com" or ":443"). This made sense for most situations, and
helped ensure that certificates were obtained as early and reliably as
possible.
With this change, Caddy will NOT obtain certificates for hostnames it
knows at startup (even if they qualify) if OnDemand is enabled.
But I think this change generalizes well, because a user who specifies
max_certs is deliberately turning on On-Demand TLS, fully aware of
the consequences. It seems dubious to ignore that config when the user
deliberately put it there. We'll see how this goes.
* Fixed issue with {path} actually {uri}
* Test added for path rewrite
* add in uri_escaped
* added rewrite_uri and test
* fix broken test. Just checks for existance of rewrite header
* gitignore
* Use context to store uri value
* ignore .vscode
* tidy up, removal of comments and invalidated tests
* Remove commented out code.
* added comment as requested by lint
* fixed spelling mistake
* clarified code with variable name
* added context for uri and test
* added TODO comment to move consts
* add support for listener middleware
* add proxyprotocol directive
* make caddy.Listener interface required
* Remove tcpKeepAliveListener wrapper from Serve()
This is now done in the Listen() function, along with other potential middleware.
This commit removes _almost_ all instances of hard-coded ports 80 and
443 strings, and now allows the user to define what the HTTP and HTTPS
ports are by the -http-port and -https-ports flags.
(One instance of "80" is still hard-coded in tls.go because it cannot
import httpserver to get access to the HTTP port variable. I don't
suspect this will be a problem in practice, but one workaround would be
to define an exported variable in the caddytls package and let the
httpserver package set it as well as its own HTTPPort variable.)
The port numbers required by the ACME challenges HTTP-01 and TLS-SNI-01
are hard-coded into the spec as ports 80 and 443 for good reasons,
but the big question is whether they necessarily need to be the HTTP
and HTTPS ports. Although the answer is probably no, they chose those
ports for convenience and widest compatibility/deployability. So this
commit also assumes that the "HTTP port" is necessarily the same port
on which to serve the HTTP-01 challenge, and the "HTTPS port" is
necessarily the same one on which to serve the TLS-SNI-01 challenge. In
other words, changing the HTTP and HTTPS ports also changes the ports
the challenges will be served on.
If you change the HTTP and HTTPS ports, you are responsible for
configuring your system to forward ports 80 and 443 properly.
Closes#918 and closes#1293. Also related: #468.
* Use RequestURI when redirecting to canonical path.
Caddy may trim a request's URL path when it starts with the path that's
associated with the virtual host. This change uses the path from the request's
RequestURI when performing a redirect.
Fix issue #1327.
* Rename redirurl to redirURL.
* Redirect to the full URL.
The scheme and host from the virtual host's site configuration is used
in order to redirect to the full URL.
* Add comment and remove redundant check.
* Store the original URL path in request context.
By storing the original URL path as a value in the request context,
middlewares can access both it and the sanitized path. The default
default FileServer handler will use the original URL on redirects.
* Replace contextKey type with CtxKey.
In addition to moving the CtxKey definition to the caddy package, this
change updates the CtxKey references in the httpserver, fastcgi, and
basicauth packages.
* httpserver: Fix reference to CtxKey
Timeouts are important for mitigating slowloris, yes. But after a number
of complaints and seeing that default timeouts are a sore point of
confusion, we're disabling them now. However, the code that sets
default timeouts remains intact; the defaults are just the zero value.
While Caddy aims to be secure by default, Caddy also aims to serve a
worldwide audience. Even my own internet here in Utah is poor at times,
with bad WiFi signal, causing some connections to take over 10s to
be established. Many use the Internet while commuting on slower
connection speeds. Latency across country borders is another concern.
As such, disabling default timeouts will serve a greater population of
users than enabling them, as slowloris is easy to mitigate and does
not seem to be reported often (I've only seen it once). It's also very
difficult sometimes to distinguish slowloris from genuine slow networks.
That decision is best left to the site owner for now.
* tls: Refactor TLS config innards with a few minor syntax changes
muststaple -> must_staple
"http2 off" -> "alpn" with list of ALPN values
* Fix typo
* Fix QUIC handler
* Inline struct field assignments
* WIP: Implement HTTPS interception detection by Durumeric, et. al.
Special thanks to @FiloSottile for guidance with the custom listener.
* Add {{.IsMITM}} context action and {mitm} placeholder
* Improve MITM detection heuristics for Firefox and Edge
* Add tests for MITM detection heuristics
* Improve Safari heuristics for interception detection
* Read ClientHello during first Read() instead of during Accept()
As far as I can tell, reading the ClientHello during Accept() prevents
new connections from being accepted during the read. Since Read() should
be called in its own goroutine, this keeps Accept() non-blocking.
* Clean up MITM detection handler; make possible to close connection
* Use standard lib cipher suite values when possible
* Improve Edge heuristics and test cases
* Refactor MITM checking logic; add some debug statements for now
* Fix bug in MITM heuristic tests and actual heuristic code
* Fix gofmt
* Remove debug statements; preparing for merge
* Add request placeholder support for querying request cookies.
This adds the ability to query the request cookies for placeholders
using the syntax "@cookiename".
For example, this would allow rewriting based on a cookie:
rewrite {
if @version is 'dev'
to /dev/index.html
}
* Switch cookie special char from @ to :
* Switch special char for cookies from : to ~
* Use new subdirectives and flatten rolling config
* Set default rotate config
* Set default rolling config (hopefully) errwhere
* Make private
* Flatten errors directive and remove c.IncrNest()
* Don't skip first error log roller subdirective we see
* Remove hadBlock
* Try lumberjack import
* Unname import
* Add {whenISO} to record timestamp in ISO 8601 format in UTC.
ISO 8601 is the standard time format and is easy to parse.
This change assumes users desiring ISO 8016 generally prefer UTC for simplicity.
This results in {whenISO} to be significantly shorter than {when}:
{when} = "02/Jan/2006:15:04:05 +0000"
{whenISO} = "2006-01-02T15:04:12Z"
Add unit test to verify both, as there was no unit test for {when}.
* Rename {whenISO} to {when_iso}
* Added path cleanup functions with masking to preserve certain patterns + unit tests, #1298
* Use custom PathClean function instead of path.Clean to apply masks to preserve protocol separator in the path
* Indentation corrected in the test data map to pass the lint
* Fixing ineffassign of a temporary string variable
* Improved variable naming and documentation
* Improved variable naming
* Added benchmarks and improved variable naming in tests
* Removed unnecessary value capture when iterating over a map for keys
* A typo correction
* Add Files action to template context. (#1198)
* Fixes to testFiles().
- Set os.ModePerm on directories created during test.
- Use filepath.Join() to create directory path.
- Use Fatalf, not Fatal.
* Make additional fixes to test cases.
* Fix test cases to use correct path format.
Dir.Open() in net/http requires '/'-separated paths while
filepath.Join() may produce paths with different separator.
* Remove directory created by test at end of loop.
* Close the FileSystem before returning.
* Initialize names slice to the number of entries.
Also, do not call os.RemoveAll() unless the path to the directory
is a valid one.
This function should not be used outside of development. It destroys the
absolute ordering and guarantees of correctness. Multiple uses of it
may work fine, but maybe not if they overlap, causing non-deterministic
builds which is bad. However, this can be convenient when developing
a plugin by calling it from an init() function, since you don't have
to modify the Caddy source code just to try your plugin.
* * Added support for environment variables to 'templates' module.
* Fixed flaw in test caused by environment variable ordering during testing on CI.
* Updated some local variables to camel-case.
* Reverted changes to replacer as environment variables are processed elsewhere.
* Removed PrintEnv functionality in favour of documenting using template range.
First, great job on the 0.9 release! It seems caddy's path lead into a bright future. Thanks also for including the locale plugin.
Trying it, I've figured out, that there might be a problem with the order of the directives. In the typical use case, the result of the locale detection might be used in the `rewrite` and `log` plugin. If I'm not mistaken, it makes sense to put the `locale` directive before those.
* Allow for UDP servers
Extend the Server interface with ServePacket and ListenPacket - this is
in the same vein as the net package.
Plumb the packetconn through the start and restart phases.
Rename RestartPair to RestartTriple as it now also contains a Packet.
Not that these can now be nil, so we need to check for that when
restarting.
* Update the documentation
If we listen on 127.0.0.1:80 for `localhost` but :80 for everything else,
then a hostname in the hosts file that resolves to 127.0.0.1 will be
served on :80 (unless the bind directive is used) but the OS will use
the socket listening at 127.0.0.1:80, thus giving a "No such site" error
even though the site is there, but it's on the other listener at :80.
Two ways to fix this: 1) Leave as-is and require the user to set "bind
127.0.0.1" in their Caddyfile for all sites that are resolved in the
hosts file, or 2) Take out this special case and let localhost sites
listen on :80 (unless the user changes that with the bind directive, of
course). Having localhost bind to any interface is a little annoying
(unsettling?) but probably best in the long run.
https://forum.caddyserver.com/t/wildcard-virtual-domains-with-wildcard-roots/221/9?u=matt
* checkpoint
* Added RequestMatcher interface. Extract 'if' condition into a RequestMatcher.
* Added tests for IfMatcher
* Minor refactors
* Refactors
* Use if_op
* conform with new 0.9 beta function changes.
- Server types no longer need to store their own contexts; they are
stored on the caddy.Instance, which means each context will be
properly GC'ed when the instance is stopped. Server types should use
type assertions to convert from caddy.Context to their concrete
context type when they need to use it.
- Pass the entire context into httpserver.GetConfig instead of only the
Key field.
- caddy.NewTestController now requires a server type string so it can
create a controller with the proper concrete context associated with
that server type.
Tests still need more attention so that we can test the proper creation
of startup functions, etc.
Also we change the scheme of the site's address if TLS is enabled and
no other scheme is explicitly set; this makes it appear as "https" when
we print it; otherwise it would show "http" when TLS is turned on
implicitly, and that is confusing/incorrect.
These changes span work from the last ~4 months in an effort to make
Caddy more extensible, reduce the coupling between its components, and
lay a more robust foundation of code going forward into 1.0. A bunch of
new features have been added, too, with even higher future potential.
The most significant design change is an overall inversion of
dependencies. Instead of the caddy package knowing about the server
and the notion of middleware and config, the caddy package exposes an
interface that other components plug into. This does introduce more
indirection when reading the code, but every piece is very modular and
pluggable. Even the HTTP server is pluggable.
The caddy package has been moved to the top level, and main has been
pushed into a subfolder called caddy. The actual logic of the main
file has been pushed even further into caddy/caddymain/run.go so that
custom builds of Caddy can be 'go get'able.
The HTTPS logic was surgically separated into two parts to divide the
TLS-specific code and the HTTPS-specific code. The caddytls package can
now be used by any type of server that needs TLS, not just HTTP. I also
added the ability to customize nearly every aspect of TLS at the site
level rather than all sites sharing the same TLS configuration. Not all
of this flexibility is exposed in the Caddyfile yet, but it may be in
the future. Caddy can also generate self-signed certificates in memory
for the convenience of a developer working on localhost who wants HTTPS.
And Caddy now supports the DNS challenge, assuming at least one DNS
provider is plugged in.
Dozens, if not hundreds, of other minor changes swept through the code
base as I literally started from an empty main function, copying over
functions or files as needed, then adjusting them to fit in the new
design. Most tests have been restored and adapted to the new API,
but more work is needed there.
A lot of what was "impossible" before is now possible, or can be made
possible with minimal disruption of the code. For example, it's fairly
easy to make plugins hook into another part of the code via callbacks.
Plugins can do more than just be directives; we now have plugins that
customize how the Caddyfile is loaded (useful when you need to get your
configuration from a remote store).
Site addresses no longer need be just a host and port. They can have a
path, allowing you to scope a configuration to a specific path. There is
no inheretance, however; each site configuration is distinct.
Thanks to amazing work by Lucas Clemente, this commit adds experimental
QUIC support. Turn it on using the -quic flag; your browser may have
to be configured to enable it.
Almost everything is here, but you will notice that most of the middle-
ware are missing. After those are transferred over, we'll be ready for
beta tests.
I'm very excited to get this out. Thanks for everyone's help and
patience these last few months. I hope you like it!!
