* caddytls: Change clustering to be a plugin to the caddytls package
Should resolve the failure in
https://github.com/coredns/coredns/pull/2541.
This change is breaking to clustering plugin developers (not Caddy
users), but logical, since only the caddytls package uses CertMagic
directly (the httpserver package also uses it, but only because it also
uses the caddytls plugin); and it is early enough that no clustering
plugins really exist yet.
This will also require a change of devportal
so that it looks for a different registration function, which has moved
to the caddytls package.
* Remove unused variable
* caddyhttp: Fix test (adjust plugin counting)
* ummmm, remove extra line break
somehow VS Code didn't fmt on save... weird.
* caddy: Purge event hooks after USR1 reload
* caddy: Remove event hook purge logging
* caddy: Remove deleteEventHook
* caddy: use old event hooks in case of an unsuccessful restart
* caddy: implement restoreEventHooks
Also introduce caddy.OnProcessExit which is a list of functions that
run before exiting the process cleanly; these do not count as shutdown
callbacks, so they do not return errors and must execute quickly.
- Expose the list of Caddy instances through caddy.Instances()
- Added arbitrary storage to caddy.Instance
- The cache of loaded certificates is no longer global; now scoped
per-instance, meaning upon reload (like SIGUSR1) the old cert cache
will be discarded entirely, whereas before, aggressively reloading
config that added and removed lots of sites would cause unnecessary
build-up in the cache over time.
- Key certificates in the cache by their SHA-256 hash instead of
by their names. This means certificates will not be duplicated in
memory (within each instance), making Caddy much more memory-efficient
for large-scale deployments with thousands of sites sharing certs.
- Perform name-to-certificate lookups scoped per caddytls.Config instead
of a single global lookup. This prevents certificates from stepping on
each other when they overlap in their names.
- Do not allow TLS configurations keyed by the same hostname to be
different; this now throws an error.
- Updated relevant tests, with a stark awareness that more tests are
needed.
- Change the NewContext function signature to include an *Instance.
- Strongly recommend (basically require) use of caddytls.NewConfig()
to create a new *caddytls.Config, to ensure pointers to the instance
certificate cache are initialized properly.
- Update the TLS-SNI challenge solver (even though TLS-SNI is disabled
currently on the CA side). Store temporary challenge cert in instance
cache, but do so directly by the ACME challenge name, not the hash.
Modified the getCertificate function to check the cache directly for
a name match if one isn't found otherwise. This will allow any
caddytls.Config to be able to help solve a TLS-SNI challenge, with one
extra side-effect that might actually be kind of interesting (and
useless): clients could send a certificate's hash as the SNI and
Caddy would be able to serve that certificate for the handshake.
- Do not attempt to match a "default" (random) certificate when SNI
is present but unrecognized; return no certificate so a TLS alert
happens instead.
- Store an Instance in the list of instances even while the instance
is still starting up (this allows access to the cert cache for
performing renewals at startup, etc). Will be removed from list again
if instance startup fails.
- Laid groundwork for ACMEv2 and Let's Encrypt wildcard support.
Server type plugins will need to be updated slightly to accommodate
minor adjustments to their API (like passing in an Instance). This
commit includes the changes for the HTTP server.
Certain Caddyfile configurations might error out with this change, if
they configured different TLS settings for the same hostname.
This change trades some complexity for other complexity, but ultimately
this new complexity is more correct and robust than earlier logic.
Fixes#1991Fixes#1994Fixes#1303
Make ServerListeners public and add two helper methods to get access
to the address they listen on. This is useful for tests (among other
things)
Also make DefaultConfigFile a var so it can be overridden by server
types.
* Allow for UDP servers
Extend the Server interface with ServePacket and ListenPacket - this is
in the same vein as the net package.
Plumb the packetconn through the start and restart phases.
Rename RestartPair to RestartTriple as it now also contains a Packet.
Not that these can now be nil, so we need to check for that when
restarting.
* Update the documentation
- Server types no longer need to store their own contexts; they are
stored on the caddy.Instance, which means each context will be
properly GC'ed when the instance is stopped. Server types should use
type assertions to convert from caddy.Context to their concrete
context type when they need to use it.
- Pass the entire context into httpserver.GetConfig instead of only the
Key field.
- caddy.NewTestController now requires a server type string so it can
create a controller with the proper concrete context associated with
that server type.
Tests still need more attention so that we can test the proper creation
of startup functions, etc.
These changes span work from the last ~4 months in an effort to make
Caddy more extensible, reduce the coupling between its components, and
lay a more robust foundation of code going forward into 1.0. A bunch of
new features have been added, too, with even higher future potential.
The most significant design change is an overall inversion of
dependencies. Instead of the caddy package knowing about the server
and the notion of middleware and config, the caddy package exposes an
interface that other components plug into. This does introduce more
indirection when reading the code, but every piece is very modular and
pluggable. Even the HTTP server is pluggable.
The caddy package has been moved to the top level, and main has been
pushed into a subfolder called caddy. The actual logic of the main
file has been pushed even further into caddy/caddymain/run.go so that
custom builds of Caddy can be 'go get'able.
The HTTPS logic was surgically separated into two parts to divide the
TLS-specific code and the HTTPS-specific code. The caddytls package can
now be used by any type of server that needs TLS, not just HTTP. I also
added the ability to customize nearly every aspect of TLS at the site
level rather than all sites sharing the same TLS configuration. Not all
of this flexibility is exposed in the Caddyfile yet, but it may be in
the future. Caddy can also generate self-signed certificates in memory
for the convenience of a developer working on localhost who wants HTTPS.
And Caddy now supports the DNS challenge, assuming at least one DNS
provider is plugged in.
Dozens, if not hundreds, of other minor changes swept through the code
base as I literally started from an empty main function, copying over
functions or files as needed, then adjusting them to fit in the new
design. Most tests have been restored and adapted to the new API,
but more work is needed there.
A lot of what was "impossible" before is now possible, or can be made
possible with minimal disruption of the code. For example, it's fairly
easy to make plugins hook into another part of the code via callbacks.
Plugins can do more than just be directives; we now have plugins that
customize how the Caddyfile is loaded (useful when you need to get your
configuration from a remote store).
Site addresses no longer need be just a host and port. They can have a
path, allowing you to scope a configuration to a specific path. There is
no inheretance, however; each site configuration is distinct.
Thanks to amazing work by Lucas Clemente, this commit adds experimental
QUIC support. Turn it on using the -quic flag; your browser may have
to be configured to enable it.
Almost everything is here, but you will notice that most of the middle-
ware are missing. After those are transferred over, we'll be ready for
beta tests.
I'm very excited to get this out. Thanks for everyone's help and
patience these last few months. I hope you like it!!
