Biggest change is no longer using standard library's tls.Config.getCertificate function to get a certificate during TLS handshake. Implemented our own cache which can be changed dynamically at runtime, even during TLS handshakes. As such, restarts are no longer required after certificate renewals or OCSP updates.
We also allow loading multiple certificates and keys per host, even by specifying a directory (tls got a new 'load' command for that).
Renamed the letsencrypt package to https in a gradual effort to become more generic; and https is more fitting for what the package does now.
There are still some known bugs, e.g. reloading where a new certificate is required but port 80 isn't currently listening, will cause the challenge to fail. There's still plenty of cleanup to do and tests to write. It is especially confusing right now how we enable "on-demand" TLS during setup and keep track of that. But this change should basically work so far.
Before, Caddy couldn't support graceful (zero-downtime) restarts when the reloaded Caddyfile had a host in it that was elligible for a LE certificate because the port was already in use. This commit makes it possible to do zero-downtime reloads and issue certificates for new hosts that need it. Supports only http-01 challenge at this time.
OCSP stapling is improved in that it updates before the expiration time when the validity window has shifted forward. See 30c949085c. Before it only used to update when the status changed.
This commit also sets the user agent for Let's Encrypt requests with a string containing "Caddy".
Added a -grace flag to customize graceful shutdown period, fixed bugs related to closing file descriptors (and dup'ed fds), improved healthcheck signaling to parent, fixed a race condition with the graceful listener, etc. These improvements mainly provide better support for frequent reloading or unusual use cases of Start and Stop after a Restart (POSIX systems). This forum thread was valuable help in debugging: https://forum.golangbridge.org/t/bind-address-already-in-use-even-after-listener-closed/1510?u=matt
Fixed pidfile writing problem where a pidfile would be written even if child failed, also cleaned up restarts a bit and fixed a few bugs, it's more robust now in case of failures and with logging.
Whether the original parent process or a child process as part of a restart, the pidfile will not be written/changed until that process has started successfully. It is written every time caddy.Start() succeeds (may be reundant, but that's probably okay).
Some programs (Node.js, supervisor, etc.) open a stdin pipe by default and don't use it, causing Caddy to block. It is their error, but we have to try to accommodate unfortunately. To fix this more universally, parent must explicitly set -conf to "stdin" to read from pipe.
Log file can also be stdout or stderr. Log output is disabled by default now, which makes it more feasible to add more log statements to trace program flow in debugging situations.
Fixed bug where manually specifying port 443 disabled TLS (whoops); otherHostHasScheme was the culprit, since it would return true even if it was the same config that had that scheme.
Also, an error at startup (if not a restart) is now fatal, rather than keeping a half-alive zombie server.
Merged config and app packages into one called caddy. Abstracted away caddy startup functionality making it easier to embed Caddy in any Go application and use it as a library. Graceful restart (should) now ensure child starts properly. Now piping a gob bundle to child process so that the child can match up inherited listeners to server address. Much cleanup still to do.
Lots of refinement still needed and runs only on POSIX systems. Windows will not get true graceful restarts (for now), but we will opt for very, very quick forceful restarts. Also, server configs are no longer put into a map; it is critical that they stay ordered so that they can be matched with their sockets in the child process after forking.
This implementation of graceful restarts is probably not perfect, but it is a good start. Lots of details to attend to now.
Turns out having each server block share a single server.Config during initialization when the Setup functions are being called was a bad idea. Sure, startup and shutdown functions were only executed once, but they had no idea what their hostname or port was. So here we revert to the old way of doing things where Setup may be called multiple times per server block (once per host associated with the block, to be precise), but the Setup functions now know their host and port since the config belongs to exactly one virtualHost. To have something happen just once per server block, use OncePerServerBlock, a new function available on each Controller.
Even if defined for multiple hosts. Startup or shutdown callbacks registered by any directive (startup, shutdown, markdown, git, log, etc.) will only run as many times as it appears in the Caddyfile, not repeated for each host that shares that server block. Fixing this involved refactoring three packages (yeesh) and we need to restore some tests that are no longer valid (that used to verify splitting a multiServerBlock into multiple serverBlocks).
