Commit Graph

302 Commits

Author SHA1 Message Date
Matthew Holt
754fe4f7b4
httpcaddyfile: Fix sorting of repeated directives
Fixes #5037
2022-09-13 13:43:21 -06:00
Francis Lavoie
8cc8f9fddd
httpcaddyfile: Add a couple more placeholder shortcuts (#5015)
This adds:
- `{file.*}` -> `{http.request.uri.path.file.*}`
- `{file_match.*}` -> `{http.matchers.file.*}`

This is a follow-up to #4993 which introduces the new URI file placeholders, and a shortcut for using `file` matcher output.

For example, where the `try_files` directive is a shortcut for this:

```
@try_files file <files...>
rewrite @try_files {http.matchers.file.relative}
```

It could instead be:
```
@try_files file <files...>
rewrite @try_files {file_match.relative}
```
2022-09-05 21:41:48 -06:00
Matt Holt
d3c3fa10bd
core: Refactor listeners; use SO_REUSEPORT on Unix (#4705)
* core: Refactor listeners; use SO_REUSEPORT on Unix

Just an experiment for now

* Fix lint by logging error

* TCP Keepalive configuration (#4865)

* initial attempt at TCP Keepalive configuration

* core: implement tcp-keepalive for linux

* move canSetKeepAlive interface

* Godoc for keepalive server parameter

* handle return values

* log keepalive errors

* Clean up after bad merge

* Merge in pluggable network types

From 1edc1a45e3

* Slight refactor, fix from recent merge conflict

Co-authored-by: Karmanyaah Malhotra <karmanyaah.gh@malhotra.cc>
2022-09-02 16:59:11 -06:00
Matthew Holt
73d4a8ba02 map: Coerce val to string, fix #4987
Also prevent infinite recursion, and enforce placeholder syntax.
2022-09-01 21:15:44 -06:00
Francis Lavoie
7d5108d132
httpcaddyfile: Add shortcut for expression matchers (#4976) 2022-09-01 23:12:37 -04:00
Francis Lavoie
68d8ac9802
httpcaddyfile: Add {cookie.*} placeholder shortcut (#5001) 2022-08-31 10:18:29 -06:00
Ben Burkert
b540f195b1
httpcaddyfile: Add ocsp_interval global option (#4980) 2022-08-24 11:22:56 -06:00
WilczyńskiT
c7772588bd
core: Change net.IP to netip.Addr; use netip.Prefix (#4966)
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2022-08-17 16:10:57 -06:00
Matt Holt
c79c08627d
caddyhttp: Enable HTTP/3 by default (#4707) 2022-08-15 12:01:58 -06:00
Matthew Holt
91ab0e6066
httpcaddyfile: redir with "html" emits 200, no Location (fix #4940)
The intent of "html" is to redirect browser clients only, or those which can evaluate JS and/or meta tags. So return HTTP 200 and no Location header. See #4940.
2022-08-09 11:12:09 -06:00
WilczyńskiT
2642bd72b7
Replace strings.Index usages with strings.Cut (#4930) 2022-08-04 11:17:35 -06:00
Matt Holt
1960a0dc11
httpserver: Configurable shutdown delay (#4906) 2022-08-03 11:04:51 -06:00
Francis Lavoie
141872ed80
chore: Bump up to Go 1.19, minimum 1.18 (#4925) 2022-08-02 16:39:09 -04:00
WingLim
1e0cdc54f8
core: Windows service integration (#4790)
Co-authored-by: Matthew Holt <mholt@users.noreply.github.com>
2022-07-29 14:06:54 -06:00
Matthew Holt
1e18afb5c8
httpcaddyfile: Detect ambiguous site definitions (fix #4635)
Previously, our "duplicate key in server block" logic was flawed because
it did not account for the site's bind address. We defer this check to
when the listener addresses have been assigned, but before we commit
a server block to its listener.

Also refined how network address parsing and joining works, which was
necessary for a less convoluted fix.
2022-07-25 17:28:20 -06:00
Matt Holt
660c59b6f3
admin: Implement /adapt endpoint (close #4465) (#4846) 2022-06-29 00:43:57 -04:00
Matthew Holt
886ba84baa
Fix #4822 and fix #4779
The fix for 4822 is the change at the top of the file, and
4779's fix is toward the bottom of the file.
2022-06-01 15:12:57 -06:00
Francis Lavoie
58970cae92
httpcaddyfile: Add {err.*} placeholder shortcut (#4798) 2022-05-24 10:06:46 -06:00
Francis Lavoie
b687d7b967
httpcaddyfile: Support multiple values for default_bind (#4774)
* httpcaddyfile: Support multiple values for `default_bind`

* Fix ordering of server blocks
2022-05-08 21:32:10 -04:00
Francis Lavoie
f7be0ee101
map: Prevent output destinations overlap with Caddyfile shorthands (#4657) 2022-05-06 10:25:31 -06:00
Francis Lavoie
f6900fcf53
reverseproxy: Support performing pre-check requests (#4739) 2022-05-06 10:50:26 -04:00
Francis Lavoie
af7321511c
httpcaddyfile: Fix duplicate access log when debug is on (#4746) 2022-04-28 12:16:25 -04:00
Francis Lavoie
a8bb4a665a
httpcaddyfile: Add {vars.*} placeholder shortcut, reverse vars sort order (#4726)
* httpcaddyfile: Add `{vars.*}` placeholder shortcut

I'm yoinking this from my https://github.com/caddyserver/caddy/pull/4657 PR because I think we should get this in ASAP for v2.5.0 along with the new `vars` directive.

* Sort vars by matchers in reverse
2022-04-25 10:47:12 -06:00
Francis Lavoie
3a1e0dbf47
httpcaddyfile: Deprecate paths in site addresses; use zap logs (#4728) 2022-04-25 10:12:10 -06:00
Matthew Holt
db62942d63
Make file modes consistent
No need to have executable bit on .go or .txt files
2022-04-21 15:06:55 -06:00
Matthew Holt
d06d0e79f8
go.mod: Upgrade CertMagic to v0.16.0
Includes several breaking changes; code base updated accordingly.

- Added lots of context arguments
- Use fs.ErrNotExist
- Rename ACMEManager -> ACMEIssuer; CertificateManager -> Manager
2022-03-25 11:28:54 -06:00
Francis Lavoie
a58f240d3e
httpcaddyfile: Fix #4640 (auto-HTTPS edgecase) (#4661)
Guh, this is complicated.

Fixes #4640

This also follows up on #4398 (reverting it) which made a change that technically worked, but was incorrect. It changed the condition in `hostsFromKeysNotHTTP` from `&&` to `||`, but then the function no longer did what its name said it would do, and it would return hosts even if they were marked with `http://`, if they used a non-HTTP port. That wasn't the intent of it. The test added in there was kept though, because it is a valid usecase.

The actual fix is to check _earlier_ whether all the addresses explicitly have `http://`, and if so we can short circuit and skip considering the rest.
2022-03-24 22:54:03 -06:00
Francis Lavoie
134b805644
caddyfile: Prevent bad block opening tokens (#4655)
* caddyfile: Prevent bad block opening tokens

* Clarifying comments
2022-03-23 12:34:13 -06:00
Matthew Holt
79cbe7bfd0
httpcaddyfile: Add 'vars' directive
See discussion in #4650
2022-03-22 10:47:21 -06:00
Francis Lavoie
c5fffb4ac2
caddyfile: Support for raw token values; improve map, expression (#4643)
* caddyfile: Support for raw token values, improve `map`, `expression`

* Applied code review comments

* Rename RawVal to ValRaw

Co-authored-by: Matthew Holt <mholt@users.noreply.github.com>
2022-03-18 15:08:23 -06:00
Francis Lavoie
4e9fbee1e2
ci: Build on Go 1.18, bump actions versions (#4637)
* ci: Build on Go 1.18, bump actions versions

* Revert linter version bump for now

* Try linter again
2022-03-15 22:09:19 +00:00
Francis Lavoie
a9c7e94a38
chore: Comment fixes (#4634) 2022-03-13 01:38:11 -05:00
Francis Lavoie
c7d6c4cbb9
reverseproxy: copy_response and copy_response_headers for handle_response routes (#4391)
* reverseproxy: New `copy_response` handler for `handle_response` routes

Followup to #4298 and #4388.

This adds a new `copy_response` handler which may only be used in `reverse_proxy`'s `handle_response` routes, which can be used to actually copy the proxy response downstream. 

Previously, if `handle_response` was used (with routes, not the status code mode), it was impossible to use the upstream's response body at all, because we would always close the body, expecting the routes to write a new body from scratch.

To implement this, I had to refactor `h.reverseProxy()` to move all the code that came after the `HandleResponse` loop into a new function. This new function `h.finalizeResponse()` takes care of preparing the response by removing extra headers, dealing with trailers, then copying the headers and body downstream.

Since basically what we want `copy_response` to do is invoke `h.finalizeResponse()` at a configurable point in time, we need to pass down the proxy handler, the response, and some other state via a new `req.WithContext(ctx)`. Wrapping a new context is pretty much the only way we have to jump a few layers in the HTTP middleware chain and let a handler pick up this information. Feels a bit dirty, but it works.

Also fixed a bug with the `http.reverse_proxy.upstream.duration` placeholder, it always had the same duration as `http.reverse_proxy.upstream.latency`, but the former was meant to be the time taken for the roundtrip _plus_ copying/writing the response.

* Delete the "Content-Length" header if we aren't copying

Fixes a bug where the Content-Length will mismatch the actual bytes written if we skipped copying the response, so we get a message like this when using curl:

```
curl: (18) transfer closed with 18 bytes remaining to read
```

To replicate:

```
{
	admin off
	debug
}

:8881 {
	reverse_proxy 127.0.0.1:8882 {
		@200 status 200
		handle_response @200 {
			header Foo bar
		}
	}
}

:8882 {
	header Content-Type application/json
	respond `{"hello": "world"}` 200
}
```

* Implement `copy_response_headers`, with include/exclude list support

* Apply suggestions from code review

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2022-03-09 11:00:51 -07:00
Andrii Kushch
d0b608af31
tracing: New OpenTelemetry module (#4361)
* opentelemetry: create a new module

* fix imports

* fix test

* Update modules/caddyhttp/opentelemetry/README.md

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* Update modules/caddyhttp/opentelemetry/README.md

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* Update modules/caddyhttp/opentelemetry/README.md

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* Update modules/caddyhttp/opentelemetry/tracer.go

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* rename error ErrUnsupportedTracesProtocol

* replace spaces with tabs in the test data

* Update modules/caddyhttp/opentelemetry/README.md

Co-authored-by: Francis Lavoie <lavofr@gmail.com>

* Update modules/caddyhttp/opentelemetry/README.md

Co-authored-by: Francis Lavoie <lavofr@gmail.com>

* replace spaces with tabs in the README.md

* use default values for a propagation and exporter protocol

* set http attributes with helper

* simplify code

* Cleanup modules/caddyhttp/opentelemetry/README.md

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* Update link in README.md

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* Update documentation in README.md

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* Update link to naming spec in README.md

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* Rename module from opentelemetry to tracing

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* Rename span_name to span

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* Rename span_name to span

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* Simplify otel resource creation

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* handle extra attributes

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* update go.opentelemetry.io/otel/semconv to 1.7.0

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* update go.opentelemetry.io/otel version

* remove environment variable handling

* always use tracecontext,baggage as propagators

* extract tracer name into variable

* rename OpenTelemetry to Tracing

* simplify resource creation

* update go.mod

* rename package from opentelemetry to tracing

* cleanup tests

* update Caddyfile example in README.md

* update README.md

* fix test

* fix module name in README.md

* fix module name in README.md

* change names in README.md and tests

* order imports

* remove redundant tests

* Update documentation README.md

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* Fix grammar

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* Update comments

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* Update comments

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* update go.sum

* update go.sum

* Add otelhttp instrumentation, update OpenTelemetry libraries.

* Use otelhttp instrumentation for instrumenting HTTP requests.

This change uses context.WithValue to inject the next handler into the
request context via a "nextCall" carrier struct, and pass it on to a
standard Go HTTP handler returned by otelhttp.NewHandler. The
underlying handler will extract the next handler from the context,
call it and pass the returned error to the carrier struct.

* use zap.Error() for the error log

* remove README.md

* update dependencies

* clean up the code

* change comment

* move serveHTTP method from separate file

* add syntax to the UnmarshalCaddyfile comment

* go import the file

* admin: Write proper status on invalid requests (#4569) (fix #4561)

* update dependencies

Co-authored-by: Dave Henderson <dhenderson@gmail.com>
Co-authored-by: Francis Lavoie <lavofr@gmail.com>
Co-authored-by: Vibhav Pant <vibhavp@gmail.com>
Co-authored-by: Alok Naushad <alokme123@gmail.com>
Co-authored-by: Cedric Ziel <cedric@cedric-ziel.com>
2022-03-08 12:18:32 -07:00
Ran Chen
d9b1d46325
caddytls: dns_challenge_override_domain for challenge delegation (#4596)
* Add a override_domain option to allow DNS chanllenge delegation

CNAME can be used to delegate answering the chanllenge to another DNS
zone. One usage is to reduce the exposure of the DNS credential [1].
Based on the discussion in caddy/certmagic#160, we are adding an option
to allow the user explicitly specify the domain to delegate, instead of
following the CNAME chain.

This needs caddy/certmagic#160.

* rename override_domain to dns_challenge_override_domain

* Update CertMagic; fix spelling

Co-authored-by: Matthew Holt <mholt@users.noreply.github.com>
2022-03-08 12:03:43 -07:00
Francis Lavoie
d058dee11d
reverseproxy: Refactor dial address parsing, augment command parsing (#4616) 2022-03-05 16:34:19 -07:00
Francis Lavoie
5bd96a6ac2
httpcaddyfile: Support explicitly turning off strict_sni_host (#4592) 2022-03-01 20:02:39 -05:00
Francis Lavoie
26d633baf8
httpcaddyfile: Disabling OCSP stapling for both managed and unmanaged (#4589) 2022-02-19 14:20:38 -07:00
Matthew Holt
ff137d17d0
caddyconfig: Support placeholders in HTTP loader 2022-02-17 22:58:25 -07:00
Matt Holt
57a708d189
caddytls: Support external certificate Managers (like Tailscale) (#4541)
Huge thank-you to Tailscale (https://tailscale.com) for making this change possible!
This is a great feature for Caddy and Tailscale is a great fit for a standard implementation.

* caddytls: GetCertificate modules; Tailscale

* Caddyfile support for get_certificate

Also fix AP provisioning in case of empty subject list (persist loaded
module on struct, much like Issuers, to surive reprovisioning).

And implement start of HTTP cert getter, still WIP.

* Update modules/caddytls/automation.go

Co-authored-by: Francis Lavoie <lavofr@gmail.com>

* Use tsclient package, check status for name

* Implement HTTP cert getter

And use reuse CertMagic's PEM functions for private keys.

* Remove cache option from Tailscale getter

Tailscale does its own caching and we don't need the added complexity...
for now, at least.

* Several updates

- Option to disable cert automation in auto HTTPS
- Support multiple cert managers
- Remove cache feature from cert manager modules
- Minor improvements to auto HTTPS logging

* Run go mod tidy

* Try to get certificates from Tailscale implicitly

Only for domains ending in .ts.net.

I think this is really cool!

Co-authored-by: Francis Lavoie <lavofr@gmail.com>
2022-02-17 15:40:34 -07:00
Francis Lavoie
5a07156894
httpcaddyfile: Add pki app root and intermediate cert/key config (#4514) 2022-01-18 12:18:31 -07:00
Francis Lavoie
bcb7a19cd3
rewrite: Add method Caddyfile directive (#4528) 2022-01-18 12:17:35 -07:00
Francis Lavoie
1b7ff5d76c
httpcaddyfile: Add default_bind global option (#4531) 2022-01-18 11:29:07 -07:00
Francis Lavoie
93a7a45e7e
httpcaddyfile: Fix incorrect handling of IPv6 bind addresses (#4532)
The `net.JoinHostPort()` function has some naiive logic for handling IPv6, it just checks if the host part has a `:` and if so it wraps the host part with `[ ]` but this causes our network type prefix to get wrapped as well, which is invalid for `caddy.NetworkAddress`. Instead, we can just concatenate the host and port manually here to avoid this side-effect.
2022-01-18 11:27:43 -07:00
Francis Lavoie
4b9849c792
httpcaddyfile: Support configuring pki app names via global options (#4450) 2022-01-05 22:45:41 -05:00
Francis Lavoie
81ee34e962
httpcaddyfile: Fix sorting edgecase for nested handle_path (#4477) 2021-12-13 13:42:08 -05:00
Runzhi He
e90d751732
caddyfile: impove fmt warning message (#4444)
Co-authored-by: Francis Lavoie <lavofr@gmail.com>
2021-12-07 10:03:58 -07:00
Francis Lavoie
5bf0adad87
caddyhttp: Make logging of credential headers opt-in (#4438) 2021-12-02 13:26:24 -07:00
Francis Lavoie
f55b123d63
caddyhttp: Split up logged remote address into IP and port (#4403) 2021-11-29 01:18:35 -05:00
Matt Holt
0eb0b60f47
logging: Remove common_log field and single_field encoder (#4149) (#4282) 2021-11-29 01:08:52 -05:00