Commit Graph

1077 Commits

Author SHA1 Message Date
Matt Holt
1edc1a45e3
core: Plugins can register listener networks (#5002)
* core: Plugins can register listener networks

This can be useful for custom listeners.

This feature/API is experimental and may change!

* caddyhttp: Expose server listeners
2022-09-01 16:30:03 -06:00
Matthew Holt
cb849bd664
caddyhttp: Disable draft versions of QUIC
See comment in #4996
2022-08-31 18:49:34 -06:00
Matthew Holt
3cd7437b3d
events: Tune logging and context cancellation 2022-08-31 18:48:46 -06:00
Francis Lavoie
d4d8bbcfc6
events: Implement event system (#4912)
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2022-08-31 15:01:30 -06:00
Matt Holt
2d5a30b908
caddyhttp: Set Content-Type for static response (#4999) 2022-08-31 09:43:46 -06:00
Matthew Holt
8cb3cf540c Minor cleanup, resolve a couple lint warnings 2022-08-29 12:31:56 -06:00
Abirdcfly
e1801fdb19
Remove duplicate words in comments (#4986) 2022-08-27 14:39:26 -06:00
Dávid Szabó
0c57facc67
reverseproxy: Add upstreams healthy metrics (#4935) 2022-08-27 12:30:23 -06:00
Matthew Holt
5fb5b81439
reverseproxy: Multiple dynamic upstreams
This allows users to, for example, get upstreams from multiple SRV
endpoints in order (such as primary and secondary clusters).

Also, gofmt went to town on the comments, sigh
2022-08-25 21:42:48 -06:00
Matthew Holt
2cc5d38229
Fix comment indentation 2022-08-25 13:28:58 -06:00
Simon Legner
66596f2d74
zstd: fix typo in comment (#4985) 2022-08-25 12:00:05 +03:00
Matthew Holt
3aabbc49a2 caddytls: Log error if ask request fails
Errors returned from the DecisionFunc (whether to get a cert on-demand)
are used as a signal whether to allow a cert or not; *any* error
will forbid cert issuance.

We bubble up the error all the way to the caller, but that caller is the
Go standard library which might gobble it up.
Now we explicitly log connection errors so sysadmins can
ensure their ask endpoints are working.

Thanks to our sponsor AppCove for reporting this!
2022-08-23 22:28:15 -06:00
jedy
e289ba6187
templates: cap of slice should not be smaller than length (#4975) 2022-08-23 08:26:02 -06:00
Francis Lavoie
a22c08a638
caddyhttp: Fix for nil handlerErr.Err (#4977) 2022-08-23 08:17:46 -06:00
Francis Lavoie
72541f1cb8
caddyhttp: Set http.error.message to the HandlerError message (#4971) 2022-08-22 23:31:07 -06:00
WilczyńskiT
c7772588bd
core: Change net.IP to netip.Addr; use netip.Prefix (#4966)
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2022-08-17 16:10:57 -06:00
Matthew Holt
a944de4ab7
Clean up metrics test code
No need to use != for booleans
2022-08-16 10:03:19 -06:00
Matt Holt
a479943acd
caddyhttp: Smarter path matching and rewriting (#4948)
Co-authored-by: RussellLuo <luopeng.he@gmail.com>
2022-08-16 08:48:57 -06:00
Abdussamet Koçak
dc62d468e9
fileserver: reset buffer before using it (#4962) (#4963) 2022-08-15 22:31:45 -06:00
Matt Holt
c79c08627d
caddyhttp: Enable HTTP/3 by default (#4707) 2022-08-15 12:01:58 -06:00
Francis Lavoie
e2a5e2293a
reverseproxy: Add unix+h2c Caddyfile network shortcut (#4953) 2022-08-12 17:09:18 -04:00
Matt Holt
f5dce84a70
reverseproxy: Ignore context cancel in stream mode (#4952) 2022-08-12 13:15:41 -06:00
Francis Lavoie
922d9f5c25
reverseproxy: Fix H2C dialer using new stdlib DialTLSContext (#4951) 2022-08-12 13:11:13 -06:00
Kévin Dunglas
085df25c7e
reverseproxy: Support 1xx status codes (HTTP early hints) (#4882) 2022-08-09 10:53:24 -06:00
Francis Lavoie
fe61209df2
logging: Fix cookie filter (#4943) 2022-08-08 19:11:02 -06:00
Matthew Holt
7ab61f46f0
fileserver: Better fix for Etag of compressed files 2022-08-08 13:09:57 -06:00
Matthew Holt
8c72f34357
fileserver: Generate Etag from sidecar file
Don't use the primary/uncompressed file for Etag when serving sidecars.

This was just overlooked initially.
2022-08-08 12:50:06 -06:00
Matthew Holt
b9618b8b98
Improve docs for ZeroSSL issuer 2022-08-08 12:50:06 -06:00
Chirag Maheshwari
d26559316f
Replace strings.Index with strings.Cut (#4932) 2022-08-06 22:03:37 -06:00
WilczyńskiT
2642bd72b7
Replace strings.Index usages with strings.Cut (#4930) 2022-08-04 11:17:35 -06:00
Matt Holt
17ae5acaba
cmd: Use newly-available version information (#4931) 2022-08-04 11:16:59 -06:00
Matt Holt
1960a0dc11
httpserver: Configurable shutdown delay (#4906) 2022-08-03 11:04:51 -06:00
Francis Lavoie
141872ed80
chore: Bump up to Go 1.19, minimum 1.18 (#4925) 2022-08-02 16:39:09 -04:00
Matt Holt
f783290f40
caddyhttp: Implement caddy respond command (#4870) 2022-08-01 13:36:22 -06:00
Matthew Holt
ebd6abcbd5
fileserver: Support virtual file system in Caddyfile 2022-07-31 21:41:26 -06:00
Matt Holt
6668271661
fileserver: Support virtual file systems (#4909)
* fileserver: Support virtual file systems (close #3720)

This change replaces the hard-coded use of os.Open() and os.Stat() with
the use of the new (Go 1.16) io/fs APIs, enabling virtual file systems.
It introduces a new module namespace, caddy.fs, for such file systems.

Also improve documentation for the file server. I realized it was one of
the first modules written for Caddy 2, and the docs hadn't really been
updated since!

* Virtualize FS for file matcher; minor tweaks

* Fix tests and rename dirFS -> osFS

(Since we do not use a root directory, it is dynamic.)
2022-07-30 13:07:44 -06:00
WingLim
1e0cdc54f8
core: Windows service integration (#4790)
Co-authored-by: Matthew Holt <mholt@users.noreply.github.com>
2022-07-29 14:06:54 -06:00
Matthew Holt
35a81d7c5b
Ignore linter warnings
Use of non-cryptographic random numbers in the load balancing
is intentional.
2022-07-28 15:40:23 -06:00
Francis Lavoie
ff2ba6de8a
caddyhttp: Clear out matcher error immediately after grabbing it (#4916)
Co-authored-by: Matthew Holt <mholt@users.noreply.github.com>
2022-07-28 15:19:48 -06:00
Matthew Holt
4fced0b6e1
Finish fixing lint errors from ea8df6ff
Follows up #4915
2022-07-28 15:16:36 -06:00
Matthew Holt
1bdd451913
caddytls: Remove PreferServerCipherSuites
It has been deprecated by Go
2022-07-28 14:50:51 -06:00
Matthew Holt
ea8df6ff11
caddyhttp: Use new CEL APIs (fix #4915)
Hahaha this is the ultimate "I have no idea what I'm doing" commit but it
compiles and the tests pass and I declare victory!

... probably broke something, should be tested more.

It is nice that the protobuf dependency becomes indirect now.
2022-07-28 14:50:28 -06:00
Matthew Holt
0bebea0d4c
caddyhttp: Log shutdown errors, don't return (fix #4908) 2022-07-25 10:39:59 -06:00
Matt Holt
a379fa4c6c
reverseproxy: Implement read & write timeouts for HTTP transport (#4905) 2022-07-23 22:38:41 -06:00
Francis Lavoie
abad9bc256
cmd: Fix reload with stdin (#4900) 2022-07-20 18:14:33 -06:00
Matthew Holt
8bdee04651
caddyhttp: Enhance comment 2022-07-16 23:33:49 -06:00
Francis Lavoie
7d1f7771c9
reverseproxy: Implement retry count, alternative to try_duration (#4756)
* reverseproxy: Implement retry count, alternative to try_duration

* Add Caddyfile support for `retry_match`

* Refactor to deduplicate matcher parsing logic

* Fix lint
2022-07-13 14:15:00 -06:00
Matthew Holt
04a14ee37a
caddyhttp: Make query matcher more efficient
Only parse query string once
2022-07-13 12:20:00 -06:00
Matthew Holt
c2bbe42fc3
reverseproxy: Export SetScheme() again
Turns out the NTLM transport uses it. Oops.
2022-07-13 08:52:30 -06:00
Francis Lavoie
53c4d788d4
headers: Only replace known placeholders (#4880) 2022-07-12 12:16:03 -06:00
Matthew Holt
d6bc9e0b5c
reverseproxy: Err 503 if all upstreams unavailable 2022-07-08 13:01:32 -06:00
Francis Lavoie
54d1923ccb
reverseproxy: Adjust new TLS Caddyfile directive names (#4872) 2022-07-08 13:04:22 -04:00
Matthew Holt
c0f76e9ed4
fileserver: Use safe redirects in file browser 2022-07-07 14:10:19 -06:00
Francis Lavoie
8bac134f26
go.mod: Bump up quic-go to v0.28.0, fixes for BC breaks (#4867) 2022-07-06 12:14:32 -06:00
Matt Holt
412dcc07d3
caddytls: Reuse issuer between PreCheck and Issue (#4866)
This enables EAB reuse for ZeroSSLIssuer (which is now supported by ZeroSSL).
2022-07-05 18:12:25 -06:00
Francis Lavoie
58e05cab15
forwardauth: Fix case when copy_headers is omitted (#4856)
See https://caddy.community/t/using-forward-auth-and-writing-my-own-authenticator-in-php/16410, apparently it didn't work when `copy_headers` wasn't used. This is because we were skipping adding a handler to the routes in the "good response handler", but this causes the logic in `reverseproxy.go` to ignore the response handler since it's empty. Instead, we can just always put in the `header` handler, even with an empty `Set` operation, it's just a no-op, but it fixes that condition in the proxy code.
2022-06-28 19:23:30 -06:00
Tristan Swadell
10f85558ea
Expose several Caddy HTTP Matchers to the CEL Matcher (#4715)
Co-authored-by: Francis Lavoie <lavofr@gmail.com>
2022-06-22 18:53:46 -04:00
Francis Lavoie
98468af8b6
reverseproxy: Fix double headers in response handlers (#4847) 2022-06-22 15:10:14 -04:00
Francis Lavoie
25f10511e7
reverseproxy: Fix panic when TLS is not configured (#4848)
* reverseproxy: Fix panic when TLS is not configured

* Refactor and simplify setScheme

Co-authored-by: Matthew Holt <mholt@users.noreply.github.com>
2022-06-22 15:01:57 -04:00
Kiss Károly Pál
b6e96fa3c5
reverseproxy: Skip TLS for certain configured ports (#4843)
* Make reverse proxy TLS server name replaceable for SNI upstreams.

* Reverted previous TLS server name replacement, and implemented thread safe version.

* Move TLS servername replacement into it's own function

* Moved SNI servername replacement into httptransport.

* Solve issue when dynamic upstreams use wrong protocol upstream.

* Revert previous commit.

Old commit was: Solve issue when dynamic upstreams use wrong protocol upstream.
Id: 3c9806ccb6

* Added SkipTLSPorts option to http transport.

* Fix typo in test config file.

* Rename config option as suggested by Matt

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>

* Update code to match renamed config option.

* Fix typo in config option name.

* Fix another typo that I missed.

* Tests not completing because of apparent wrong ordering of options.

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2022-06-20 11:51:42 -06:00
Francis Lavoie
0b6f764356
forwardauth: Support renaming copied headers, block support (#4783) 2022-06-16 14:28:11 -06:00
Matt Holt
0bcd02d5f6
headers: Support wildcards for delete ops (close #4830) (#4831) 2022-06-15 09:57:43 -06:00
Kiss Károly Pál
c82fe91104
reverseproxy: Dynamic ServerName for TLS upstreams (#4836)
* Make reverse proxy TLS server name replaceable for SNI upstreams.

* Reverted previous TLS server name replacement, and implemented thread safe version.

* Move TLS servername replacement into it's own function

* Moved SNI servername replacement into httptransport.

* Solve issue when dynamic upstreams use wrong protocol upstream.

* Revert previous commit.

Old commit was: Solve issue when dynamic upstreams use wrong protocol upstream.
Id: 3c9806ccb6

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2022-06-14 21:53:05 -06:00
Matthew Holt
f9b42c3772
reverseproxy: Make TLS renegotiation optional 2022-06-14 09:05:25 -06:00
Yaacov Akiba Slama
aaf6794b31
reverseproxy: Add renegotiation param in TLS client (#4784)
* Add renegotiation option in reverseproxy tls client

* Update modules/caddyhttp/reverseproxy/httptransport.go

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2022-06-10 09:33:35 -06:00
Matthew Holt
1498132ea3
caddyhttp: Log error from CEL evaluation (fix #4832) 2022-06-08 16:42:24 -06:00
Francis Lavoie
7f9b1f43c9
reverseproxy: Correct the tls_server_name docs (#4827)
* reverseproxy: Correct the `tls_server_name` docs

* Update modules/caddyhttp/reverseproxy/httptransport.go

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2022-06-06 12:37:09 -06:00
Matt Holt
5e729c1e85
reverseproxy: HTTP 504 for upstream timeouts (#4824)
Closes #4823
2022-06-03 14:13:47 -06:00
Gr33nbl00d
0a14f97e49
caddytls: Make peer certificate verification pluggable (#4389)
* caddytls: Adding ClientCertValidator for custom client cert validations

* caddytls: Cleanups for ClientCertValidator changes

caddytls: Cleanups for ClientCertValidator changes

* Update modules/caddytls/connpolicy.go

Co-authored-by: Francis Lavoie <lavofr@gmail.com>

* Update modules/caddytls/connpolicy.go

Co-authored-by: Francis Lavoie <lavofr@gmail.com>

* Update modules/caddytls/connpolicy.go

Co-authored-by: Francis Lavoie <lavofr@gmail.com>

* Update modules/caddytls/connpolicy.go

Co-authored-by: Francis Lavoie <lavofr@gmail.com>

* Update modules/caddytls/connpolicy.go

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>

* Update modules/caddytls/connpolicy.go

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>

* Unexported field Validators, corrected renaming of LeafVerificationValidator to LeafCertClientAuth

* admin: Write proper status on invalid requests (#4569) (fix #4561)

* Apply suggestions from code review

* Register module; fix compilation

* Add log for deprecation notice

Co-authored-by: Roettges Florian <roettges.florian@scheidt-bachmann.de>
Co-authored-by: Francis Lavoie <lavofr@gmail.com>
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
Co-authored-by: Alok Naushad <alokme123@gmail.com>
2022-06-02 14:25:07 -06:00
Matthew Holt
9864b138fb
reverseproxy: api: Remove misleading 'healthy' value
In v2.5.0, upstream health was fixed such that whether an upstream is
considered healthy or not is mostly up to each individual handler's
config. Since "healthy" is an opinion, it is not a global value.

I unintentionally left in the "healthy" field in the API endpoint for
checking upstreams, and it is now misleading (see #4792).

However, num_requests and fails remains, so health can be determined by
the API client, rather than having it be opaquely (and unhelpfully)
determined for the client.

If we do restore this value later on, it'd need to be replicated once
per reverse_proxy handler according to their individual configs.
2022-06-02 12:32:23 -06:00
Matthew Holt
3d18bc56b9
go.mod: Update go-yaml to v3 2022-06-01 15:15:20 -06:00
Matthew Holt
886ba84baa
Fix #4822 and fix #4779
The fix for 4822 is the change at the top of the file, and
4779's fix is toward the bottom of the file.
2022-06-01 15:12:57 -06:00
Alexander M
a9267791c4
reverseproxy: Add --internal-certs CLI flag #3589 (#4817)
added flag --internal-certs
when set, for non-local domains the internal CA will be used for cert generation
2022-05-29 14:33:01 -06:00
Aleks
6891f7f421
templates: Add humanize function (#4767)
Co-authored-by: Francis Lavoie <lavofr@gmail.com>
2022-05-24 19:47:08 -04:00
David Larlet
9e760e2e0c
templates: Documentation consistency (#4796) 2022-05-17 18:56:40 -04:00
Matt Holt
57d27c1b58
reverseproxy: Support http1.1>h2c (close #4777) (#4778) 2022-05-10 17:25:58 -04:00
Matthew Holt
693e9b5283
rewrite: Handle fragment before query (fix #4775) 2022-05-09 11:09:42 -06:00
Francis Lavoie
f7be0ee101
map: Prevent output destinations overlap with Caddyfile shorthands (#4657) 2022-05-06 10:25:31 -06:00
Francis Lavoie
f6900fcf53
reverseproxy: Support performing pre-check requests (#4739) 2022-05-06 10:50:26 -04:00
Francis Lavoie
ec86a2f7a3
caddyfile: Shortcut for remote_ip for private IP ranges (#4753) 2022-05-04 12:42:37 -06:00
Francis Lavoie
e7fbee8c82
reverseproxy: Permit resolver addresses to not specify a port (#4760)
Context: https://caddy.community/t/caddy-2-5-dynamic-upstreams-and-consul-srv-dns/15839

I realized it probably makes sense to allow `:53` to be omitted, since it's the default port for DNS.
2022-05-04 12:40:39 -06:00
Tyler Kropp
e84e19a04e
templates: Add custom template function registration (#4757)
* Add custom template function registration

* Rename TemplateFunctions to CustomFunctions

* Add documentation

* Document CustomFunctions interface

* Preallocate custom functions map list

* Fix interface name in error message
2022-05-02 14:55:34 -06:00
Francis Lavoie
4a223f5203
reverseproxy: Fix Caddyfile support for replace_status (#4754) 2022-05-02 11:44:28 -06:00
Francis Lavoie
0be3d99543
logging: Implement rename filter, changes field key names (#4745) 2022-04-28 11:38:44 -04:00
Francis Lavoie
dcc98da4d2
caddyhttp: Improve listen addr error message for IPv6 (#4740) 2022-04-28 08:18:45 -06:00
Marco Kaufmann
3ab648382d
templates: Add missing backticks in docs (#4737) 2022-04-27 11:41:37 -06:00
Matt Holt
40b193fb79
reverseproxy: Improve hashing LB policies with HRW (#4724)
* reverseproxy: Improve hashing LB policies with HRW

Previously, if a list of upstreams changed, hash-based LB policies
would be greatly affected because the hash relied on the position of
upstreams in the pool. Highest Random Weight or "rendezvous" hashing
is apparently robust to pool changes. It runs in O(n) instead of
O(log n), but n is very small usually.

* Fix bug and update tests
2022-04-27 10:39:22 -06:00
Francis Lavoie
d543ad1ffd
caddypki: Fix caddy trust command to use the correct API endpoint (#4730) 2022-04-25 22:00:39 -06:00
Francis Lavoie
3a1e0dbf47
httpcaddyfile: Deprecate paths in site addresses; use zap logs (#4728) 2022-04-25 10:12:10 -06:00
Francis Lavoie
77a77c0219
caddytls: Add propagation_delay, support propagation_timeout -1 (#4723) 2022-04-22 16:09:11 -06:00
Francis Lavoie
bc15b4b0e7
caddypki: Load intermediate for signing on-the-fly (#4669)
* caddypki: Load intermediate for signing on-the-fly

Fixes #4517

Big thanks to @maraino for adding an API in `smallstep/certificates` so that we can fix this

* Debug log

* Trying a hunch, does it need to be a pointer receiver?

* Clarify pointer receiver

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2022-04-13 10:20:42 -06:00
Francis Lavoie
00234c8ac2
templates: Switch to BurntSushi/toml (#4700) 2022-04-12 13:48:42 -06:00
Francis Lavoie
3e3bb00265
reverseproxy: Add _ms placeholders for proxy durations (#4666)
* reverseproxy: Add `_ms` placeholders for proxy durations

* Add http.request.duration_ms

Also add comments, and change duration_sec to duration_ms

* Add response.duration_ms for consistency

* Add missing godoc comment

Co-authored-by: Matthew Holt <mholt@users.noreply.github.com>
2022-04-11 13:04:05 -06:00
Francis Lavoie
e4ce40f8ff
reverseproxy: Sync up handleUpgradeResponse with stdlib (#4664)
* reverseproxy: Sync up `handleUpgradeResponse` with stdlib

I had left this as a TODO for when we bump to minimum 1.17, but I should've realized it was under `internal` so it couldn't be used directly.

Copied the functions we needed for parity. Hopefully this is ok!

* Add tests and fix godoc comments

Co-authored-by: Matthew Holt <mholt@users.noreply.github.com>
2022-04-11 12:49:56 -06:00
Y.Horie
afca242111
staticfiles: Expand placeholder for index files (#4679) 2022-04-07 15:01:09 -06:00
Francis Lavoie
7d229665ed
logging: Caddyfile support for duration_format (#4684)
Somehow, this was missed. Oops!
2022-04-07 11:23:28 -06:00
Matthew Holt
d06d0e79f8
go.mod: Upgrade CertMagic to v0.16.0
Includes several breaking changes; code base updated accordingly.

- Added lots of context arguments
- Use fs.ErrNotExist
- Rename ACMEManager -> ACMEIssuer; CertificateManager -> Manager
2022-03-25 11:28:54 -06:00
Matthew Holt
b8dbecb841
reverseproxy: Include port in A upstreams cache
Should fix #4659
2022-03-24 10:44:36 -06:00
Artem Mikheev
c9b5e7f77b
Fix http3 servers dying after reload (#4654) 2022-03-22 19:47:57 -04:00
Matthew Holt
79cbe7bfd0
httpcaddyfile: Add 'vars' directive
See discussion in #4650
2022-03-22 10:47:21 -06:00
Matthew Holt
55b4c12e04
map: Evaluate placeholders in output vals (#4650) 2022-03-21 17:05:38 -06:00
Matthew Holt
2196c92c0e
reverseproxy: Don't clear name in SRV upstreams
Fix for dc4d147388
2022-03-21 08:33:24 -06:00
Francis Lavoie
c5fffb4ac2
caddyfile: Support for raw token values; improve map, expression (#4643)
* caddyfile: Support for raw token values, improve `map`, `expression`

* Applied code review comments

* Rename RawVal to ValRaw

Co-authored-by: Matthew Holt <mholt@users.noreply.github.com>
2022-03-18 15:08:23 -06:00
Matthew Holt
dc4d147388
reverseproxy: Expand SRV/A addrs for cache key
Hopefully fix #4645
2022-03-18 13:42:29 -06:00
Matthew Holt
93c99f6734
map: Support numeric and bool types with Caddyfile
Based on caddyserver/website#221
2022-03-17 17:53:32 -06:00
Francis Lavoie
a9c7e94a38
chore: Comment fixes (#4634) 2022-03-13 01:38:11 -05:00
Matthew Holt
3d616e8c6d
requestbody: Return HTTP 413 (fix #4558) 2022-03-11 12:34:55 -07:00
Mohammed Al Sahaf
b82e22b459
caddyhttp: retain all values of vars matcher when specified multiple times (#4629) 2022-03-11 10:55:37 -05:00
Francis Lavoie
c7d6c4cbb9
reverseproxy: copy_response and copy_response_headers for handle_response routes (#4391)
* reverseproxy: New `copy_response` handler for `handle_response` routes

Followup to #4298 and #4388.

This adds a new `copy_response` handler which may only be used in `reverse_proxy`'s `handle_response` routes, which can be used to actually copy the proxy response downstream. 

Previously, if `handle_response` was used (with routes, not the status code mode), it was impossible to use the upstream's response body at all, because we would always close the body, expecting the routes to write a new body from scratch.

To implement this, I had to refactor `h.reverseProxy()` to move all the code that came after the `HandleResponse` loop into a new function. This new function `h.finalizeResponse()` takes care of preparing the response by removing extra headers, dealing with trailers, then copying the headers and body downstream.

Since basically what we want `copy_response` to do is invoke `h.finalizeResponse()` at a configurable point in time, we need to pass down the proxy handler, the response, and some other state via a new `req.WithContext(ctx)`. Wrapping a new context is pretty much the only way we have to jump a few layers in the HTTP middleware chain and let a handler pick up this information. Feels a bit dirty, but it works.

Also fixed a bug with the `http.reverse_proxy.upstream.duration` placeholder, it always had the same duration as `http.reverse_proxy.upstream.latency`, but the former was meant to be the time taken for the roundtrip _plus_ copying/writing the response.

* Delete the "Content-Length" header if we aren't copying

Fixes a bug where the Content-Length will mismatch the actual bytes written if we skipped copying the response, so we get a message like this when using curl:

```
curl: (18) transfer closed with 18 bytes remaining to read
```

To replicate:

```
{
	admin off
	debug
}

:8881 {
	reverse_proxy 127.0.0.1:8882 {
		@200 status 200
		handle_response @200 {
			header Foo bar
		}
	}
}

:8882 {
	header Content-Type application/json
	respond `{"hello": "world"}` 200
}
```

* Implement `copy_response_headers`, with include/exclude list support

* Apply suggestions from code review

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2022-03-09 11:00:51 -07:00
Andrii Kushch
d0b608af31
tracing: New OpenTelemetry module (#4361)
* opentelemetry: create a new module

* fix imports

* fix test

* Update modules/caddyhttp/opentelemetry/README.md

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* Update modules/caddyhttp/opentelemetry/README.md

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* Update modules/caddyhttp/opentelemetry/README.md

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* Update modules/caddyhttp/opentelemetry/tracer.go

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* rename error ErrUnsupportedTracesProtocol

* replace spaces with tabs in the test data

* Update modules/caddyhttp/opentelemetry/README.md

Co-authored-by: Francis Lavoie <lavofr@gmail.com>

* Update modules/caddyhttp/opentelemetry/README.md

Co-authored-by: Francis Lavoie <lavofr@gmail.com>

* replace spaces with tabs in the README.md

* use default values for a propagation and exporter protocol

* set http attributes with helper

* simplify code

* Cleanup modules/caddyhttp/opentelemetry/README.md

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* Update link in README.md

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* Update documentation in README.md

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* Update link to naming spec in README.md

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* Rename module from opentelemetry to tracing

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* Rename span_name to span

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* Rename span_name to span

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* Simplify otel resource creation

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* handle extra attributes

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* update go.opentelemetry.io/otel/semconv to 1.7.0

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* update go.opentelemetry.io/otel version

* remove environment variable handling

* always use tracecontext,baggage as propagators

* extract tracer name into variable

* rename OpenTelemetry to Tracing

* simplify resource creation

* update go.mod

* rename package from opentelemetry to tracing

* cleanup tests

* update Caddyfile example in README.md

* update README.md

* fix test

* fix module name in README.md

* fix module name in README.md

* change names in README.md and tests

* order imports

* remove redundant tests

* Update documentation README.md

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* Fix grammar

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* Update comments

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* Update comments

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* update go.sum

* update go.sum

* Add otelhttp instrumentation, update OpenTelemetry libraries.

* Use otelhttp instrumentation for instrumenting HTTP requests.

This change uses context.WithValue to inject the next handler into the
request context via a "nextCall" carrier struct, and pass it on to a
standard Go HTTP handler returned by otelhttp.NewHandler. The
underlying handler will extract the next handler from the context,
call it and pass the returned error to the carrier struct.

* use zap.Error() for the error log

* remove README.md

* update dependencies

* clean up the code

* change comment

* move serveHTTP method from separate file

* add syntax to the UnmarshalCaddyfile comment

* go import the file

* admin: Write proper status on invalid requests (#4569) (fix #4561)

* update dependencies

Co-authored-by: Dave Henderson <dhenderson@gmail.com>
Co-authored-by: Francis Lavoie <lavofr@gmail.com>
Co-authored-by: Vibhav Pant <vibhavp@gmail.com>
Co-authored-by: Alok Naushad <alokme123@gmail.com>
Co-authored-by: Cedric Ziel <cedric@cedric-ziel.com>
2022-03-08 12:18:32 -07:00
Ran Chen
d9b1d46325
caddytls: dns_challenge_override_domain for challenge delegation (#4596)
* Add a override_domain option to allow DNS chanllenge delegation

CNAME can be used to delegate answering the chanllenge to another DNS
zone. One usage is to reduce the exposure of the DNS credential [1].
Based on the discussion in caddy/certmagic#160, we are adding an option
to allow the user explicitly specify the domain to delegate, instead of
following the CNAME chain.

This needs caddy/certmagic#160.

* rename override_domain to dns_challenge_override_domain

* Update CertMagic; fix spelling

Co-authored-by: Matthew Holt <mholt@users.noreply.github.com>
2022-03-08 12:03:43 -07:00
Francis Lavoie
c8f2834b51
fastcgi: Protect against requests with null bytes in the path (#4614) 2022-03-07 10:06:33 -07:00
Matt Holt
ab0455922a
reverseproxy: Dynamic upstreams (with SRV and A/AAAA support) (#4470)
* reverseproxy: Begin refactor to enable dynamic upstreams

Streamed here: https://www.youtube.com/watch?v=hj7yzXb11jU

* Implement SRV and A/AAA upstream sources

Also get upstreams at every retry loop iteration instead of just once
before the loop. See #4442.

* Minor tweaks from review

* Limit size of upstreams caches

* Add doc notes deprecating LookupSRV

* Provision dynamic upstreams

Still WIP, preparing to preserve health checker functionality

* Rejigger health checks

Move active health check results into handler-specific Upstreams.

Improve documentation regarding health checks and upstreams.

* Deprecation notice

* Add Caddyfile support, use `caddy.Duration`

* Interface guards

* Implement custom resolvers, add resolvers to http transport Caddyfile

* SRV: fix Caddyfile `name` inline arg, remove proto condition

* Use pointer receiver

* Add debug logs

Co-authored-by: Francis Lavoie <lavofr@gmail.com>
2022-03-06 17:43:39 -07:00
Francis Lavoie
c50094fc9d
reverseproxy: Implement trusted proxies for X-Forwarded-* headers (#4507) 2022-03-06 18:51:55 -05:00
Francis Lavoie
d058dee11d
reverseproxy: Refactor dial address parsing, augment command parsing (#4616) 2022-03-05 16:34:19 -07:00
Francis Lavoie
09ba9e994e
fileserver: Add pass_thru Caddyfile option (#4613) 2022-03-04 20:50:05 -07:00
Matt Holt
2bb8550a4c
caddyhttp: Honor wildcard hosts in log SkipHosts (#4606) 2022-03-04 13:44:59 -07:00
Matthew Holt
a6199cf814
templates: Fix docs for .Args 2022-03-03 11:12:37 -07:00
Francis Lavoie
f5e104944e
reverseproxy: Make shallow-ish clone of the request (#4551)
* reverseproxy: Make shallow-ish clone of the request

* Refactor request cloning into separate function

Co-authored-by: Matthew Holt <mholt@users.noreply.github.com>
2022-03-03 09:54:45 -07:00
Matthew Holt
6b385a36f9
caddyhttp: Don't attempt to manage Tailscale certs
If .ts.net domains are explicitly added to config,
don't try to manage a cert for them (it will fail, and our
implicit Tailscale module will
get those certs at run-time).
2022-03-02 13:42:38 -07:00
Matthew Holt
9b7cdfa2f2
caddypki: Try to fix lint warnings 2022-03-02 13:38:05 -07:00
Matthew Holt
78e381b29f
caddypki: Refactor /pki/ admin endpoints
Remove /pki/certificates/<ca> endpoint and split into two endpoints:

- GET /pki/ca/<id> to get CA info and certs in JSON format
- GET /pki/ca/<id>/certificates to get cert in PEM chain
2022-03-02 13:00:37 -07:00
ttys3
de490c7cad
fastcgi: Set SERVER_PORT to 80 or 443 depending on scheme (#4572) 2022-03-02 11:24:16 -07:00
Francis Lavoie
bbad6931e3
pki: Implement API endpoints for certs and caddy trust (#4443)
* admin: Implement /pki/certificates/<id> API

* pki: Lower "skip_install_trust" log level to INFO

See https://github.com/caddyserver/caddy/issues/4058#issuecomment-976132935

It's not necessary to warn about this, because this was an option explicitly configured by the user. Still useful to log, but we don't need to be so loud about it.

* cmd: Export functions needed for PKI app, return API response to caller

* pki: Rewrite `caddy trust` command to use new admin endpoint instead

* pki: Rewrite `caddy untrust` command to support using admin endpoint

* Refactor cmd and pki packages for determining admin API endpoint
2022-03-02 11:08:36 -07:00
BitWuehler
ac14b64e08
caddyhttp: Support zone identifiers in remote_ip matcher (#4597)
* Update matchers.go

* Update matchers.go

* implementation of zone_id handling

* last changes in zone handling

* give return true values instead of bool

* Apply suggestions from code review

Co-authored-by: Francis Lavoie <lavofr@gmail.com>

* changes as suggested

* Apply suggestions from code review

Co-authored-by: Francis Lavoie <lavofr@gmail.com>

* Update matchers.go

* shortened the Match function

* changed mazcher handling

* Update matchers.go

* delete space

Co-authored-by: Francis Lavoie <lavofr@gmail.com>
2022-03-01 15:50:12 -07:00
Francis Lavoie
15c95e9d5b
fileserver: Canonical redir when whole path is stripped (#4549) 2022-03-01 15:32:39 -07:00
Francis Lavoie
87a1f228b4
reverseproxy: Move status replacement intercept to replace_status (#4300) 2022-03-01 14:12:43 -07:00
Francis Lavoie
186fdba916
caddyhttp: Move HTTP redirect listener to an optional module (#4585) 2022-02-19 15:36:36 -07:00
Francis Lavoie
c921e08296
logging: Add roll_local_time Caddyfile option (#4583) 2022-02-19 15:12:28 -07:00
Francis Lavoie
ddbb234d91
caddyhttp: Always log handled errors at debug level (#4584) 2022-02-19 15:10:49 -07:00
Matt Holt
57a708d189
caddytls: Support external certificate Managers (like Tailscale) (#4541)
Huge thank-you to Tailscale (https://tailscale.com) for making this change possible!
This is a great feature for Caddy and Tailscale is a great fit for a standard implementation.

* caddytls: GetCertificate modules; Tailscale

* Caddyfile support for get_certificate

Also fix AP provisioning in case of empty subject list (persist loaded
module on struct, much like Issuers, to surive reprovisioning).

And implement start of HTTP cert getter, still WIP.

* Update modules/caddytls/automation.go

Co-authored-by: Francis Lavoie <lavofr@gmail.com>

* Use tsclient package, check status for name

* Implement HTTP cert getter

And use reuse CertMagic's PEM functions for private keys.

* Remove cache option from Tailscale getter

Tailscale does its own caching and we don't need the added complexity...
for now, at least.

* Several updates

- Option to disable cert automation in auto HTTPS
- Support multiple cert managers
- Remove cache feature from cert manager modules
- Minor improvements to auto HTTPS logging

* Run go mod tidy

* Try to get certificates from Tailscale implicitly

Only for domains ending in .ts.net.

I think this is really cool!

Co-authored-by: Francis Lavoie <lavofr@gmail.com>
2022-02-17 15:40:34 -07:00
Francis Lavoie
1d0425b26f
templates: Elaborate on what's supported by the markdown function (#4564) 2022-02-06 22:14:41 -07:00
Francis Lavoie
7557d1d922
reverseproxy: Avoid returning a nil error during GetClientCertificate (#4550) 2022-02-01 23:33:36 -07:00
Dave Henderson
7ca5921a87
move common metrics-related funcs to internal package
Signed-off-by: Dave Henderson <dhenderson@gmail.com>
2022-01-25 15:07:17 -05:00
Francis Lavoie
da4a759bad
Update modules/caddyhttp/metrics_test.go 2022-01-25 15:07:17 -05:00
Dave Henderson
042abeb431
other is not uppercase
Signed-off-by: Dave Henderson <dhenderson@gmail.com>
2022-01-25 15:07:17 -05:00
Dave Henderson
eb891d4683
metrics: Enforce smaller set of method labels
Signed-off-by: Dave Henderson <dhenderson@gmail.com>
2022-01-25 15:07:17 -05:00
Kevin Daudt
44e5e9e43f
caddyhttp: Fix test when /tmp/etc already exists (#4544)
The TestFileListing test in tplcontext_test has one test that verifies
if directory traversal is not happening. The context root is set to
'/tmp' and then it tries to open '../../../../../etc', which gets
normalized to '/tmp/etc'.

The test then expects an error to be returned, assuming that '/tmp/etc'
does not exist on the system. When it does exist, it results in a test
failure:

```
--- FAIL: TestFileListing (0.00s)
    tplcontext_test.go:422: Test 4: Expected error but had none
    FAIL
    FAIL
    github.com/caddyserver/caddy/v2/modules/caddyhttp/templates	0.042s
```

Instead of using '/tmp' as root, use a dedicated directory created with
`os.MkdirTemp()` instead. That way, we know that the directory is empty.
2022-01-24 14:41:08 -07:00
Matt Holt
bf380d00ab
caddyhttp: Reject absurd methods (#4538)
* caddyhttp: Reject absurdly long methods

* Limit method to 32 chars and truncate

* Just reject the request and debug-log it

* Log remote address
2022-01-19 13:44:09 -07:00
Vojtech Vitek
94035c1797
Improve the reverse-proxy CLI --to flag help message (#4535) 2022-01-19 14:51:46 -05:00
Francis Lavoie
a79b4055e5
caddytls: Add internal Caddyfile lifetime, sign_with_root opts (#4513) 2022-01-18 12:19:50 -07:00
Francis Lavoie
bcb7a19cd3
rewrite: Add method Caddyfile directive (#4528) 2022-01-18 12:17:35 -07:00
Francis Lavoie
6e6ce2be6b
caddyhttp: Fix HTTP->HTTPS redir not preferring HTTPS port if ambiguous (#4530) 2022-01-18 11:56:00 -07:00
Francis Lavoie
1feb65952a
rewrite: Fix a double-encode issue when using the {uri} placeholder (#4516) 2022-01-13 12:17:15 -05:00
GallopingKylin
66de438a98
caddytls: Fix MatchRemoteIP provisoning with multiple CIDR ranges (#4522) 2022-01-13 11:56:18 -05:00
rayjlinden
850e1605df
caddyhttp: Return HTTP 421 for mismatched Host header (#4023)
Potential fix for #4017 although the consensus is unclear.

Made change to return status code 421 instead of 403 when StrictSNIHost matching is on.
2022-01-12 14:24:22 -07:00
Matthew Holt
c634bbe9cc
caddypki: Return error if no PEM data found
Best guess for https://caddy.community/t/on-fly-certificate-generation-based-on-sni/14639/4
2022-01-07 10:55:11 -07:00
Francis Lavoie
80d7a356b3
caddyhttp: Redirect HTTP requests on the HTTPS port to https:// (#4313)
* caddyhttp: Redirect HTTP requests on the HTTPS port to https://

* Apply suggestions from code review

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2022-01-05 18:01:15 -07:00
Matthew Holt
6cadb60fa2
templates: Document .OriginalReq
Close caddyserver/website#91
2022-01-05 13:59:59 -07:00
Денис Телюх
2e46c2ac1d
admin, reverseproxy: Stop timers if canceled to avoid goroutine leak (#4482) 2022-01-04 12:14:18 -07:00
Francis Lavoie
249adc1c87
logging: Support turning off roll compression via Caddyfile (#4505) 2022-01-04 12:11:27 -07:00
Francis Lavoie
e9dde23024
headers: Fix + in Caddyfile to properly append rather than set (#4506) 2022-01-04 10:10:11 -07:00
Francis Lavoie
3fe2c73dd0
caddyhttp: Fix MatchPath sanitizing (#4499)
This is a followup to #4407, in response to a report on the forums: https://caddy.community/t/php-fastcgi-phishing-redirection/14542

Turns out that doing `TrimRight` to remove trailing dots, _before_ cleaning the path, will cause double-dots at the end of the path to not be cleaned away as they should. We should instead remove the dots _after_ cleaning.
2021-12-30 04:15:48 -05:00
Francis Lavoie
5333c3528b
reverseproxy: Fix incorrect health_headers Caddyfile parsing (#4485)
Fixes #4481
2021-12-17 08:53:11 -07:00
Rainer Borene
180ae0cc48
caddyhttp: Implement http.request.uuid placeholder (#4285) 2021-12-15 00:17:53 -07:00
Matthew Holt
a1c41210d3 caddypki: Minor tweak, don't use context pointer 2021-12-13 16:13:38 -07:00
Matt Holt
ecac03cdcb
caddyhttp: Enhance vars matcher (#4433)
* caddyhttp: Enhance vars matcher

Enable "or" logic for multiple values.
Fall back to checking placeholders if not a var name.

* Fix tests (thanks @mohammed90 !)
2021-12-13 13:59:58 -07:00
Francis Lavoie
c04d24cafa
pki: Avoid provisioning the local CA when not necessary (#4463)
* pki: Avoid provisioning the `local` CA when not necessary

* pki: Refactor CA loading to keep the logic in the PKI app
2021-12-13 12:25:35 -07:00
Mohammed Al Sahaf
78b5356f2b
fileserver: do not double-escape paths (#4447) 2021-12-11 09:26:21 -05:00
Adam Burgess
dce81e85d5
docs: use backticks to not italicise glob path (#4460) 2021-12-05 23:48:40 -07:00
Kévin Dunglas
a1b417c832
logging: add support for hashing data (#4434)
* logging: add support for hashing data

* Update modules/logging/filters.go

Co-authored-by: wiese <wiese@users.noreply.github.com>

* Update modules/logging/filters.go

Co-authored-by: wiese <wiese@users.noreply.github.com>

Co-authored-by: wiese <wiese@users.noreply.github.com>
2021-12-02 13:51:37 -07:00
Francis Lavoie
5bf0adad87
caddyhttp: Make logging of credential headers opt-in (#4438) 2021-12-02 13:26:24 -07:00
Francis Lavoie
8e5aafa5cd
fastcgi: Fix a TODO, prevent zap using reflection for logging env (#4437)
* fastcgi: Fix a TODO, prevent zap using reflection for logging env

* Update modules/caddyhttp/reverseproxy/fastcgi/fastcgi.go

Co-authored-by: Mohammed Al Sahaf <msaa1990@gmail.com>

Co-authored-by: Mohammed Al Sahaf <msaa1990@gmail.com>
2021-12-02 13:23:19 -07:00
Tim Culverhouse
ec14ccdd40
templates: fix inconsistent nested includes (#4452) 2021-11-29 12:29:40 -05:00
Francis Lavoie
f55b123d63
caddyhttp: Split up logged remote address into IP and port (#4403) 2021-11-29 01:18:35 -05:00
Matt Holt
0eb0b60f47
logging: Remove common_log field and single_field encoder (#4149) (#4282) 2021-11-29 01:08:52 -05:00
Francis Lavoie
9ee68c1bd5
reverseproxy: Adjust defaults, document defaults (#4436)
* reverseproxy: Adjust defaults, document defaults

Related to some of the issues in https://github.com/caddyserver/caddy/issues/4245, a complaint about the proxy transport defaults not being properly documented in https://caddy.community/t/default-values-for-directives/14254/6.

- Dug into the stdlib to find the actual defaults for some of the timeouts and buffer limits, documenting them in godoc so the JSON docs get them next release.

- Moved the keep-alive and dial-timeout defaults from `reverseproxy.go` to `httptransport.go`. It doesn't make sense to set defaults in the proxy, because then any time the transport is configured with non-defaults, the keep-alive and dial-timeout defaults are lost!

- Sped up the dial timeout from 10s to 3s, in practice it rarely makes sense to wait a whole 10s for dialing. A shorter timeout helps a lot with the load balancer retries, so using something lower helps with user experience.

* reverseproxy: Make keepalive interval configurable via Caddyfile

* fastcgi: DialTimeout default for fastcgi transport too
2021-11-24 01:32:25 -05:00
Kévin Dunglas
789efa5dee
logging: add a regexp filter (#4426) 2021-11-23 10:00:20 -07:00
Kévin Dunglas
8887adb027
logging: add a filter for cookies (#4425)
* feat(logging): add a filter for cookies

* Improve godoc and add validation
2021-11-23 09:40:20 -07:00
Kévin Dunglas
bcac2beee7
logging: add a filter for query parameters (#4424)
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
Co-authored-by: Francis Lavoie <lavofr@gmail.com>
2021-11-23 04:01:43 -05:00
Mohammed Al Sahaf
1e10f6f725
fileserver: browse: do not encode the paths in breadcrumbs and page title (#4410) 2021-11-23 03:13:09 -05:00
Jeremy Lin
c8b5a81607
fileserver: Fix handling of symlink sizes in directory listings (#4415) 2021-11-22 14:59:09 -07:00
Francis Lavoie
eead337324
caddyhttp: Log non-500 handler errors at debug level (#4429)
Fixes #4428

It's best to still log handler errors at debug level so that they're hidden by default, but still accessible if additional details are necessary.
2021-11-22 11:58:25 -07:00
Matthew Holt
7d5047c1f1
caddyhttp: Log empty value for typical password headers
Work around for common misconfiguration
2021-11-22 11:31:50 -07:00
Jeremy Lin
e81369e220
fileserver: Move default browse template into a separate file (#4417)
This makes it easier for users to find the default browse template if they
want to create a custom template based on that. It also makes it easier to
view the template with proper syntax highlighting.
2021-11-15 11:53:54 -07:00
Francis Lavoie
e7457b43e4
caddyhttp: Sanitize the path before evaluating path matchers (#4407) 2021-11-08 13:45:03 -07:00
Matt Holt
24fda7514d
caddytls: Mark storage clean timestamp at end of routine (#4401)
See discussion on 42b7134ffa
2021-11-02 08:27:25 -06:00
Matthew Holt
3385856966
Fix lint message in metrics tests 2021-10-27 13:44:46 -06:00
Francis Lavoie
f73f55dba7
reverseproxy: Sanitize scheme and host on incoming requests (#4237)
* caddyhttp: Sanitize scheme and host on incoming requests

* reverseproxy: Sanitize the URL scheme and host before proxying

* Apply suggestions from code review

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2021-10-26 14:41:28 -06:00
Klaus Helenius
a21d5a001f
fileserver: Prevent focusing filter from scrolling on page load (#4393) 2021-10-20 12:15:58 -04:00
Matthew Holt
a2119c09e9
map: Fix 95c03506 (avoid repeated expansions) 2021-10-19 12:25:36 -06:00
Francis Lavoie
b092061591
reverseproxy: Prevent copying the response if a response handler ran (#4388) 2021-10-18 14:00:43 -04:00
Y.Horie
64f8b557b1
fileserver: Fix compression breaks using httpInclude (#4352) (#4358) 2021-10-16 11:09:16 -04:00
Matthew Holt
95c035060f
map: Fix regex mappings
It didn't really make sense how we were doing them before. See https://caddy.community/t/map-directive-and-regular-expressions/13866/6?u=matt
2021-10-13 17:58:20 -06:00
Simão Gomes Viana
837cdc566d
caddyhttp: reverseproxy: clarify warning for -insecure (#4379)
The question would only receive bad answers so it's better
to just say what the option actually does.
2021-10-11 16:15:00 -06:00
Oleg
cbb045a121
caddyhttp: Placeholder for client cert in DER + base64 format (#4241)
* client.certificate_pem_encoded in base64 format

* base64-encoding without pem encoding;naming change

* fix cert.Raw instead of block.bytes
2021-10-01 16:27:29 -06:00
KallyDev
c48fadc4a7
Move from deprecated ioutil to os and io packages (#4364) 2021-09-29 11:17:48 -06:00
Matthew Holt
059fc32f00
Revert 3336faf2 (close #4360)
Debug log is correct level for this
2021-09-27 12:06:06 -06:00
Matthew Holt
501da21f20
General minor improvements to docs 2021-09-24 18:31:01 -06:00
Matthew Holt
3336faf254
reverseproxy: Log error at error level (fix #4360) 2021-09-24 18:29:23 -06:00
Tim Culverhouse
16f752125f
templates: Add tests for funcInclude and funcImport (#4357)
* Update tplcontext.go

Add {{ render "/path/to/file.ext" $data }} via funcRender

* Update tplcontext.go

* Refactor funcInclude, add funcImport to enable {{block}} and {{template}}

* Fix funcImport return of nil showing up in html

* Update godocs for  and

* Add tests for funcInclude

* Add tests for funcImport

* os.RemoveAll -> os.Remove for TestFuncInclude and TestFuncImport
2021-09-20 12:29:37 -06:00
Slavik
0a5f7a677f
fileserver: Make file listing links purple once visited (#4356) 2021-09-19 22:01:11 -06:00
HayatoShiba
d3a0259944
fileserver: Fix displayed file size if it is symlink (#4354)
* Fix file size if it is symlink

* change the variable name for readability
2021-09-18 05:51:59 -06:00
Tim Culverhouse
5fda9610f9
templates: Add 'import' action (#4321)
Related to (closed) Issue #2094 on template inheritance. This PR adds a new function called "import" which works like "include", except it only takes one argument and passes it to the referenced file to be used as "." in that file.

* Update tplcontext.go

Add {{ render "/path/to/file.ext" $data }} via funcRender

* Update tplcontext.go

* Refactor funcInclude, add funcImport to enable {{block}} and {{template}}

* Fix funcImport return of nil showing up in html

* Update godocs for  and
2021-09-17 13:00:36 -06:00
Francis Lavoie
3f2c3ecf85
fastcgi: Implement try_files override in Caddyfile directive (#4347) 2021-09-17 08:23:06 -06:00
Francis Lavoie
907e2d8d3a
caddyhttp: Add support for triggering errors from try_files (#4346)
* caddyhttp: Add support for triggering errors from `try_files`

* caddyhttp: Use vars instead of placeholders/replacer for matcher errors

* caddyhttp: Add comment for matcher error var key
2021-09-17 00:52:32 -06:00
Mohammed Al Sahaf
33c70f418f
fileserver: properly handle escaped/non-ascii paths (#4332)
* fileserver: properly handle escaped/non-ascii paths

* fileserver: tests: accommodate Windows hate of colons in files names
2021-09-16 20:40:31 +00:00
Matthew Holt
2392478bd3
templates: Propagate httpError to HTTP response
Now possible with Go 1.17.
See https://github.com/golang/go/issues/34201.
2021-09-15 09:55:57 -06:00
Matthew Holt
a437206643
headers: Canonicalize case in replace (fix #4330) 2021-09-13 10:13:32 -06:00
Francis Lavoie
a779e1b383
fastcgi: Fix Caddyfile parsing when handle_response is used (#4342) 2021-09-11 14:12:21 -06:00
Steffen Brüheim
4ebf100f09
encode: ignore flushing until after first write (#4318)
* encode: ignore flushing until after first write (fix #4314)

The first write will determine if encoding has to be done and will add an Content-Encoding. Until then Flushing has to be delayed so the Content-Encoding header can be added before headers and status code is written. (A passthrough flush would write header and status code)

* Update modules/caddyhttp/encode/encode.go

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2021-08-31 13:36:36 -06:00
Francis Lavoie
f70a7578fa
reverseproxy: Remove redundant flushing (#4299)
From reading through the code, I think this code path is now obsoleted by the changes made in https://github.com/caddyserver/caddy/pull/4266.

Basically, `h.flushInterval()` will set the flush interval to `-1` if we're in a bi-directional stream, and the recent PR ensured that `h.copyResponse()` properly flushes headers immediately when the flush interval is non-zero. So now there should be no need to call Flush before calling `h.copyResponse()`.
2021-08-23 11:54:28 -06:00
Scott Mebberson
c1cd192ee7
caddyhttp: Updated the documentation for MatchQuery (#4295) 2021-08-19 22:44:28 -06:00
Matthew Holt
8a974a4f8f
logging: Warn for deprecated single_field encoder 2021-08-17 10:51:26 -06:00
Francis Lavoie
e6c29ce081
reverseproxy: Incorporate latest proxy changes from stdlib (#4266)
I went through the commits that touched stdlib's `reverseproxy.go` file, and copied over all the changes that are to code that was copied into Caddy.

The commits I pulled changes from:

- 2cc347382f
- a5cea062b3
- ecdbffd4ec
- 21898524f6
-ca3c0df1f8
- 9c017ff30d

This may also fix https://github.com/caddyserver/caddy/issues/4247 because of the change to `copyResponse` to set `mlw.flushPending = true` right away.
2021-08-12 10:48:24 -06:00
Francis Lavoie
124ba1ba71
logging: Prep for common_log removal (#4149)
See https://github.com/caddyserver/caddy/issues/4148#issuecomment-833207811
2021-07-14 11:07:38 -06:00
Francis Lavoie
1c6c7714a3
caddyhttp: Fix edgecase with auto HTTP->HTTPS logic (#4243) 2021-07-14 10:49:34 -06:00
Leo Di Donato
46d99aba85
logging: Add missing interface guards for replace filter (#4244)
Signed-off-by: Leonardo Di Donato <leodidonato@gmail.com>
2021-07-12 11:13:01 -04:00
diamondburned
9e16e80f3c
fileserver: Fix browse name_dir_first sorting (#4218)
This commit fixes the `sortByNameDirFirst` variable inside fileserver to
match what browse's default template has.

Co-authored-by: Francis Lavoie <lavofr@gmail.com>
2021-07-07 02:54:54 -04:00
elnoro
d882211080
reverseproxy: Keep path to unix socket as dial address (#4232) 2021-07-06 23:43:45 -04:00
hmol233
42e140b1b2
caddyhttp: Fix incorrect determination of gRPC protocol (#4236) 2021-07-06 12:09:44 -04:00
mritd
4245ceb67d
fileserver: Add disable_canonical_uris Caddyfile subdirective (#4222)
* feat(fileserver): add 'canonical_uris' parameter to caddyfile

add 'canonical_uris' parameter to caddyfile

reference #2741

Signed-off-by: mritd <mritd@linux.com>

* feat(file_server): rename subdirective canonical_uris to disable_canonical_uris

rename subdirective canonical_uris to disable_canonical_uris

Signed-off-by: mritd <mritd@linux.com>

* test(caddyfile_adapt): add disable_canonical_uris subdirective test file

add disable_canonical_uris subdirective test file

Signed-off-by: mritd <mritd@linux.com>
2021-07-01 17:22:16 -06:00
Matthew Holt
0bdb8aa82d
acmeserver: Don't set host for directory links by default
This makes the server more easily proxied.
2021-07-01 17:20:51 -06:00
Matthew Holt
191dc86f9e
fileserver: Clarify docs about canonicalization
Related to https://github.com/caddyserver/caddy/issues/4205.
2021-06-25 11:33:18 -06:00
Matthew Holt
81e5318021
caddytls: Remove "IssuerRaw" field
Has been deprecated and printing warnings for about 8 months now.
Replaced by "IssuersRaw" field in v2.3.0.
2021-06-25 11:29:56 -06:00
Matthew Holt
2de7e14e1c
acmeserver: Trim slashes from path prefix
See https://caddy.community/t/mtls-tls-internal-error/12807
2021-06-21 11:56:41 -06:00
Klaus Post
69c914483d
encode: Tweak compression settings (#4215)
* Tweak compression settings

zstd: Limit window sizes to 128K to keep memory in control both server and client size.
zstd: Write 0 length frames. This may be needed for compatibility.
zstd: Create fewer encoders. Small memory improvement.
gzip: Allow -2 (Huffman only) and -3 (stateless) compression modes.

* Update modules/caddyhttp/encode/zstd/zstd.go

Update docs.

Co-authored-by: Francis Lavoie <lavofr@gmail.com>

Co-authored-by: Francis Lavoie <lavofr@gmail.com>
2021-06-18 11:49:49 -06:00
Matt Holt
9d4ed3a323
caddyhttp: Refactor and export SanitizedPathJoin for use in fastcgi (#4207) 2021-06-17 09:59:08 -06:00
Matthew Holt
fbd6560976
fileserver: Only redirect if filename not rewritten (fix #4205)
This is the more correct implementation of  23dadc0d86 (#4179)... I think. This commit effectively undoes the revert in 8848df9c5d, but with corrections to the logic.

We *do* need to use the original request path (the path the browser knows) for redirects, since they are external, and rewrites are only internal.

However, if the path was rewritten to a non-canonical path, we should not redirect to canonicalize that, since rewrites are intentional by the site owner. Canonicalizing the path involves modifying only the suffix (base element, or filename) of the path. Thus, if a rewrite involves only the prefix (like how handle_path strips a path prefix), then we can (hopefully!) safely redirect using the original URI since the filename was not rewritten.

So basically, if rewrites modify the filename, we should not canonicalize those requests. If rewrites only modify another part of the path (commonly a prefix), we should be OK to redirect.
2021-06-17 09:55:49 -06:00
Matthew Holt
238914d70b
Some misc. cleanup
The fastcgi changes came from v1 which don't make sense in v2.

Fix comment about default value in reverse proxy keep alive.
2021-06-16 14:29:42 -06:00
Matthew Holt
e8ae80adca
fileserver: Don't persist parsed template (fix #4202)
Templates are parsed at request-time (like they are in the templates middleware) to allow live changes to the template while the server is running. Fixes race condition.

Also refactored use of a buffer so a buffer put back in the pool will not continue to be used (written to client) in the meantime.

A couple of benchmarks removed due to refactor, which is fine, since we know pooling helps here.
2021-06-16 14:28:34 -06:00
Matthew Holt
7c68809f4e
reverseproxy: Fix overwriting of max_idle_conns_per_host (closes #4201)
Also split the Caddyfile subdirective keepalive_idle_conns into two properties so the conns and conns_per_host can be set separately.

This is technically a breaking change, but probably anyone who this breaks already had a broken config anyway, and silently fixing it won't help them fix their configs.
2021-06-15 14:54:48 -06:00
Matthew Holt
8848df9c5d
Revert "fileserver: Redirect within the original URL (#4179)"
This reverts commit f9b54454a1.
/cc @diamondburned (see #4205)
2021-06-14 09:04:30 -06:00
Klooven
1e92258dd6
httpcaddyfile: Add preferred_chains global option and issuer subdirective (#4192)
* Added preferred_chains option to Caddyfile

* Caddyfile adapt tests for preferred_chains
2021-06-08 14:10:37 -06:00
diamondburned
76913b19ff
fileserver: Fix browse not redirecting query parameters (#4196)
This commit is a follow up to PR #4179 that introduced a bug where
browse redirections to the right URL would not preserve query
parameters.
2021-06-07 17:33:54 -06:00
Peter Magnusson
4c2da18841
caddytls: Add Caddyfile support for propagation_timeout (#4178)
* add propagation_timeout to UnmarshalCaddyfile

- Closes #4177

* added caddyfile_adapt test
2021-06-07 12:25:12 -06:00
diamondburned
f9b54454a1
fileserver: Redirect within the original URL (#4179)
This commit changes the file_server directive to redirect using the
original request's URL instead of the possibly trimmed URL. This should
make file_server work with handle_path.

This fix is taken from mholt's comment in
https://caddy.community/t/file-servers-on-different-paths-not-working/11698/11.
2021-06-07 12:20:08 -06:00
Matthew Holt
2a8109468c
reverseproxy: Always remove hop-by-hop headers
See golang/go#46313

Based on 950fa11c4c
2021-06-04 15:21:16 -06:00
Francis Lavoie
94b712009a
logging: Actually use level_key (#4189) 2021-06-04 14:15:43 -06:00
Matthew Holt
ecd5eeab38
go.mod: Update direct dependencies 2021-06-03 12:18:25 -06:00
Matt Holt
e3c369d452
logging: Implement dial timeout for net writer (fix #4083) (#4172)
* logging: Implement dial timeout for net writer (fix #4083)

* Limit how often redials are attempted

This should cause dial blocking to occur only once every 10 seconds at most, but it also means the logger connection might be down for up to 10 seconds after it comes back online; oh well. We shouldn't block for DialTimeout at every single log emission.

* Clarify offline behavior
2021-05-19 15:14:03 -06:00
Francis Lavoie
aef8d4decc
reverseproxy: Set the headers in the replacer before handle_response (#4165)
Turns out this was an oversight, we assumed we could use `{http.response.header.*}` but that doesn't work because those are grabbed from the response writer, and we haven't copied any headers into the response writer yet.

So the fix is to set all the response headers into the replacer at a new namespace before running the handlers.

This adds the `{http.reverse_proxy.header.*}` replacer.

See https://caddy.community/t/empty-http-response-header-x-accel-redirect/12447
2021-05-12 14:19:08 -06:00
Matthew Penner
bc22102478
caddyfile: Fix caddy fmt nesting not decrementing (#4157)
* caddyfile(formatter): fix nesting not decrementing

This is an extremely weird edge-case where if you had a environment variable {}
on one line, a comment on the next line, and the closing of the block on the
following line; the rest of the Caddyfile would be indented further than it
should've been.

ref; https://github.com/matthewpi/vscode-caddyfile-support/issues/13

* run gofmt

* fmt: better way of handling edge case
2021-05-10 12:01:27 -06:00
Francis Lavoie
f5db41ce1d
encode: Drop prefer from Caddyfile (#4156)
Followup to #4150, #4151 /cc @ueffel @polarathene

After a bit of discussion with @mholt, we decided to remove `prefer` as a subdirective and just go with using the order implicitly always. Simpler config, simpler docs, etc.

Effectively changes 7776471 and reverts a small part of f35a7fa.
2021-05-10 11:12:59 -06:00
Francis Lavoie
77764714ad
encode: Default to order the formats are enabled for prefer in Caddyfile (#4151) 2021-05-10 10:06:38 -06:00
Francis Lavoie
61642b766b
caddytls: Run replacer on ask URL, for env vars (#4154)
Fixes #3922
2021-05-08 22:37:27 -06:00
Francis Lavoie
d4b2f1bcee
caddyhttp: Fix fallback for the error handler chain (#4131)
* caddyhttp: Fix fallback for the error handler chain

The fix I went with in the end (after realizing some mistaken assumptions in #4131) is to just make the routes fall back to errorEmptyHandler instead of the non-error empty handler, if Terminal is true, making the routes error-aware. Ultimately this was probably just an oversight when errors was implemented at some point in the early betas of v2.

See https://caddy.community/t/problem-with-basicauth-handle-errors/12243/9 for context.

* Revert "caddyhttp: Fix fallback for the error handler chain"

This reverts commit 95b6ac44a6.

* caddyhttp: Fix via `routes.go`
2021-05-05 15:55:40 -06:00
Matthew Holt
a17c3b568d
reverseproxy: Minor logging improvements 2021-05-05 14:52:24 -06:00
Francis Lavoie
74f5d66c48
fileserver: Fix file matcher with empty try_files (#4147)
* fileserver: Fix `file` matcher with empty `try_files`

Fixes https://github.com/caddyserver/caddy/issues/4146

If `TryFiles` is empty, we fill it with `r.URL.Path`. In this case, this is `/`. Then later, in `prepareFilePath()`, we run the replacer (which turns `{path}` into `/` at that point) but `file` remains the original value (and the placeholder is still the placeholder there).

So then `strings.HasSuffix(file, "/")` will be `false` for the placeholder, but `true` for the empty `TryFiles` codepath, because `file` was `/` due to being set to the actual request value beforehand.

This means that `suffix` becomes `//` in that case, so after `sanitizedPathJoin`, it becomes `./`, so `strictFileExists`'s `strings.HasSuffix(file, separator)` codepath will return true.

I think we should change the `m.TryFiles == nil` codepath to `m.TryFiles = []string{"{http.request.uri.path}"}` for consistency. (And maybe consider hoisting this to `Provision` cause there's no point doing this on every request). I don't think this "optimization" of directly using `r.URL.Path` is so valuable, cause it causes this edgecase with directories.

* Update modules/caddyhttp/fileserver/matcher.go

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2021-05-04 09:49:13 -06:00
Francis Lavoie
e4a22de9d1
reverseproxy: Add handle_response blocks to reverse_proxy (#3710) (#4021)
* reverseproxy: Add `handle_response` blocks to `reverse_proxy` (#3710)

* reverseproxy: complete handle_response test

* reverseproxy: Change handle_response matchers to use named matchers

reverseproxy: Add support for changing status code

* fastcgi: Remove obsolete TODO

We already have d.Err("transport already specified") in the reverse_proxy parsing code which covers this case

* reverseproxy: Fix support for "4xx" type status codes

* Apply suggestions from code review

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>

* caddyhttp: Reorganize response matchers

* reverseproxy: Reintroduce caddyfile.Unmarshaler

* reverseproxy: Add comment mentioning Finalize should be called

Co-authored-by: Maxime Soulé <btik-git@scoubidou.com>
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2021-05-02 12:39:06 -06:00
Calvin Xiao
53ececda21
caddyhttp: performance improvement in HeaderRE Matcher (#4143)
Below is the report using `benchstat` and cmd:

`go test -run=BenchmarkHeaderREMatcher -bench=BenchmarkHeaderREMatcher -benchmem -count=10`

```
name                old time/op    new time/op    delta
HeaderREMatcher-16     869ns ± 1%     658ns ± 0%  -24.29%  (p=0.000 n=10+10)

name                old alloc/op   new alloc/op   delta
HeaderREMatcher-16      144B ± 0%      112B ± 0%  -22.22%  (p=0.000 n=10+10)

name                old allocs/op  new allocs/op  delta
HeaderREMatcher-16      7.00 ± 0%      5.00 ± 0%  -28.57%  (p=0.000 n=10+10)
```
2021-05-02 10:35:28 -06:00
Jason Du
637fd8f67b
fileserver: Share template logic for both templates and file_server browse (#4093)
Co-authored-by: Matthew Holt <mholt@users.noreply.github.com>
2021-04-30 22:17:23 -04:00
Matt Holt
956f01163d
caddytls: Implement remote IP connection matcher (#4123)
* caddytls: Implement remote IP connection matcher

* Implement IP range negation

If both Ranges and NotRanges are specified, both must match.
2021-04-30 10:14:52 -06:00
Simão Gomes Viana
9017557169
reverseproxy: fix hash selection policy (#4137)
* caddyhttp: reverseproxy: fix hash selection policy

Fixes: #4135
Test: go test './...' -count=1

* caddyhttp: reverseproxy: add test to catch #4135

If you revert the last commit, the test will fail.
2021-04-29 10:52:22 -06:00
Francis Lavoie
3a1e81dbf6
fileserver: Better handling of HTTP status override (#4132) 2021-04-29 02:01:48 -04:00
Francis Lavoie
1e218e1d2e
caddytls: Add load_storage module (#4055)
An idea that came up in https://caddy.community/t/save-internally-issued-wildcard-certificate-in-consul/11740, this a simple module that might be useful for anyone who uses storage modules that aren't filesystem, to let them load certs/keys externally issued for use by Caddy.

Bit goofy, since we need to fetch the certmagic.Storage during provisioning, it needs a wrapping struct instead of just being an array like `load_files`.

Future work might involve adding Caddyfile support via a subdirective of the `tls` directive maybe?
2021-04-21 17:05:55 -06:00
Francis Lavoie
4d0474e3b8
reverseproxy: Admin endpoint for reporting upstream statuses (#4125) 2021-04-21 13:43:34 -06:00
Francis Lavoie
d789596bc0
caddyhttp: Implement better logic for inserting the HTTP->HTTPS redirs (#4033)
* caddyhttp: Implement better logic for inserting the HTTP->HTTPS redirs

* caddyhttp: Add integration test
2021-04-19 19:54:12 -06:00
Matthew Holt
2250920e1d
caddytls: Disable OCSP stapling for manual certs (#4064) 2021-04-12 16:09:02 -06:00
Matthew Holt
42b7134ffa
caddytls: Configurable storage clean interval
Can drastically reduce costs on storage backends where scans are expensive.

Also reduced default interval to 24h.

See https://github.com/silinternational/certmagic-storage-dynamodb/issues/18
2021-04-12 15:41:22 -06:00