Updated ifCondition test to deep test all fields.
Changed NewComplexRule to not return a pointer.
Corrected panic detection in formatting.
Fixed failing test cases.
Fixed review bug for test.
Fixes bug caused by Replacer running on the regular expressions in IfMatcher. We also now compile regular expressions up front to detect errors.
Fixes rewrite bugs that come from formatting a rule as a string and failing with nil dereference caused by embedding Regexp pointer in a Rule. Re: Issue #1794
Use httpserver.IndexFile() to determine index files
Test if middleware pushes indexfile when requesting directory
Fix codereview issues
Serve original request first, push later
Revert "Serve original request first, push later"
This reverts commit 2c66f01115747e5665ba7f2d33e2fd551dc31877.
According to https://tools.ietf.org/html/rfc7232#section-2.1
> Likewise, a validator is weak if it is shared by two or more
representations of a given resource at the same time, unless those
representations have identical representation data. For example, if
the origin server sends the same validator for a representation with
a gzip content coding applied as it does for a representation with no
content coding, then that validator is weak.
Therefore, after gzip, we should change the original etag to weak etag.
go vet caused a build fail in https://travis-ci.org/mholt/caddy/jobs/248392875:
upstream_test.go:480::error: possible formatting directive in Error call (vet)
This patch changes the Error call added in commit 078c991574 to
an Errorf call to support the use of the %d token.
* vendor: update Lumberjack dep
* httpserver/roller: introduce rotate_compress directive
This directive will enable gzip compression provided by [Lumberjack](https://github.com/natefinch/lumberjack/pull/43).
The directive `rotate_compress` can be `true` or `false`, being `false` by default.
* httpserver/roller: remove need to set bool with rotate_compress option
See discussion on commit c9b022b5e0
If we overwrite the Server header, it becomes difficult/impossible to
know from the client whether the request was proxied through Caddy.
* Browse: Show symbolic links and targets type properly
* gofmt
Signed-off-by: Jonas Östanbäck <jonas.ostanback@gmail.com>
* Move symbolic link check in to isSymlinkTargetDir
Signed-off-by: Jonas Östanbäck <jonas.ostanback@gmail.com>
* Revert template change and show sym link folders as normal folders
* browse: Updated icons including symlink indicators
The vendor/ folder was created with the help of @FiloSottile's gvt and
vendorcheck.
Any dependencies of Caddy plugins outside this repo are not vendored.
We do not remove any unused, vendored packages because vendorcheck -u
only checks using the current build configuration; i.e. packages that
may be imported by files toggled by build tags of other systems.
CI tests have been updated to ignore the vendor/ folder. When Go 1.9 is
released, a few of the go commands should be revised to again use ./...
as it will ignore the vendor folder by default.
* httpserver/all: Clean up and standardize request URL handling
The HTTP server now always creates a context value on the request which
is a copy of the request's URL struct. It should not be modified by
middlewares, but it is safe to get the value out of the request and make
changes to it locally-scoped. Thus, the value in the context always
stores the original request URL information as it was received. Any
rewrites that happen will be to the request's URL field directly.
The HTTP server no longer cleans /sanitizes the request URL. It made too
many strong assumptions and ended up making a lot of middleware more
complicated, including upstream proxying (and fastcgi). To alleviate
this complexity, we no longer change the request URL. Middlewares are
responsible to access the disk safely by using http.Dir or, if not
actually opening files, they can use httpserver.SafePath().
I'm hoping this will address issues with #1624, #1584, #1582, and others.
* staticfiles: Fix test on Windows
@abiosoft: I still can't figure out exactly what this is for. 😅
* Use (potentially) changed URL for browse redirects, as before
* Use filepath.ToSlash, clean up a couple proxy test cases
* Oops, fix variable name
* Raise syntax error if no '/' prefix to rewrite. Added Tests
* fix case where to keyword is used.
* Fixed spelling issue
* Changes to use Errf rather than new Err function
* Remove new RewritePathErr Function
Original feature request in forum:
https://forum.caddyserver.com/t/caddy-with-specific-hosts-but-on-demand-tls/1704?u=matt
Before, Caddy obtained certificates for every name it could at startup.
And it would only obtain certificates during the handshake for sites
defined with a hostname that didn't qualify at startup (like
"*.example.com" or ":443"). This made sense for most situations, and
helped ensure that certificates were obtained as early and reliably as
possible.
With this change, Caddy will NOT obtain certificates for hostnames it
knows at startup (even if they qualify) if OnDemand is enabled.
But I think this change generalizes well, because a user who specifies
max_certs is deliberately turning on On-Demand TLS, fully aware of
the consequences. It seems dubious to ignore that config when the user
deliberately put it there. We'll see how this goes.
* Create list of index files based on extensions and check on a per config
basis
* remove log lines
* fixed tests
* made gofmt suggested change
* Changes made to simplify
* Respect the 'insecure_skip_verify' for the health check.
* WIP: Trying to add a test. Non functional.
* Fixing tests.
* Creating better error messages.
* Optimize two more error messages.
* Move the tests into an extra function.
* Add a shutdown function and context to staticUpstream so that running goroutines can be cancelled. Add a GetShutdownFunc to Upstream interface to expose the shutdown function to the caddy Controller for performing it on restarts.
* Make fakeUpstream implement new Upstream methods.
Implement new Upstream method for fakeWSUpstream as well.
* Rename GetShutdownFunc to Stop(). Add a waitgroup to the staticUpstream for controlling individual object's goroutines. Add the Stop function to OnRestart and OnShutdown. Add tests for checking to see if healthchecks continue hitting a backend server after stop has been called.
* Go back to using a stop channel since the context adds no additional benefit.
Only register stop function for onShutdown since it's called as part of restart.
* Remove assignment to atomic value
* Incrementing WaitGroup outside of goroutine to avoid race condition. Loading atomic values in test.
* Linting: change counter to just use the default zero value instead of setting it
* Clarify Stop method comments, add comments to stop channel and waitgroup and remove out of date comment about handling stopping the proxy. Stop the ticker when the stop signal is sent
Previously directories have been merely pulled to the front, and then
sorted arbitrarily. That is, their order among themselves depended on
the filesystem implementations. Something opaque to the visitor.
This fixes said inconsistency, and implements the by-size-then-by-name
order I initially intended for this.
* Use helper functions in staticfiles to redirect.
Previously the browse package invoked staticfiles.Redirect when
redirecting clients who requested a directory but with a Request-URI
that did not contain a trailing '/'. staticfiles.Redirect only used a
relative URI. This change defers the decision of how to format the
Location header value to the helper methods in the staticfiles package.
* Update const URLPathCtxKey in browse package.
* Add the first policy which sends the request to the first available host
* Make the error message clear. As we expect the second not first upstream
host.
* Fixed issue with {path} actually {uri}
* Test added for path rewrite
* add in uri_escaped
* added rewrite_uri and test
* fix broken test. Just checks for existance of rewrite header
* gitignore
* Use context to store uri value
* ignore .vscode
* tidy up, removal of comments and invalidated tests
* Remove commented out code.
* added comment as requested by lint
* fixed spelling mistake
* clarified code with variable name
* added context for uri and test
* added TODO comment to move consts
* Fixed#1484
Fixed a nil pointer runtime error in newConnHijackerTransport,
where the access to the TLSClientConfig did not check for nil values.
* Minor improvement to UseInsecureTransport
This prevents overwriting a possibly preexisting TLSClientConfig,
even though only a single field should be changed.
* add support for listener middleware
* add proxyprotocol directive
* make caddy.Listener interface required
* Remove tcpKeepAliveListener wrapper from Serve()
This is now done in the Listen() function, along with other potential middleware.
* Fix for missing content-length header when using QUIC
If request.ContentLength is set then it will be used instead of getting
it from request.Header map since quic-go(lucas-clemente/quic-go@bb24be8)
will not store (and pass) the Content-Length header using its header
map.
This fixes a potential issue where FastCGI POST requests body empty when
QUIC is enabled. (#1370)
* Change the data type for fastcgi contentLength to int64
quic-go uses int64 for contentLength
* Fix an error for undeclared variable
* Fix test for fcgiclient
the data type for contentLength
* Support realms with basic authentication
* Add test for default basicauth directive in which realm is not specified
* Correct typo: missing space
* Remove 'path' subdirective
If use gzip and templates at the same time, the response body will
be gzipped data. And in this case, the Content-Type header won't be
set by Caddy code. Then Go http package will set "Content-Type" to
wrong value "application/x-gzip" which is determined by response body.
So the header Contenty-Type should be set in templates middleware.
This commit removes _almost_ all instances of hard-coded ports 80 and
443 strings, and now allows the user to define what the HTTP and HTTPS
ports are by the -http-port and -https-ports flags.
(One instance of "80" is still hard-coded in tls.go because it cannot
import httpserver to get access to the HTTP port variable. I don't
suspect this will be a problem in practice, but one workaround would be
to define an exported variable in the caddytls package and let the
httpserver package set it as well as its own HTTPPort variable.)
The port numbers required by the ACME challenges HTTP-01 and TLS-SNI-01
are hard-coded into the spec as ports 80 and 443 for good reasons,
but the big question is whether they necessarily need to be the HTTP
and HTTPS ports. Although the answer is probably no, they chose those
ports for convenience and widest compatibility/deployability. So this
commit also assumes that the "HTTP port" is necessarily the same port
on which to serve the HTTP-01 challenge, and the "HTTPS port" is
necessarily the same one on which to serve the TLS-SNI-01 challenge. In
other words, changing the HTTP and HTTPS ports also changes the ports
the challenges will be served on.
If you change the HTTP and HTTPS ports, you are responsible for
configuring your system to forward ports 80 and 443 properly.
Closes#918 and closes#1293. Also related: #468.
* Use RequestURI when redirecting to canonical path.
Caddy may trim a request's URL path when it starts with the path that's
associated with the virtual host. This change uses the path from the request's
RequestURI when performing a redirect.
Fix issue #1327.
* Rename redirurl to redirURL.
* Redirect to the full URL.
The scheme and host from the virtual host's site configuration is used
in order to redirect to the full URL.
* Add comment and remove redundant check.
* Store the original URL path in request context.
By storing the original URL path as a value in the request context,
middlewares can access both it and the sanitized path. The default
default FileServer handler will use the original URL on redirects.
* Replace contextKey type with CtxKey.
In addition to moving the CtxKey definition to the caddy package, this
change updates the CtxKey references in the httpserver, fastcgi, and
basicauth packages.
* httpserver: Fix reference to CtxKey
Timeouts are important for mitigating slowloris, yes. But after a number
of complaints and seeing that default timeouts are a sore point of
confusion, we're disabling them now. However, the code that sets
default timeouts remains intact; the defaults are just the zero value.
While Caddy aims to be secure by default, Caddy also aims to serve a
worldwide audience. Even my own internet here in Utah is poor at times,
with bad WiFi signal, causing some connections to take over 10s to
be established. Many use the Internet while commuting on slower
connection speeds. Latency across country borders is another concern.
As such, disabling default timeouts will serve a greater population of
users than enabling them, as slowloris is easy to mitigate and does
not seem to be reported often (I've only seen it once). It's also very
difficult sometimes to distinguish slowloris from genuine slow networks.
That decision is best left to the site owner for now.
* tls: Refactor TLS config innards with a few minor syntax changes
muststaple -> must_staple
"http2 off" -> "alpn" with list of ALPN values
* Fix typo
* Fix QUIC handler
* Inline struct field assignments
* Store name of authenticated user in basicauth for use by upstream middleware such as fastcgi and cgi.
* Use request context to transfer name of authorized user from basicauth to upstream middleware. Test retrieval of name from context.
* Remove development code that was inadvertently left in place
* Use keys of type httpserver.CtxKey to access Context values
* WIP: Implement HTTPS interception detection by Durumeric, et. al.
Special thanks to @FiloSottile for guidance with the custom listener.
* Add {{.IsMITM}} context action and {mitm} placeholder
* Improve MITM detection heuristics for Firefox and Edge
* Add tests for MITM detection heuristics
* Improve Safari heuristics for interception detection
* Read ClientHello during first Read() instead of during Accept()
As far as I can tell, reading the ClientHello during Accept() prevents
new connections from being accepted during the read. Since Read() should
be called in its own goroutine, this keeps Accept() non-blocking.
* Clean up MITM detection handler; make possible to close connection
* Use standard lib cipher suite values when possible
* Improve Edge heuristics and test cases
* Refactor MITM checking logic; add some debug statements for now
* Fix bug in MITM heuristic tests and actual heuristic code
* Fix gofmt
* Remove debug statements; preparing for merge
* fixing panic when root is symlink
checking root path is a symlink before os.Stat which panics
* fixing formatting
* adding test to verify symlink root path check
* fixing typo
* Add request placeholder support for querying request cookies.
This adds the ability to query the request cookies for placeholders
using the syntax "@cookiename".
For example, this would allow rewriting based on a cookie:
rewrite {
if @version is 'dev'
to /dev/index.html
}
* Switch cookie special char from @ to :
* Switch special char for cookies from : to ~
* Fix data race for max connection limiting in proxy directive.
The Conns and Unhealthy fields are updated concurrently across all active
requests. Because of this, they must use atomic operations for reads and
writes.
Prior to this change, Conns was incremented atomically, but read unsafely.
Unhealthly was updated & read unsafely. The new test
TestReverseProxyMaxConnLimit exposes this race when run with -race.
Switching to atomic operations makes the race detector happy.
* oops, remove leftover dead code.
* Use new subdirectives and flatten rolling config
* Set default rotate config
* Set default rolling config (hopefully) errwhere
* Make private
* Flatten errors directive and remove c.IncrNest()
* Don't skip first error log roller subdirective we see
* Remove hadBlock
* Try lumberjack import
* Unname import
The standard lib pprof library doesn't set its own Content-Type header
properly. If pprof is used with gzip, the index endpoint will be
interpreted as a .gz file; so we force its hand and set the header.
* proxy: use a new context for the outgoing request
fix issue #1345
Signed-off-by: Tw <tw19881113@gmail.com>
* proxy: add test for canceling the request
Signed-off-by: Tw <tw19881113@gmail.com>
* Generate meta elements from useful front matters.
Limited to the default template and specific elements.
* Rerun gofmt
* Add "keywords" and remove "language" to/from the list of meta tags.
* Add a simple positive list test for the meta tag generation.
* Move the meta tag list to a var at the begin of the file.
Seperate the Meta tags from the other front matters:
- Don't override user settings with name `meta`
- Cleaner Code.
* Remove the uneccessary `[:]` in the []Bytes to String casting.
@mholt was right ;)
* One minor refinement. Combining two statements.
Because of this commit(6e36811c37399d60cbce587b7c48e611009c5aec) on go tip,
it will probe the request's body to determine whether to use chunked transfer
encoding which trailers depend on it.
So we just offer a non empty body to make trailers work.
fix issue #1359
Signed-off-by: Tw <tw19881113@gmail.com>
* Add {whenISO} to record timestamp in ISO 8601 format in UTC.
ISO 8601 is the standard time format and is easy to parse.
This change assumes users desiring ISO 8016 generally prefer UTC for simplicity.
This results in {whenISO} to be significantly shorter than {when}:
{when} = "02/Jan/2006:15:04:05 +0000"
{whenISO} = "2006-01-02T15:04:12Z"
Add unit test to verify both, as there was no unit test for {when}.
* Rename {whenISO} to {when_iso}
If only one upstream is defined we don't need to buffer the body.
Instead we directly stream the body to the upstream host,
which reduces memory usage as well as latency.
Furthermore this enables different kinds of HTTP streaming
applications like gRPC for instance.
If a site owner protects a path with basicauth, no need
to use the Authorization header elsewhere upstream, especially since it
contains credentials.
If this breaks anyone, it means they're double-dipping. It's usually
good practice to clear out credentials as soon as they're not needed
anymore. (Note that we only clear credentials after they're used,
they stay for any other reason.)
* Added path cleanup functions with masking to preserve certain patterns + unit tests, #1298
* Use custom PathClean function instead of path.Clean to apply masks to preserve protocol separator in the path
* Indentation corrected in the test data map to pass the lint
* Fixing ineffassign of a temporary string variable
* Improved variable naming and documentation
* Improved variable naming
* Added benchmarks and improved variable naming in tests
* Removed unnecessary value capture when iterating over a map for keys
* A typo correction
This issue was caused by connHijackerTransport trying to record HTTP
response headers by "hijacking" the Read() method of the plain net.Conn.
This does not simply work over TLS though since this will record the TLS
handshake and encrypted data instead of the actual content.
This commit fixes the problem by providing an alternative transport.DialTLS
which correctly hijacks the overlying tls.Conn instead.
* Fixed HTTP/2 support for the proxy middleware
http.Transport instances whose TLSClientConfig, Dial, or DialTLS field
is non-nil will be configured without HTTP/2 support by default.
This commit adds the proper calls to http2.ConfigureTransport()
everywhere a http.Transport is created and thus fixes HTTP/2 in the
proxy middleware whenever insecure_skip_verify or keepalive is provided.
* Added HTTP/2 support check to TestReverseProxyInsecureSkipVerify
