Commit Graph

43 Commits

Author SHA1 Message Date
Matthew Holt
2ed1dd6afc
Merge branch 'master' into acmev2
# Conflicts:
#	caddyhttp/httpserver/replacer.go
#	caddyhttp/httpserver/replacer_test.go
2018-03-25 21:56:11 -06:00
Matthew Fay
f1eaae9b0d httpserver: Rework Replacer loop to ignore escaped braces (#2075)
* httpserver.Replacer: Rework loop to ignore escaped placeholder braces

* Fix typo and ineffectual assignment to ret

* Remove redundant idxOffset declaration, simplify escape check

* Add benchmark tests for new Replacer code
2018-03-18 20:42:43 -06:00
David Somers
ca34a3e1aa httpserver: Placeholders for tls_protocol and tls_cipher (#2062)
Also add SSL_PROTOCOL and SSL_CIPHER env vars for fastcgi.

* Implement placeholders for ssl_protocol and ssl_cipher

* gofmt

* goimports

* Housekeeping and implement as {tls_protocol} and {tls_cipher}
2018-03-17 17:27:10 -06:00
Matthew Holt
aaec7e469c
httpserver: Add {labelN} placeholders for parts of hostnames
For example, {label1} would match "sub" in "sub.example.com" or whatever
value is in the wildcard spot of "*.example.com". Useful for rewrite!
2018-03-14 21:57:25 -06:00
Toby Allen
3b66865da5 httpserver: Placeholder for response header fields (#2029)
* Allow Response Headers in logs

* Remove log line

* remove unneeded log import

* Check if rr is nil.  Added test to check

* merge if statements

* remove temp file
2018-02-18 14:21:06 -07:00
Aaron Taylor
9a22cda15d httpserver: give each req context a Replacer that preserves custom values (#1937)
This allows custom replacements to be defined in a way that propagates
throughout all plugins.
2017-11-07 10:10:03 -07:00
Matthew Holt
baf6db5b57
Apply Apache license to all .go source files (closes #1865)
I am not a lawyer, but according to the appendix of the license,
these boilerplate notices should be included with every source file.
2017-09-22 23:56:58 -06:00
bananenmannfrau
f6a96227c4 adds unix timestamp placeholder 2017-07-05 22:08:07 +02:00
Toby Allen
133ed18374 Create request_id directive #1590 (#1711)
* Create request_id directive #1590

* Address Comments

* Fix TestListenerAddrEqual

* requestid: Add some tests

* Address Comments by tobya

* Address Comments
2017-06-24 13:54:35 -07:00
Matthew Holt
0da76e2b76
mitm: Add experimental Tor support for interception detection 2017-06-07 14:20:15 -06:00
Jonas Östanbäck
05ea5c32be Fix lint warning by renaming MaxBytesExceededErr -> ErrMaxBytesExceeded (#1676)
Signed-off-by: Jonas Östanbäck <jonas.ostanback@gmail.com>
2017-05-17 09:57:11 -06:00
Tw
ae645ef2e9 Introduce limits middleware
1. Replace original `maxrequestbody` directive.
2. Add request header limit.

fix issue #1587

Signed-off-by: Tw <tw19881113@gmail.com>
2017-05-08 17:18:04 +08:00
Matt Holt
d5371aff22 httpserver/all: Clean up and standardize request URL handling (#1633)
* httpserver/all: Clean up and standardize request URL handling

The HTTP server now always creates a context value on the request which
is a copy of the request's URL struct. It should not be modified by
middlewares, but it is safe to get the value out of the request and make
changes to it locally-scoped. Thus, the value in the context always
stores the original request URL information as it was received. Any
rewrites that happen will be to the request's URL field directly.

The HTTP server no longer cleans /sanitizes the request URL. It made too
many strong assumptions and ended up making a lot of middleware more
complicated, including upstream proxying (and fastcgi). To alleviate
this complexity, we no longer change the request URL. Middlewares are
responsible to access the disk safely by using http.Dir or, if not
actually opening files, they can use httpserver.SafePath().

I'm hoping this will address issues with #1624, #1584, #1582, and others.

* staticfiles: Fix test on Windows

@abiosoft: I still can't figure out exactly what this is for. 😅

* Use (potentially) changed URL for browse redirects, as before

* Use filepath.ToSlash, clean up a couple proxy test cases

* Oops, fix variable name
2017-05-01 23:11:10 -06:00
Aaron Ellington
7ee4ea244f lint fixes 2017-04-29 20:53:58 -04:00
Toby Allen
d5cc10f7aa Added Const for use of CtxKeys (#1511)
* Added Const for CtxKeys

* Move CtxKey Const declarations

* Fixed tests

* fix test
2017-03-13 09:22:46 -06:00
Toby Allen
cfe52084aa Fix issue #1346 {path} logging {uri} and add {rewrite_uri} placeholder (#1481)
* Fixed issue with {path} actually {uri}

* Test added for path rewrite

* add in uri_escaped

* added rewrite_uri and test

* fix broken test.  Just checks for existance of rewrite header

* gitignore

* Use context to store uri value

* ignore .vscode

* tidy up, removal of comments and invalidated tests

* Remove commented out code.

* added comment as requested by lint

* fixed spelling mistake

* clarified code with variable name

* added context for uri and test

* added TODO comment to move consts
2017-03-11 14:59:47 -07:00
Samuel BERTHE
75ccc05d84 Request placeholders: extract query argument (#1478)
* feat(request placeholders): adds {?arg}

* test(request placeholders): test query argument extractor {?arg}
2017-03-02 23:25:28 -07:00
ericdreeves
0a0d2cc1cf Use RequestURI when redirecting to canonical path. (#1331)
* Use RequestURI when redirecting to canonical path.

Caddy may trim a request's URL path when it starts with the path that's
associated with the virtual host. This change uses the path from the request's
RequestURI when performing a redirect.

Fix issue #1327.

* Rename redirurl to redirURL.

* Redirect to the full URL.

The scheme and host from the virtual host's site configuration is used
in order to redirect to the full URL.

* Add comment and remove redundant check.

* Store the original URL path in request context.

By storing the original URL path as a value in the request context,
middlewares can access both it and the sanitized path. The default
default FileServer handler will use the original URL on redirects.

* Replace contextKey type with CtxKey.

In addition to moving the CtxKey definition to the caddy package, this
change updates the CtxKey references in the httpserver, fastcgi, and
basicauth packages.

* httpserver: Fix reference to CtxKey
2017-02-28 05:54:12 -07:00
Matt Holt
82cbd7a96b Detect HTTPS interception (#1430)
* WIP: Implement HTTPS interception detection by Durumeric, et. al.

Special thanks to @FiloSottile for guidance with the custom listener.

* Add {{.IsMITM}} context action and {mitm} placeholder

* Improve MITM detection heuristics for Firefox and Edge

* Add tests for MITM detection heuristics

* Improve Safari heuristics for interception detection

* Read ClientHello during first Read() instead of during Accept()

As far as I can tell, reading the ClientHello during Accept() prevents
new connections from being accepted during the read. Since Read() should
be called in its own goroutine, this keeps Accept() non-blocking.

* Clean up MITM detection handler; make possible to close connection

* Use standard lib cipher suite values when possible

* Improve Edge heuristics and test cases

* Refactor MITM checking logic; add some debug statements for now

* Fix bug in MITM heuristic tests and actual heuristic code

* Fix gofmt

* Remove debug statements; preparing for merge
2017-02-17 14:07:57 -07:00
Augusto Roman
dc3efc939c Add request placeholder support for querying request cookies. (#1392)
* Add request placeholder support for querying request cookies.

This adds the ability to query the request cookies for placeholders
using the syntax "@cookiename".

For example, this would allow rewriting based on a cookie:
  rewrite {
    if @version is 'dev'
    to /dev/index.html
  }

* Switch cookie special char from @ to :

* Switch special char for cookies from : to ~
2017-02-15 21:59:24 -07:00
Toby Allen
bbee961415 Introduce new Replacer fields {rewrite_path}, {rewrite_path_escaped}; issue #1185 (#1364)
* Fix #1185

* Return normal path if no rewrite has happened

* Revert change, not required

* Updated tests
2017-01-23 22:15:27 -07:00
M-A
8464020f7c Add {whenISO} to record timestamp in ISO 8601 format in UTC. (#1353)
* Add {whenISO} to record timestamp in ISO 8601 format in UTC.

ISO 8601 is the standard time format and is easy to parse.

This change assumes users desiring ISO 8016 generally prefer UTC for simplicity.
This results in {whenISO} to be significantly shorter than {when}:
{when}    = "02/Jan/2006:15:04:05 +0000"
{whenISO} = "2006-01-02T15:04:12Z"

Add unit test to verify both, as there was no unit test for {when}.

* Rename {whenISO} to {when_iso}
2017-01-14 15:54:27 -07:00
Ngo The Trung
0cdaaba4b8 Add maxrequestbody directive (#1163) 2016-11-04 08:25:49 +08:00
Aish Raj Dahal
733f622f7a
Add new placeholder for latency in milliseconds 2016-10-05 21:06:15 -07:00
Tw
99a6b2db67 replacer: evaluate header placeholder when replacing
fix issue #1137

Signed-off-by: Tw <tw19881113@gmail.com>
2016-09-28 19:32:16 +00:00
Matthew Holt
a4d70262aa
Use strings.Contains instead of IndexOf for readability 2016-09-24 12:09:28 -06:00
Tw
590862a962 replacer: capture request body normally
fix issue #1015

Signed-off-by: Tw <tw19881113@gmail.com>
2016-08-23 08:20:49 +08:00
Tw
40c09d6789 replacer: code refactor
Signed-off-by: Tw <tw19881113@gmail.com>
2016-08-23 08:20:49 +08:00
Tw
bba1059ef9 log: add log request body test
Signed-off-by: Tw <tw19881113@gmail.com>
2016-08-23 08:20:49 +08:00
Matt Holt
ac0dd303be Merge branch 'master' into log-request-body 2016-08-11 17:36:09 -06:00
Carter
676202a31e Fixed styling and byte count 2016-08-11 19:08:49 -04:00
Matthew Holt
c8a99d2f81
Don't use X-Forwarded-For for {remote} placeholder (closes #1025) 2016-08-11 16:54:17 -06:00
Carter
532ab661c7 Fully read and close the request body 2016-08-11 07:03:14 -04:00
Carter
d56ac28bec Using a LimitReader and fixed test and log format. 2016-08-10 22:43:26 -04:00
Carter
3fd8218f67 refactor and added test 2016-08-10 11:04:37 -04:00
Carter
d06c15cae6 Set the request body to a new ReadCloser 2016-08-10 10:36:16 -04:00
Carter
59b1e8b0bc Now logging the request body
Logging the request body if the Content-Type is application/json or
application/xml
2016-08-10 10:04:57 -04:00
Carter
4d76ccb1c4 Rounding the latency in certain scenarios (#1005)
* Rounding the latency in certain scenarios

* run gofmt
2016-08-08 10:14:53 -06:00
Tw
beae16f07c Proxy performance (#946)
* proxy: add benchmark

Signed-off-by: Tw <tw19881113@gmail.com>

* replacer: prepare lazily

update issue#939

benchmark            old ns/op     new ns/op     delta
BenchmarkProxy-4     83865         72544         -13.50%

Signed-off-by: Tw <tw19881113@gmail.com>

* proxy: use buffer pool to avoid temporary allocation

Signed-off-by: Tw <tw19881113@gmail.com>
2016-07-20 19:06:14 -06:00
Matthew Holt
227664336e Misc. changes: {hostonly} placeholder, self_signed port fix 2016-07-02 14:11:17 -06:00
Abiola Ibrahim
b1cd0bfeff Support for placeholders in fastcgi env vars. 2016-06-29 13:41:52 +01:00
Gustavo Chaín
6c847d0723 New {request} placeholder to log entire requests (sans body) (#871)
Add a {request} placeholder to the replacer.

Closes #858.
2016-06-07 11:06:24 -06:00
Matthew Holt
ac4fa2c3a9
Rewrote Caddy from the ground up; initial commit of 0.9 branch
These changes span work from the last ~4 months in an effort to make
Caddy more extensible, reduce the coupling between its components, and
lay a more robust foundation of code going forward into 1.0. A bunch of
new features have been added, too, with even higher future potential.

The most significant design change is an overall inversion of
dependencies. Instead of the caddy package knowing about the server
and the notion of middleware and config, the caddy package exposes an
interface that other components plug into. This does introduce more
indirection when reading the code, but every piece is very modular and
pluggable. Even the HTTP server is pluggable.

The caddy package has been moved to the top level, and main has been
pushed into a subfolder called caddy. The actual logic of the main
file has been pushed even further into caddy/caddymain/run.go so that
custom builds of Caddy can be 'go get'able.

The HTTPS logic was surgically separated into two parts to divide the
TLS-specific code and the HTTPS-specific code. The caddytls package can
now be used by any type of server that needs TLS, not just HTTP. I also
added the ability to customize nearly every aspect of TLS at the site
level rather than all sites sharing the same TLS configuration. Not all
of this flexibility is exposed in the Caddyfile yet, but it may be in
the future. Caddy can also generate self-signed certificates in memory
for the convenience of a developer working on localhost who wants HTTPS.
And Caddy now supports the DNS challenge, assuming at least one DNS
provider is plugged in.

Dozens, if not hundreds, of other minor changes swept through the code
base as I literally started from an empty main function, copying over
functions or files as needed, then adjusting them to fit in the new
design. Most tests have been restored and adapted to the new API,
but more work is needed there.

A lot of what was "impossible" before is now possible, or can be made
possible with minimal disruption of the code. For example, it's fairly
easy to make plugins hook into another part of the code via callbacks.
Plugins can do more than just be directives; we now have plugins that
customize how the Caddyfile is loaded (useful when you need to get your
configuration from a remote store).

Site addresses no longer need be just a host and port. They can have a
path, allowing you to scope a configuration to a specific path. There is
no inheretance, however; each site configuration is distinct.

Thanks to amazing work by Lucas Clemente, this commit adds experimental
QUIC support. Turn it on using the -quic flag; your browser may have
to be configured to enable it.

Almost everything is here, but you will notice that most of the middle-
ware are missing. After those are transferred over, we'll be ready for
beta tests.

I'm very excited to get this out. Thanks for everyone's help and
patience these last few months. I hope you like it!!
2016-06-04 17:00:29 -06:00