Do not expand errorpage %codes injected by X509 certificates#2416
Do not expand errorpage %codes injected by X509 certificates#2416rousskov wants to merge 14 commits intosquid-cache:masterfrom
Conversation
This WIP commit records a failed attempt to solve the problem by
exposing ErrorState::compileLeadingCode() code to Security::ErrorDetail.
This code does not compile, but that is easy to fix. The primary changes
committed here already show many XXXs that would be difficult to
address. I hope to find a better solution. The problem description below
should still apply, and even the high-level solution analysis below may
apply.
----
When available, Security::ErrorDetail is added to Squid-generated errors
that use customizable errorpage formats containing `%D` errorpage
`%code`. Security::ErrorDetail itself supports customizable verbose
reporting format (see `detail` in `errors/templates/error-details.txt`).
ErrorDetail format may contain both ErrorDetail-specific %codes and
generic legacy errorpage %codes handled by ErrorState.
To support the above "nested" formats when processing `%D`,
ErrorState::compileLegacyCode() compiled ErrorDetail output,
substituting any legacy errorpage %codes outputted by ErrorDetail. That
re-compilation could not distinguish a `%code` sequence configured by
`error-details.txt` from a `%code` sequence that came from, say, a
received X509 certificate field. Both would be expanded!
This bug applies to both legacy errorpage `%code` sequences like `%R`
and modern `@Squid{logformat %code}` errorpage sequences.
X509 certificate fields are the only known injection vector, but it is
conceivable that some other error details may contain character
sequences that match errorpage legacy %codes (e.g., SysErrorDetail
returns strerror(3) output that Squid does not control and did not
escape.
This bug is similar to a bug fixed in 2018 commit 6feeb15, but this one
deals with injected errorpage %codes rather than HTML tags.
Two primary solution candidates were considered:
A) Escape-then-compile-to-unescape: Security::ErrorDetail could escape
`%code` sequences received from "external" sources, so that
ErrorState::compile() would see `%%X` and replace that with `%X`,
correctly relaying received `%X`. This solution was rejected because
we risk forgetting to escape some input, now or during code
refactoring, especially since the risk applies to all ErrorDetail
classes. Also, escaping what we know we will immediately un-escape is
wasteful.
B) Never compile "external" input. This implemented solution required
giving ErrorDetail::verbose() access to the `%code` compiling code in
ErrorState, so that ErrorDetail can decide what to compile and what
not to. Now, Security::ErrorDetail only compiles `%code` sequences
found in `detail`; ErrorState does not compile ErrorDetail output.
Side effects
------------
The `details` field in `error-details.txt` now supports modern
`@Squid{logformat %code}` errorpage sequences, addressing an old TODO.
The `descr` field in `error-details.txt` no longer supports `%code`
sequences. It was never documented to support them. It is not meant to
contain such sequences -- they belong to the `details` field. Hopefully,
no deployed configurations use `%code` sequences in that field.
Removed the compilation loop from Security::ErrorDetail::verbose(). The old ErrorState::compile() loop is now responsible for calling Security::ErrorDetail for parsing custom %codes. Besides out-of-scope legacy const-correctness problems and basic build fixes, we need to resolve one new API problem: An ErrorDetail::verbose() caller supplies build.output, but the method returns its output instead of updating the supplied field. The solution may affect verbose() API, so it blocks those "basic build fixes".
ErrorDetail::verbose() does not need access to Build. Unlike internal ErrorPage methods, current ErrorDetail objects do not need to parse portions of error templates, engaging multiple portion-specific parsers. ErrorDetail::verbose() API is much simpler -- "dump details" (into the returned SBuf value). The only wrinkle here is that Security::ErrorDetail needs ErrorState compilation abilities to do that, but that can be handled by adding an ErrorState parameter. And since ErrorState objects already have `request` data members, we can drop the existing HttpRequest parameter. TODO: ErrorDetail::verbose() may benefit from switching to an std::ostream-based API, but doing so now would increase out-of-scope noise. It is probably best done together with upgrading ErrorState "compilation" methods to use std::ostream.
ld: errorpage.o: warning: relocation against `err_type_str'
in read-only section `.text'
src/errorpage.cc:276: undefined reference to `err_type_str'
src/errorpage.cc:631: undefined reference to `err_type_str'
...
Branch changes added forward declarations for classes declared
inside an ErrorPage namespace. The AWK script incorrectly applied
that namespace to the generated err_type_str definition.
This surgical fix does not cover all possible namespace-related
variations. The correct fix is to stop parsing C++ using AWK,
at least as far as namespaces are concerned: The script caller
knows what namespace(s) must be used for the container definition.
... to verbose() methods that did not use HttpRequest.
|
The work on this fix was triggered by @jro-calif report implications. Here is a diff between Squid error response generated by official and PR code (when the origin server is using a bogus x509 certificate field): <pre>[No Error] (TLS code: SQUID_X509_V_ERR_DOMAIN_MISMATCH+broken_cert)</pre>
- <p>Certificate does not match domainname: /OU=28/Apr/2026:10:28:39 -0400s.%03tu 0 squid/8.0.0-VCSsl_ca_name %>a ... %<a [not available]t/O=O1</p>
+ <p>Certificate does not match domainname: /OU=%ts.%03tu @Squid{%6tr} %ssl_ca_name %>a ... %<a %mt/O=O1</p> |
rousskov
left a comment
There was a problem hiding this comment.
These comments do not request any PR changes.
|
|
||
| SBuf | ||
| Ftp::ErrorDetail::verbose(const HttpRequest::Pointer &) const | ||
| Ftp::ErrorDetail::verbose(const ErrorTemplateCompiler &) const |
There was a problem hiding this comment.
It is possible to avoid noisy changes like this one in this PR by adding a temporary API shim. I have not done that because these changes do not impact v7 backporting in my quick-and-dirty tests (cherry-picking fails because v7 lacks some src/security/ErrorDetail.cc changes in master/v8), and because we would have to post another official PR to remove that shim, of course.
|
|
||
| for (const auto &detail: request->error.details) { | ||
| mb.appendf("%i-Error-Detail-Brief: " SQUIDSBUFPH "\r\n", scode, SQUIDSBUFPRINT(detail->brief())); | ||
| mb.appendf("%i-Error-Detail-Verbose: " SQUIDSBUFPH "\r\n", scode, SQUIDSBUFPRINT(detail->verbose(*err))); |
There was a problem hiding this comment.
Disclaimer: I have not tested this FTP code path.
There was a problem hiding this comment.
Please do. It should be just a matter of running a few FTP requests to a non-FTP server and looking at the cache.log level 11,2 entries for those messages.
kinkie
left a comment
There was a problem hiding this comment.
LGTM, also looks cleaner than current state
| // below, adjust compile*() methods to avoid ErrorState modifications. | ||
|
|
||
| auto blockStart = build.input; | ||
| while (const auto letter = *build.input) { |
There was a problem hiding this comment.
Francesco: also looks cleaner than current state
Yes, albeit slightly. This change removes one (poor) duplicate of this "find and replace all %codes" loop. It also provides better access to detail-rendering context via the ErrorTemplateCompiler parameter; I expect future code to use that access while rendering additional error details.
A lot of work is still required to modernize legacy errorpage code, of course, including addressing const-correctness problems marked in this PR.
There was a problem hiding this comment.
Looks can be deceiving. The order of refactoring is causing regressions and an increase in technical debt.
A better approach would be to separate the refactoring from the bug fix. The former needs a lot more work, the latter can accept workarounds (wit TODO notes) for design issues.
When available, Security::ErrorDetail is added to Squid-generated errors
that use customizable errorpage formats containing `%D` errorpage
`%code`. Security::ErrorDetail itself supports customizable verbose
reporting format (see `detail` in `errors/templates/error-details.txt`).
The latter format may contain both Security::ErrorDetail-specific %codes
and generic legacy errorpage %codes handled by ErrorState.
To support the above "nested" formats when processing `%D`,
ErrorState::compileLegacyCode() compiled ErrorDetail output,
substituting any legacy errorpage %codes outputted by ErrorDetail. That
re-compilation could not distinguish a `%code` sequence configured by
`error-details.txt` from a `%code` sequence that came from, say, a
received X509 certificate field. Both would be expanded!
This bug applies to both legacy errorpage `%code` sequences like `%R`
and modern `@Squid{logformat %code}` errorpage sequences.
X509 certificate fields are the only known injection vector, but it is
conceivable that some other error details may contain character
sequences that match errorpage legacy %codes (e.g., SysErrorDetail
returns strerror(3) output that Squid does not control or escape.
This bug is similar to the bug fixed in 2018 commit 6feeb15 but deals
with injected errorpage %codes (that target Squid error response
assembling code) rather than injected HTML tags (that target browsers).
Two primary solution candidates were considered:
A) Escape-then-compile-to-unescape: Security::ErrorDetail could escape
`%code` sequences received from "external" sources, so that
ErrorState::compile() would see `%%X` and replace that with `%X`,
correctly relaying received `%X`. This solution was rejected because
we risk forgetting to escape some input, now or during code
refactoring, especially since the risk applies to all ErrorDetail
classes. Also, escaping what we know we will immediately un-escape is
wasteful.
B) Never compile "external" input. This implemented solution required
giving ErrorDetail::verbose() access to the `%code` compiling code in
ErrorState, so that ErrorDetail can decide what to compile and what
not to. Now, Security::ErrorDetail only compiles `%code` sequences
found in `detail`; ErrorState does not compile ErrorDetail output.
Side effects
------------
The `details` field in `error-details.txt` now supports modern
`@Squid{logformat %code}` errorpage sequences by design rather than by
accident, addressing an old (and essentially misleading) TODO inside
Security::ErrorDetail::convertErrorCodeToDescription().
The `descr` field in `error-details.txt` no longer supports errorpage
`%code` sequences. It was never documented to support them, and it did
not support Security::ErrorDetail %codes like %ssl_ca_name. The field is
not meant to contain %code sequences -- they belong to the `details`
field that specifies detail reporting format. Hopefully, no deployed
configurations use `%code` sequences in the `descr` field.
| class ErrorState; | ||
|
|
||
| namespace ErrorPage { | ||
| class PercentCodeCompiler; |
There was a problem hiding this comment.
Why "PercentCode" ? we dont have other types of page compiler for error pages.
| class PercentCodeCompiler; | |
| class ErrorPageCompiler; |
I do not see the required .h/.cc being added for this new class. Please keep the components in separate build units to avoid monolithic unit and dependency loop issues.
| { | ||
| public: | ||
| using Pointer = ErrorDetailPointer; | ||
| using ErrorTemplateCompiler = ErrorState; |
There was a problem hiding this comment.
"Error" word is redundant within the scope ErrorPage::
| using ErrorTemplateCompiler = ErrorState; | |
| using TemplateCompiler = ErrorState; |
| /// \sa compileDetail() | ||
| SBuf compile(const char *input, bool building_deny_info_url, bool allowRecursion); | ||
|
|
||
| void compile(Build &build) const; |
There was a problem hiding this comment.
Redundant parameter and missing documentation on new method.
| void compile(Build &build) const; | |
| void compile(Build &) const; |
| { | ||
| Assure(build.input); | ||
|
|
||
| // TODO: Instead of violating const-correctness with const_cast<ErrorState*> |
There was a problem hiding this comment.
Indeed please do that. I suggest looking into whether the member holding output be mutable would avoid a lot of the issues.
| err_type templateCode; ///< The internal code for this template. | ||
| }; | ||
|
|
||
| namespace ErrorPage { |
There was a problem hiding this comment.
This move to src/ is a regression. Please leave these classes in the error/ and include their .h files where needed.
Yes the badly named class Error causes namespace issues. If you are going to insist on the huge amount of code polish and redesign added by this PR (beyond the actual needed bug fix), then either do ont use a namespace around the classes or fix that class Error naming as well (I suggest the former).
| # XXX: We should remember the name(s) of the namespace(s) surrounding the enum | ||
| # instead. TODO: Replace this C++ parsing hack with a command-line parameter. | ||
| /^namespace *[a-zA-Z]+/ { | ||
| if (type) next |
There was a problem hiding this comment.
This change is out of scope and seems to be adding support for invalid C++ syntax:
enum Foo {
namespace Blah
{
...
}
};
Please remove, or if necessary please discuss the error being produced that requires a change.
| // below, adjust compile*() methods to avoid ErrorState modifications. | ||
|
|
||
| auto blockStart = build.input; | ||
| while (const auto letter = *build.input) { |
There was a problem hiding this comment.
Looks can be deceiving. The order of refactoring is causing regressions and an increase in technical debt.
A better approach would be to separate the refactoring from the bug fix. The former needs a lot more work, the latter can accept workarounds (wit TODO notes) for design issues.
|
|
||
| for (const auto &detail: request->error.details) { | ||
| mb.appendf("%i-Error-Detail-Brief: " SQUIDSBUFPH "\r\n", scode, SQUIDSBUFPRINT(detail->brief())); | ||
| mb.appendf("%i-Error-Detail-Verbose: " SQUIDSBUFPH "\r\n", scode, SQUIDSBUFPRINT(detail->verbose(*err))); |
There was a problem hiding this comment.
Please do. It should be just a matter of running a few FTP requests to a non-FTP server and looking at the cache.log level 11,2 entries for those messages.
| class ErrorDetail; | ||
| class ErrorState; | ||
|
|
||
| namespace ErrorPage { |
There was a problem hiding this comment.
Wrong namespace. This is the forward declarations for the errors/liberror.la library which is supposed to use namespace Error.
The maintenance work fixing class Error conflicts is stalled in backlog. For now please use Error prefix on classes that should be in the namespace Error.
When available, Security::ErrorDetail is added to Squid-generated errors
that use customizable errorpage formats containing
%Derrorpage%code. Security::ErrorDetail itself supports customizable verbosereporting format (see
detailinerrors/templates/error-details.txt).The latter format may contain both Security::ErrorDetail-specific %codes
and generic legacy errorpage %codes handled by ErrorState.
To support the above "nested" formats when processing
%D,ErrorState::compileLegacyCode() compiled ErrorDetail output,
substituting any legacy errorpage %codes outputted by ErrorDetail. That
re-compilation could not distinguish a
%codesequence configured byerror-details.txtfrom a%codesequence that came from, say, areceived X509 certificate field. Both would be expanded!
This bug applies to both legacy errorpage
%codesequences like%Rand modern
@Squid{logformat %code}errorpage sequences.X509 certificate fields are the only known injection vector, but it is
conceivable that some other error details may contain character
sequences that match errorpage legacy %codes (e.g., SysErrorDetail
returns strerror(3) output that Squid does not control or escape.
This bug is similar to the bug fixed in 2018 commit 6feeb15 but deals
with injected errorpage %codes (that target Squid error response
assembling code) rather than injected HTML tags (that target browsers).
Two primary solution candidates were considered:
A) Escape-then-compile-to-unescape: Security::ErrorDetail could escape
%codesequences received from "external" sources, so thatErrorState::compile() would see
%%Xand replace that with%X,correctly relaying received
%X. This solution was rejected becausewe risk forgetting to escape some input, now or during code
refactoring, especially since the risk applies to all ErrorDetail
classes. Also, escaping what we know we will immediately un-escape is
wasteful.
B) Never compile "external" input. This implemented solution required
giving ErrorDetail::verbose() access to the
%codecompiling code inErrorState, so that ErrorDetail can decide what to compile and what
not to. Now, Security::ErrorDetail only compiles
%codesequencesfound in
detail; ErrorState does not compile ErrorDetail output.Side effects
The
detailsfield inerror-details.txtnow supports modern@Squid{logformat %code}errorpage sequences by design rather than byaccident, addressing an old (and essentially misleading) TODO inside
Security::ErrorDetail::convertErrorCodeToDescription().
The
descrfield inerror-details.txtno longer supports errorpage%codesequences. It was never documented to support them, and it didnot support Security::ErrorDetail %codes like %ssl_ca_name. The field is
not meant to contain %code sequences -- they belong to the
detailsfield that specifies detail reporting format. Hopefully, no deployed
configurations use
%codesequences in thedescrfield.