It is a truism in the application security world (or really any security) that in order to have an effective Security Program you need executive buyoff and executive support. To many this means that you need executives to care about security, be willing to fund it, and be willing to stand behind their security personel. This is certainly true, but there is a precursor step that is necessary for many development organization to move to truly institute effective Application Security – they need to mature as a software organization and embrace process.

When most software organizations start out they often embrace a seat of the pants development style, something often mistaken this as agile development, which has three very simple steps – Have Idea, Code Idea, Market Idea. As they mature they might realize that they should decide if they can sell the idea before they try to code it, rather than figure that out after the fact, and they decide they need to bring in testers to verify that they actually did code the idea. As a dev org matures they get to a platue were they have developed some process, but it resides on either side of the developers- they actually do analysis of an idea to see if it is worth the development costs, and they do testing and customer acceptance verification to make sure that the idea was implemented in a way that will meet customer’s needs, but at the end of the day all they expect developers to do is write code and testers to make sure the code does what it is supposed to.

The problem is, that is an arrangement that will not consistently produce robust code- security wise or otherwise. Verifying functionality only finds average use case flaws, but does not verify proper behavior across all concievable uses, and simply writing code without discipline will produce unmaintainable, fragile, redundant code. It is in this state that organizations start to care about security, but they aren’t willing to make real commitments to the changes necessary to see it happen. At this point they need to decide whether or not they are willing to change their development and test culture, and this is a very difficult decision to make and enforce.

If the decision is not made, the organization will trap themselves into an entirely reactive model to security threats- Problems are found by vulnerability assessment or customer reporting and fixed- tribal knowledge is gained among some developers and there is a short term downtick in that specific type of vulnerability among those specific developers- long term the org hasn’t learned much, and the information is not sustained. Almost no prevention will occur. This is because a preventitive model would require changes to how the code is written and verified.

If the decision is made, then the development team needs process placed on them and accountability to following that process. They can’t just write code anymore- they need coding standards detailing how they can write code and they need to get used to writing documentation. Developers hate writing documentation, but it is one of the most important things they can do- with documentation they can fully think through how they will code a feature before they start doing so (we don’t start pouring foundation before we decide on the blueprints for a house after all) spotting shortcomings of a particular method before they have spent a couple of days implementing it and they will provide something were the design itself can be tested rather than just the implementation. On the topic of testing, the dev org needs to get in the habit of hiring testers who are very skilled at thinking in terms of finding problems, rather than testers who are little more than stupid users that can determine whether a feature either works or doesn’t. Skilled testers are worth as much as a skilled developers, as effectively finding technical problems is just as hard as effectively coding solutions (one of the best interview test questions I have ever heard was “What do you think of your cell phone?”- a true tester may describe a couple of things they like, but then proceed to talk about the myriad of things that they wish were done better. A true tester is someone who perpetually sees the flaws in something, no matter how good it is). The testers also need to be given a strong voice- they are the verification for the developers and if their voice is not strong, then they can’t serve their purpose.

Doing this will give the security team a framework to then build in security controls into the development process. They need an actual process in place, otherwise they have no foundation to build off of. An existing and well defined development process is an input into any effective secure development lifecylce, and an SDL is ultimately the path to a preventitive approach to security. The problem is that it is very hard to change a development culture and it requires commitment from the top. Developers hate process, and they are going to be very resistant to implementing it. Executives need to make it abundandly clear that conforming to the process is the developers’ job- it is what they will be paid to do, and if they don’t do it they will not be paid. This will make no friends among the developers. That is why it is the hardest decision that a maturing development organization can make.

Incidentally, the same thing is true for OSS as for proprietary, however it is even more difficult to implement and enforce in an open distributed development environment. The payoff for either software is much more than just security- the overall quality of the product will improve as a result and the code will be easier to maintain as their will be documentation to refer to and a level of code quality enforced.

~ Joshbw

CGI Security via AppSec Notes complains that Netflix has not fixed all of their CSRF vulnerabilities. You can no longer access account information, billing information, change shipping address, or anything of value, but you can still add movies to someone’s queue. This apparently still bothers the author who has a note of annoyance that Netflix hasn’t completely fixed everything yet. I think this loses sight of realistic business goals of security- from an enterprise perspective one addresses security vulnerabilities in order to protect revenue or prevent damage. It is a cost benefit analysis, weighing whether allocating resources to address a particular vulnerability allows the greatest capitalization of those resources.

I’d posit that Netflix does not believe preventing CSRF attacks that add movies to the top of a queue to be the most effective use of those development resources. When looking at the impact and likelihood of this sort of CSRF attack the associated risk comes out quite low:

Impact- no direct loss of capital or resources, some level of brand damage. Some customers may very well be so pissed over getting sent some embarrassing video that they leave completely, which is a drop in revenue, however I would posit that these people are probably the minority. More people are likely to be pissed, complain, and probably get a free month as a token apology (loss of 1/12 of the yearly revenue for those people) but complaining smacks of efforts so I don’t imagine that this number will be horribly large either. Then there is the group that will be pissed and do nothing, no appreciable loss. Next is the group that is confused because they didn’t order that particular embarrassing video, no appreciable loss. Finally there is the outlier group that is hard to predict- the group where a couple gets an embarrassing movie, gets in a fight, and seperates over a movie (I live in a town where two middle aged men shot each other because of an argument over who parked their oversized SUV too close to the other oversized SUV, so I fully imagine there are people who would overreact to a movie). This might be a loss of a subscription, maintain the status quo, or a gain of a subscription because now the newly separated couple both wants a subscription, whereas they accounted for only one before.

There is also brand damage should this get media coverage once it is exploited. That is a hard thing to put dollars on.

Likelihood- For this to be exploited a customer needs to visit another website that actually carries out the CSRF attack. For that to happen the other website would need to choose actively to be malicious to Netflix instead of carrying out CSRF against a website that would allow monetary gain- there is no personal gain in exploiting this, only vandalism (which isn’t quite the correct word, but in the same spirit). The visitor needs to be logged into Netflix, and they need to be people who don’t frequently manage their queues otherwise they would notice the movie prior to it being sent (and may still react badly, but I posit that even fewer people would if they caught the movie before it shipped). Of Netflix’s 10 million subscribers I would assert that the number that would fall victim to this is small enough to be indistinguishable from zero.

Let’s look at the likely attacker- this isn’t organized crime, or any accomplished Blackhat. Those people are after money. This hypothetical attacker is likely a malcontent teenager who thinks it is funny to screw with a Netflix queue. That mostly rules out the attacker owning their own site that gets enough traffic to exploit this from, and also mostly rules out the attacker knowing of XSS or SQLi flaws in websites that do get enough traffic for this to exploit this from. It leaves their dinky website that pretty much only gets visited by their friends on IRC (or whatever script kiddies use these days)- probably some random blog on blogspot or something similar. In other news this will be exploited from a location that will not attract any noticeable number of Netflix subscribers.

So the likelihood is pretty nonexistent, the impact is very low, and thus the risk of losing subscribers because they were exploited is essentially 0. The risk of brand damage from press coverage isn’t much higher- someone in the press needs to find out about this flaw, grok what the heck the flaw really is, actually have a victim so they can tell a story about, and then compete for airtime with Company X is laying of 50,000 employees and congress is too incompetent to fix things. I predict the tampered roadsigns warning about zombies gets more press coverage. Finally the viewers need to care. TJ Max is our security story, biggest public data loss to date (with Heartland possibly taking the crown), and anecdotally the people I know who shop there weren’t even aware of it, and upon me explaining it STILL DIDN’T CARE. Netflix subscribers are a slightly different demographic, but I don’t see people getting really worked up about hearing that maybe they might be sent the wrong movie.

So why should Netflix fix this problem, instead of working on features that may attract more customers? The people who found this flaw care because it is their flaw- it matters because it is theirs. Unfortunately that doesn’t sway companies. For any vulnerability that is found it isn’t enough to point out the vulnerability- a business case also needs to be made for addressing the vulnerability. That is the reality of enterprise security- everything is cost/benefit analysis. If the risk is low, the response will reflect that. I’d love to see all vulnerabilities get addressed as much as the next security professional, but that is idealism rather than realism talking.

~ Joshbw

Giblets aren’t just the disgusting parts of the turkey, the foul (or fowl if you are no stranger to puns) organs that your grandma likes to try to fool you into eating for her own perverse amusement. In certain parlance they also refer to the external code that your applications is dependent on- source code for which you have little or no control. Giblets are typically libraries, assemblies, jars, or other such modules that a third party, whether it is an external company (for example, the excellent looking Elegant Ribbon), another internal team, a vendor, etc. wrote. They are also a huge security concern as you usually have little insight into what security was incorporated in the giblets’ development cycle, nor a great deal of control when vulnerabilities are discovered in the giblet. You can make your own code rock solid, but still find your application exploited because of Giblet code. MS Office has been hit by this a number of times, as legacy and third party file filters continue to be vulnerable despite the dramatic re-engineering of the Office file formats (.docx, .xlsx, etc) with security in mind.

Recently my colleagues and I have been playing around with several different HTML Encoders in Java. Despite what OWASP has posted on the subject for an international enterprise it is not as simple as encoding everything that isn’t A-Z,a-z,0-9 (though I don’t think OWASP is making any claim at the robustness of their recommendation). Having all of your Chinese radicals all of a sudden converted to full character encoded values is going to dramatically increase page size, especially on anything like a blog or other large text field with copious blocks of user input that gets echoed to the screen. Most people don’t want to figure out how to implement their own character set aware, internationalized Entity Encoder, and so naturally prefer to use canned solutions. The desire to use a canned library is by no means unique to HTML Entity Encoding; for any number of functions this makes sense, since many functions take specific expertise (do you think your average PHP programmer wants to worry about image manipulation implementations for example). However in this specific case it easily demonstrates why Giblets are a big fat bag of security suck. What happens when there is a flaw in the Encoder giblet? You really don’t know what the inside of it looks like- you don’t know how well written it is. You trust that it works, but if your output encoder isn’t encoding all output properly your defense against XSS is now vulnerable.

Really any giblet that processes or transforms external data is a point of concern. To some extent open source giblets can be better, as you can at least look inside and see how well written something is (though I suspect that the number of people who do so are probably in the minority), but for enterprises this can actually be a liability, as legal departments worry about license compliance and IP taint. For all other Giblets, there are a couple of recommendations to make you safer -

1) Ask for details about the Giblet Developer’s secure development lifecycle, vulnerability history, mean time to patch, and other useful behavioral questions about their security posture. It might be worth going with a giblet that has a bit less functionality if the security posture of the developers seems higher.

2) External, Vendor, or Internal, get an SLA in place that clearly defines obligations. Include obligations to patch vulnerabilities of various criticality ratings, time frames for said patches, agreed upon criticality scales (do you consider SQLi to be critical while they think it high?), and maintenance length (if a vulnerability is found five years after you purchased the giblet, are you out of luck getting a fix). This is often overlooked when working with internally developed code from other teams; however just as with external developers your security vulnerability may not be the internal teams priority. You need a commitment to address vulnerabilities in a timely fashion.

For non-commercial giblets, especially open source, you probably won’t get an SLA, so explore the ramifications and practicality of fixing the flaw yourself (see afore mentioned worry about license compliance and IP Taint). Non-commercial, non open source giblets (freeware) are very dangerous as you have no assurance concerning timely fixes, nor the capacity to fix them yourself.

3) Try to validate/sanitize all data before passing it to a giblet anyway, even if the Giblet developer has a great security posture and a very aggressive SLA. An added layer of defense is never a bad thing.

4) If the Giblet is consumed by a client, make sure any agreement with the Giblet developer includes the rights to distribute updated versions to the client.

Most importantly though, just keep in mind that third party code is a risk. Security should be as much a consideration as features when incorporating third party code and all of the security effort you put in may be for naught if your 3rd party code is vulnerable. When I draw threat models I like to label Giblets as External Entities even if they are processes or libraries consumed on the same box, as I have no control over them. It just helps to draw out the ramifications of their use.

~ Joshbw

In my last post, on threat modeling, I had mentioned that there would be a follow up post would talk about the issues to authenticating the communication from the server. My cohort and fellow Josh has been working with me on some side projects with local small and mid-size business to secure their applications (things like auto dealership websites that collect information for credit checks) to offset all that money our investment accounts have taken in the past 18 months with some extra income. In that process we have noticed several issues that keep coming up that make it harder to be confident we are talking to the application we think we are talking to.

In this day and age phishing is so easy because it is hard to authenticate the server to the client. There isn’t end to end trust. Ultimately the user only has a handful of verification methods- does the website visually look like the website they want to connect to, does the URL look like the correct domain, and is the communication over a secure channel, none of which is going to be obvious to non-technical users. The visual appearance is incredibly easy to spoof, to the point where only the most lazy phisher won’t put forth the effort. Short of sternography, the visual appearance can’t be trusted, and sternography isn’t a viable option. So, here are a couple of things you can do outside of the visual appearance.

- Don’t use multiple domain names. You can register multiple domains, but always have them redirect to one domain, and never advertise anything but that one domain. This prevents dilution of the domain name, and trains the user to only trust one domain. Don’t use BobsFordMazdaLincoln.com and BobsMazdaLincolnFord.com, because a phisher can set up bobsLincolnMazdaFord.com or similar name. This is obviously a contrived example, but plenty of companies have a variety of domains for different purposes (Microsoft is really bad about this, and with their live ID being an SSO for all of their domains it is easy for me to phish that credential to great payoff).

It is better to use sub-domains if you have separate sites associated with one company. It is also better to try and have a simple and short domain name, without too many words in it. bobscars.com is way better than BobsMazdaLincolnFord.com because it is much harder to create a variant that fools the users. Bob could then have ford.bobscars.com, mazda.bobscars.com, etc (and this scales if bob ever opens other dealerships). Also, users are more likely to remember your domain name and it will have fewer common mistype permutations (which you should buy, domains are cheap) if it is simple (something for which I am probably guilty with analyticalengine.net, a url that even I mistype occasionally).

- Don’t go crazy with sub domains. You should probably only go one (or very rarely two) deep. mazda.bobscars.com is pretty simple to visually parse as being part of bobscars.com, but year2009.newcars.mazda.fordcorp.bobscars.com is going to be very easy to visually spoof to end users. We need to train customers to look at the domain name, and in order to do that it needs to be as apparent as possible. IE 8 does a good job calling this out, and there is a similar firefox plugin to highlight the domain name, but with many people still using IE 6 we need to focus on getting them to visually parse and recognize the domain (it would also be nice to get them to upgrade to a new browser). If we go overboard with subdomains we make the domain non-obvious and thus make it easy to spoof.

- Use SSL liberally, and use EV certs. The use of EV certs provides visual evidence in the location bar of the browser that this really is the corporation I intend to communicate with. It is one more protection that allows the user to visual identify the site. It offers more than that though, as it can attest to the integrity of the communication back and forth between the site. Afore mentioned cohort Josh has a post on his blog about using Ettercap to intercept communication and alter it in flight. The specific scenario he targeted was javascript based logins, where the page is served over http to improve performance but the login field uses javascript to submit over https; using ettercap he can show how he can modify the javascript while the page is being delivered so that it submits credentials elsewhere.

He recommends that to maintain performance you specifically force users to go to a separate page that is served start to finish over SSL. The problem is that 1) you don’t get the EV Cert based visual confirmation on the landing page if you do this and 2) if your landing page is sent in the clear you can use Ettercap to modify the url to the login page so the user goes offsite to submit credentials. Once a user is logged in you need to either have a ridiculously short session limit or make sure you continue to communicate over SSL, otherwise the sessionid will be sent in the clear which is almost as valuable as the user credentials.

I think it is much better to ALWAYS use SSL with EV certs, forced on every page. That way you get the visual validation of the cert on the landing page, the customer sees the little lock through the whole communication, and the entire conversation is resistant (not immune, but resistant) to tampering. This will impact performance (though I think it might have the advantageous effect of forcing cleaner website design) on dial up (I doubt it is human noticable on broadband) but if you want to provide some level of certainty to the user it is the way to go. Obviously, this should be weighed based on the risk of the site; paypal and banks should obviously do this, engadget probably can do without. As a final piece of advice, make sure that you set the cookie flag so that session information is only sent over secure channels.

These aren’t bullet proof assurances. It is all based on visually notifying the user that this is the site the really mean to be on. It is very reliant on training user behavior, a generally unreliable approach, and then on the user consistently being wary, which is even more unreliable. However, at this point we don’t have much else we can really do. Sandbags may not be an ideal replace for building on high ground, but it beats letting the water in. End to end trust is an unsolved problem but we do what we can.

~ Joshbw

A paper by Adam Shostack is circulating the blogosphere, and it is worth a read. I’d already left MS before Adam took over as the gatekeeper of threat modeling for SWI but it is nice to see that he basically agrees with the stripped down version that we regularly did in WinCE and that I still evangilize. Mainly-

1) Draw a data flow diagram (DFD) of how data moves around for your application/feature (depending on scope of the model), and where trust boundries are crossed

2) Enumerate threats. The paper has a nice little matrix of what threats are pertinent to what elements of the DFD and while he applies the necessary conditionals about it being specific to MS and perhaps not being universally applicable, I think it is a pretty good reference in general.

3) Determine Mitigations (or decide to let a threat slide)

4) Validate mitigations that are put in place

We lived by those simple steps, rather than getting into the horrid threat trees and DREAD risk modeling and all of that other overhead. I say this having been the security guy for my area in MS, supposedly the expert, and I ignored them. I can’t imagine the average non-security guy going to that trouble. Granted, being a security guy I probably find it a bit easier to subjectively (wait, am I supposed to say qualititavely now that I am a CISSP?) determine risk from a threat than put a number on it, which is what the overhead in threat modeling was supposed to do, but I think most non-experts can still make an educated guess.

Anyway, the reason I am bringing all of this up is to illuminate one aspect that isn’t touched on much in most threat modeling discussion- Don’t just focus on your process/components. When people create a data flow diagram and start thinking about threats and mitigations they tend to focus on threats specific to their own code and how to mitigate stuff entering that code. However if we look at a vulnerability like XSS, it isn’t a threat to your own code, it is a threat to external entities (in this case, client browsers), and the best mitigation is to sanitize (HTML encode) data as it leaves your application rather than validate it as it enters (now you still need to validate it as it enters to make sure it isn’t malicious to your code, but don’t worry about flat out stopping xss with input validation if you can output encode).

Similarly, in the above web app example, a threat model would have us worry about how to authenticate the client for repudiation and information disclosure purposes, but I haven’t been to many threat modeling sessions where the team originaly considered the data flow the other direction (establishing to the client that this is the server you can trust- more on that in a post in the near future).

So as you are developing your threat model be sure to consider threats to each of the entities in the DFD. Don’t focus only on the flow into your components, and don’t consider threats to just your components, but rather look at the full ecology represented in your DFD.

~ Joshbw

At Blackhat this year Mark Dowd of IBM and Alexander Sotirov or VMware presented a paper concerning how to bypass Vista’s Buffer Overflow Countermeasures. The paper is worth a read both because of the vulnerabilities, and because it does a great job detailing how much protection Microsoft has provided to people who can’t write good code. The analysis of the paper by various people on the Internet is mind numbing though (probably this post included).

Before continuing, I think some things need to be established up front. Vista provides protection that no other OS does against poorly written code. Even if the security features were completely worthless Vista would not be more dangerous than any other OS- just more dangerous than Vista from last month. This additional protection was meant to be a stop gap to secure bad code, not an excuse not to fix the bad code. The root cause is that some code sucks to begin with. It is like throwing a WAF on a wide open Internet app. That buffer overflows still exist to the extent they do is just disgraceful to us as an industry. We know exactly what causes buffer overflows and we know how to avoid them. Do not use unchecked buffer copies. Explicitly copy no more data than a buffer can hold (and please, please do not use inline calculations in strncpy, since that may as well be an unchecked buffer copy). Don’t dynamically create the first argument for *printf/*scanf without knowing exactly how many other parameters will need to be provided. And the list goes on. These are all things that static source code analysis tools are remarkably competent at finding. A clever source control system will flat out ban people from checking code in with bad APIs (strcpy and its unchecked brotheren). Finally, fuzz the crap out of all your input points (hey Jim, this is why input validation is important :-p). You will find and kill 99% of your buffer overflows if you do that, and the remaining 1% won’t be very discoverable (bad guys do fuzz testing too you know).

Anyway, there is a reason why MS has had a huge decline in buffer overflows relative to the industry, and it is because they do all of the above. They have too much legacy code and third party giblets to fully get rid of overflows, but all of the extra protection in Vista is primarily to provide some level of protection for the OS because a retarded third party (*cough*quicktime*cough*acrobat*cough*flash*cough) has yet one more buffer overflow that can be used to compromise the box. The protection was not a permanent fix, but simply something to raise the bar to perform exploitation, and I think Microsoft has been reasonably good about presenting it in that light while still trying to sell 3rd party developers on the protection. The biggest flaw in the protections that Vista presents is that applications opt into using them. Programs need to explicitly state they want the protection which is why the paper focuses on exploiting applications that do not have all of the protections (like IE 7 which doesn’t use data execution prevention because the poorly written piece of junk sun java plugin would blow chunks if they did; fortunately IE 8 does use it, as does firefox 3).

Now on to the commentary on the web. Peter Bright of Ars Technica tries to downplay it and many of his points are valid. Bruce Schneier thinks it is a big deal, which is true. A month ago people didn’t know how to bypass these protections but now they do. That is a big deal, but Adrian Kingsley-Hughes of ZDnet is still a moron for saying it renders Vista security next to useless.

So to respond to the commentary, the reason this is a big deal (moreso than Bright believes) but not the end of the world (as Kingsley-Hughes thinks) is that it makes exploits feasible but more costly. Any technological measure that increases the cost of exploiting something will decrease the number of instances that blackhats bother exploiting it. For example, to defeat ASLR the authors advocates hosting a .Net Web Component of sufficient size to dramatically impact how the address space can be randomized, and in order to facilitate downloads they also recommend the webserver hosting the component use gzip compression (which in turn means controlling web server configuration). That’s entirely possible, but if a malware author had the choice between an exploit targetting all of XP that didn’t require that extra work or an exploit for both XP and Vista, I suspect 90% will choose to just go after XP. (the exploit using Java applets is a bit more likely at the moment do to the GIFAR hack recently discussed, since you can very easily get hundreds of sites to host that for you).

On top of that, to bypass the multitude of protections requires some sophistication, which is just going to raise the bar on who crafts the exploits (well, until all of the bypasses are just built into metasploit). Raising the bar lowers the number of attackers capable of compromising the system.

Additionally, the easiest work around for DEP was to target applications that don’t use it, but this will become increasingly difficult, especially as MS enforces DEP to a greater degree. Finally, at least in terms of IE, one feature that was not addressed with the process isolation of the browser in Vista. It remains an open question on whether the IE sandbox in Vista is vulnerable, as it may still contain the attack.

So attacks are now possible, but the exploitability is still more difficult than other platforms. That’s big, but it by no means renders the protection next to useless. As the authors conclude

In this paper we demonstrated that the memory protection mechanisms available in the latest versions of Windows are not always effective when it comes to preventing the exploitation of
memory corruption vulnerabilities in browsers. They raise the bar, but the attacker still has a
good chance of being able to bypass them. Two factors contribute to this problem: the degree to
which the browser state is controlled by the attacker; and the extensible plugin architecture of
modern browsers.

At the end of the day, good OS design cannot completely mitigate bad code. Get rid of buffer overflows and this is all a moot point anyway. Its 2008 people, why aren’t buffer overflows extinct?

~ Joshbw

Ars has their panties in a twist because Kevin Turner dared to suggest that vista is more secure than OS X or Linux. The thing is, from one point of view, Kevin is absolutely right, but from other points of view he is wrong, which just points out the general failing that keeps popping up among security folks: How on earth do you measure security?

When talking about security there isn’t a clean metric.  You have

Vulnerability: Software (for this purpose) weakness that provides a door in
Threat: danger that someone will find the vulnerability
Risk: the likelihood and cost of the vulnerability being exploited
Exposure: time that a vulnerability remains

Companies/orgs love to cherry pick which one they go off of.  Firefox is big on touting their low exposure while being horribly quiet about the number of vulnerabilities.  Microsoft is big about vulnerability (and to some extent exposure, at least on the OS level) count, since they do have a much lower count then competitors that don’t have a decent secure development lifecycle (which would be Apple and most of the OSS projects), but they don’t like mentioning risk because they have more risk than freaking anyone.

The problem is, all of these approaches are at least a little right.  Sure the risk is huge on windows as a whole, but if you install patches quickly your risk is actually pretty low, and with a lower vulnerability count there is less likelihood that someone will release a zero day for a vulnerability that isn’t patched. Mozilla is also right about firefox being more secure, as the fast patch and deployment rate means that zero day isn’t going to remain a concern long (except that they have been a little too quick pushing patches and have regressed things on a number of occassions), but their higher vulnerability count does mean zero days are going to crop up more often.

The only stance that isn’t right is one that throws out vulnerability count as a metric because of “more eyes” (which I have complained about before). Being open source may mean that more good guys have access to code reviews, but it also means more bad guys can get a leg up looking for flaws. The number of known vulnerabilities is ALWAYS a problem regardless of why they are there (incidnetally, again, that more eyes arguement is crap. Code reviews are a horribly inefficient way of finding flaws, and you never know when some poorly trained reviewer will comment out the functionality that actually makes your random function random).

So if all of these are a little right, what is the problem? I think it is really twofold. First, it makes it very hard to make an informed comparison of two platforms in terms of security and best fit for your org. Second, it makes it hard to gauge your own security and communicate that to people in and out of your organization. Both are pretty important, and a common, informative, metric benefits everyone (except those that look bad because of it).

Thinking about it I like:

[Ʃ (CriticalityModifier * Exposure)] / (nTotal * 10)

or, in English, the sum of Criticality of each vulnerability in the past twelve months * the number of days that the vulnerability was unpatched (patches that get rolled back don’t count), then divided by the total number of vulnerabilities times ten, to scale down the number a bit.

The criticality modifier is 0.5 for low, 1 for moderate, 2 for high, 3 for critical. As an example, if a hypothetical product had the following vulnerabilities over the past twelve months:

would give you (3*45 + 3*35 +3*4 +2*20 + 1*45 + 1*50 + 0.5*145 + 0.5*75) / (8 * 10) = 6.15 Security Rating.

This seems like a fair metric, as it measures a level of impact (the criticality, though this isn’t horribly quantitative) of the flaw, the days of exposure to the flaw, and the number of total flaws. What it does not take into account is likelihood, as I can’t come up with a good quasi-quantitive means to measure said likelihood (open to suggestions here).

Thoughts?

~ Joshbw

Perhaps I was a bit misleading in the title as this is a flaw in an internet connected coffee machine (why, oh why, do we need this) that allows, among other things, access to a connected PC.

There are a couple lessons in this. First, no matter how trivial your product, you very likely still need to worry about security if it connects to something that does have value. Second, your system is only as secure as all of the systems that connect to it, unless the system is inherently paranoid about connections. In this web 2.0 world of mashups, in the world of connected enterprise applications, our applications are not living in a bubble. They are connected to multiple other systems, and in most cases there isn’t a whole lot of thought that went into security over these connections. A web application may be horribly paranoid about anything it receives from the user, but then blindly accept data from internal systems without a thought. If any of those systems is flawed and you don’t validate input from them, your application is just as susceptible to attack as it would be if it didn’t validate user input.

This just further cements in my head the need to push threat modeling out to more people. I think the real value of threat modeling isn’t the complex threat trees that get built for risk analysis. I actually find those pretty worthless. I think the data flow diagram is the big thing. It is a convenient visual diagram showing where a process or feature is pulling data from and where it is sending the data to. Any source of data is a source of attack, and the DFD is a useful tool for identifying where further scrutiny is necessary. Any output source is a particular asset that might be attacked through your application. I think every dev spec should require a data flow diagram simply because it gets people thinking.

Finally, the last immediate lesson from this is that it is a scary time to be a client. The OS may be pretty solid but there are a million ways to execute code that seem perfectly valid from the OS point of view. Every connected application, any application that pulls data or opens data originally from a remote source, is a potential open door. If a freaking coffee machine can be a source of attack client security is in a lot of trouble.

~ Joshbw

Seriously, the webappsec community is abuzz with the new PCI requirements that allow either code review or application firewalls, and there are people out there debating the merits of both, as if it is a competition.  Are WAFs a band-aid?  Of course they can be, to compensate for crappy input validation, but even if one’s input validation is perfect and protects against all known attacks today having an added layer of defense isn’t likely going to hurt a great deal (though specific situations can cause issues), and is quicker to update when new attacks are developed.  Also, your input validation isn’t perfect.  Even your whitelist input validation isn’t perfect.  It is probably “good enough”, but it isn’t perfect.

Are WAFs a replacement for code review- not ever.  A WAF sucks at finding back doors or easter eggs, or finding pages that *should* have an authorization check at the top but don’t, or a myriad of other issues that can still be used right through the firewall.  Code review will find those things, even if it is a very inefficient and ineffective way of tracking down vulnerabilities in general. 

In truth, none of this is sufficient, but PCI isn’t really about securing customer data, at least the way it is written, but rather a way to avoid liability.  It is a minimum number of steps to say “look, we tried to do something”.  The fact that “Private Networks” are considered secure and data can be sent in the clear over them (even though less than 30% of successful attacks actually come from external networks) shows that it isn’t really concerned with really protecting payment card information.

For companies that want to meet the minimum requirements for PCI, the only question is which solution is cheapest and easiest.  For companies that actually care about protecting their customer’s sensitive information, the question shouldn’t be which to do, but what additionally can be done.  Neither of these solutions are sufficient to ensure that data is protected by themselves.  The first requirement to protect customer data is to get upper management to care, then to put security standards in place, a secure development lifecylce in place, and educate developers on the threats they face and the protections they should employ.

An ideal solution shouldn’t just have a code review, but a feature review when the feature is proposed (with security requirements included), a risk analysis and mitigation strategy during the architecure planning, threat modeling and detail design security review, code review, and both security control testing and pentesting both with and without a WAF, so that if the WAF fails the application is still strong.

That is what PCI should be.  Yes, it is a huge burden on development teams and companies that don’t do those things already, but at the same time, ask yourself if you would trust a company that did the current bare minimum to meet PCI as it stands now with your card information.  I sure don’t, which is why I am glad that my bank now offers onetime card numbers and more and more sites accept paypal. 

~ Joshbw