Nicolas Carlini, David Wagner: Towards Evaluating the Robustness of Neural Networks

Machine learning is used in more and more applications and so far no evaluation has looked at the resilience of neural networks in general to adversarial attacks. Given that the attacker knows the model, how resilient is the neural network against adversarial input? Nicolas started his talk with how neural networks vastly improved the classification rate of images, e.g., for text recognition or object recognition. Compared to the existing heuristics, the neural networks vastly outperformed the old approach.

The classification is often used to infer some information about an image. Given that the attacker has access to the classifier (either directly or indirectly by submitting images for classification), she can probe and figure out a minimum amount of changes (minimum according to a predefined metric) to misqualify the image as something else.

In this work, Nicolas developed attack models and techniques to probe neural networks and test heir resilience against attacks. The results were rather surprising as most models degenerated with only few changed pixels. As it turns out, the models learn shallow information to classify images and don't deeply understand the images. The talk showed the limits of these automated classification approaches and won the best student paper award.

Getting security right

Stack Overflow Considered Harmful? The Impact of Copy and Paste on Android Application Security. In this paper, the authors did a large scale study on Android applications, evaluating if code snippets from open sources are used without checking their constraints. The framework first mines a large set of Android applications, recovering the Java code used in these applications. Orthogonally, the framework mines StackOverflow to find code snippets for crypto API usage. These snippets are then labelled as good or bad -- there is a surprising amount of bad or wonky crypto advice on StackOverflow that will make applications more vulnerable. Developers without crypto knowledge cannot know which parameter selection is safe, resulting in buggy applications. Given both datasets (the decompiled applications and the mined crypto API examples), the framework searches for matches. Interestingly there was a large amount of code reuse, including a large amount of bad crypto.

Comparing the Usability of Cryptographic APIs. Along a similar vein, this paper analyzes the usability of different crypto APIs. There are several crypto API that all offer similar communication primitives. The libraries are different in their API. The user study contained a set of easy and hard tasks that each user had to solve with the different APIs. Surprisingly, the simplified APIs were not useful for the more complex tasks as the users did not have enough degrees of freedom to select the correct parameters. The library that allowed complex configuration but had good documentation and a well designed API was used for most correct solutions. As it turns out good documentation, examples, and giving the developers options to select them is better than both just a complex API or a simplified API.

Attacks

How They Did It: An Analysis of Emission Defeat Devices in Modern Automobiles. In this project, folks from RUB and UCSD reverse engineered the firmware of the exhaust system of different cars and inferred how the firmware reacts to different settings, e.g., reducing the engine's power if certain conditions are met. Modern engine controls are highly complex and must react to different situations, detecting the conditions of an ongoing measurement was just one addition to these systems.

The Password MitM Attack. Second factor authorization must be designed carefully to be effective. As a user is registering with a malicious service, the service tells the user that it will have to answer a two factor challenge. At the same time, the service generates a login to a trusted service on behalf of the user. The user then relays the second factor token from the trusted service to the malicious service. If the trusted service does not identify itself (i.e., "your token is 12345") then the user does not know from which service the challenge came. A well designed second factor will send the text messages from the same well known number and identify the service as part of the challenge. Even then, some users will ignore this information and continue to log into the malicious service.

Systems Security and Authentication

Protecting bare-metal smart devices with EPOXY. In our talk we presented Abe's work on protecting bare-metal devices through a privilege overlay. Instead of running all software at the highest privilege level we drop privileges for the majority of code and selectively enable privileges for a few instructions that actually need them (e.g., for IO). Dropping privileges allows us to configure the Memory Protection Unit (MPU) to enforce access restrictions to code and data, enforcing non-executable data and non-writable code. To protect against code reuse and data-only attack we also apply diversity and a safe stack.

Norax: Enabling Execute-Only Memory for COTS Binaries on AArch64. In this project the authors enable execute-only memory for ARM binaries. The anlysis separates code and data into separate pages. Surprisingly, ARM has much more data embedded in code pages compared to x86 which makes this problem harder. The authors propose an approach that disassembles and patches the underlying code and then use a modified loader to update necessary references at runtime. In addition, a runtime monitor handles any missed references and backpatches them.

Software Security

kyfire: Data-Driven Seed Generation for Fuzzing. The authors developed a targeted fuzzer for XML libraries and found a large amount of vulnerabilities in these packages. It is surprising that no targeted fuzzing has been done on XML so far.

VUDDY: A Scalable Approach for Vulnerable Code Clone Discovery. The authors detect vulnerable code in binaries through code matching. They generate fingerprints for specific exploits and use these fingerprints to find vulnerable instances in other libraries. The goal is to identify cloned/forked code that has not been patched after a security vulnerability was detected. An alternate approach is to find stolen code or misused code (e.g., use of open-source code in closed source applications).

Conclusion

Overall, Oakland was a fun conference and this year there were a bunch of interesting system security papers. Next to the few papers I highlighted there were many other interesting sessions that I could not attend. As always, the program is diverse and covers research in information flow, software security, embedded systems, hardware, and crypto. In addition, Oakland serves as a convenient opportunity to sync up with peers at the end of the spring semester and relax after the stressful CCS deadline (which was 2 days before the conference).