ITEQS

We present a novel exploratory performance testing algorithm that uses machine learning to optimize the test suit generation process. The goal is to generate test suites that contain a large number of positive tests, revealing performance defects or other issues of interest in the system under test. The key idea is to use a deep neural network to predict what test could be positive and to train this network online during the test generation process, designing and executing the test suite simultaneously. The approach is completely automatic and it does not require any prior knowledge about the internals of the system under test. It can also be used effectively in a continuous integration setting where small variations of a system are tested successively. We evaluate our algorithm using two example problems: searching for bottlenecks in a web service and searching for efficient hardware configurations in a single board computer. In both examples, the presented algorithm performed several times better than a random tester and our previously published algorithm, producing test suites with a large proportion of positive tests.

Performance is one of the Extra-Functional Properties (EFPs), an essential part of software development with limited resources. In such a context of software development, containers represent a widely used abstraction technology that requires fewer resources, when compared to hypervisor-based virtualization. Containers provide access to environments and speed up development by abstracting the underlying infrastructure layer for different application domains. Volume sharing is a usual abstraction provided by containers for data persistence, which can be accomplished, especially with Network File System (NFS) or Docker's sharing method. Evaluate the performance of these volumes in containers is critical to software development; however, we did not find available in the literature such evaluation. This paper describes the volume sharing performance for containers using NFS and Docker, which is essential for data persistence. We evaluated the performance of these shared volumes simulating three application domains: database accesses from a benchmark, direct reads and writes in the file systems and concurrent accesses from an MPI parallel program. Our results show significant differences among the distinct volume sharing methods analyzed for Docker and NFS. While asynchronous NFS has better performances for the set composed of heavy workloads, bursting read and write sequences, the Docker volumes, by the other hand, outperformed for processes with concurrent accesses when using a parallel program.

Recently, a hazard analysis technique STAMP/STPA has been widely accepted since it is recognized as being suitable for software-intensive systems. Using STAMP/STPA, we can find hazardous scenarios of the target system that cannot be obtained by other traditional hazard analysis methods and those scenarios can be used for validation testing. However, generally the number of obtained scenarios can be huge and the validation testing involves a considerable cost. In this study, we propose a method to prioritize hazardous scenarios identified by STAMP/STPA with the help of a statistical model-checking technique. We give a procedure for systematically transforming the model defined by STAMP/STPA to a formal model for a statistical model-checking tool. We also show the usefulness of the proposed method using an example of train gate control system

Industrial products, like vehicles and trains, integrate embedded systems implementing diverse and complicated functionalities. Such functionalities are programmable by software and contain a multitude of parameters necessary for their configuration, which have been increasing due to the market diversification and customer demand. In addition, industrial products are often built by aggregating different software parts (components), constituting thus product variants. Product variants with such variability need to be tested adequately, in particular if one is concerned with security vulnerabilities. While efficient automated testing approaches already exist, such as fuzzing, no tool is able to use results from previous testing campaigns to increase the efficiency of security testing the next product variant that shares certain functionalities. This paper presents an approach that can ignore already covered functionalities by previous tests and give more importance to blocks of code that have yet to be checked. The benefit is to avoid repeating unnecessary work, hence increasing the speed and the coverage in the new variant. The approach was implemented in a tool based on the AFL fuzzer and was validated with a set of programs of different versions. The experimental results show that the tool can perform better than AFL in our testing scenario.

The ADS-B - Automatic Dependent Surveillance Broadcast - technology requires aircraft to broadcast their position and velocity periodically. The protocol was not specified with cyber security in minds and therefore provides no encryption nor identification. These issues, coupled with the reliance on aircraft to communicate on their status, expose air transport to new cyber security threats, and especially to FDIAs --- False Data Injection Attacks --- where an attacker modifies, blocks, or emits fake ADS-B messages to dupe controllers and surveillance systems. This paper is part of an ongoing research initiative toward FDIA test generation intended to improve the detection capabilities of surveillance systems. It focuses on the mechanisms used to alter existing legitimate ADS-B recordings as if an attacker had tempered with the communication flow. We propose a set of alteration algorithms covering the taxonomy of FDIA attacks for ADS-B previously defined in the literature. We experiment this approach by generating test data for an AI-based FDIA detection system. Experimental results show that the proposed approach is straightforward to generate attack situations and provides a efficient way to easily generate sophisticated alterations that were not picked up by the detection system.

Categorizing existing test specifications can provide insights on coverage of the test suite to extra-functional properties. Manual approaches for test categorization can be time-consuming and prone to error. In this short paper, we propose a semi-automated approach for semantic keywords-based textual test categorization for extra-functional properties. The approach is the first step towards coverage-based test case selection based on extra-functional properties. We report a preliminary evaluation of industrial data for test categorization for safety aspects. Results show that keyword-based approaches can be used to categorize tests for extra-functional properties and can be improved by considering contextual information of keywords.