This thesis investigates different approaches to functional testing and security testing Five common methods of generating test cases for functional testing have
Previous PDF | Next PDF |
[PDF] Mobile Application Security Testing - Deloitte
Our comprehensive mobile security testing approach will cover all the possible threats and attack vectors that affect the mobile app landscape
[PDF] Analysis of testing approaches to Android mobile application
Partnership and Mobile application security verification standard Also, we have compared methods it selves and methods of testing for vulnerabilities of mobile
[PDF] Mobile Application Security Testing - Mphasis
These facts and figures clearly state that mobile application should be subjected to periodic scan to identify vulnerabilities and subsequent fixing methods, in order
[PDF] Introduction to Mobile Security Testing - German OWASP Day
Approaches and Examples using OWASP MSTG OWASP Mobile Automotive Security Testing OWASP Mobile Application Security Verification Standard
Mobile security testing approaches and challenges - IEEE Xplore
security testing targets to detect vulnerabilities and malicious apps on a mobile device In this paper, we present four testing approaches for mobile security:
[PDF] Security Testing Guidelines for Mobile Apps - OWASP Foundation
Security • Expert for Mobile App Testing • Developed the Mobile Security Testing Guide in his Result: General process (mandatory) and supporting tools and
Functional and Security test- ing of a Mobile Application - DiVA
This thesis investigates different approaches to functional testing and security testing Five common methods of generating test cases for functional testing have
[PDF] MOBILE APPLICATION SECURITY WITH OPEN-SOURCE TOOLS
Instances of web-application security issues which lead to breaches Android Security Test Cases software for general users is not an ideal approach First
[PDF] Vetting the Security of Mobile Applications - NIST Technical Series
1 avr 2019 · details a mobile application vetting process 2 1 2 OWASP Mobile Risks, Controls and App Testing Guidance 4 1 Testing Approaches
[PDF] mobile application security testing pdf
[PDF] mobile application security testing ppt
[PDF] mobile application testing checklist xls
[PDF] mobile apps for language learning pdf
[PDF] mobile computing applications
[PDF] mobile computing architecture
[PDF] mobile computing framework
[PDF] mobile computing functions pdf
[PDF] mobile computing functions ppt
[PDF] mobile computing through internet
[PDF] mobile computing tutorial
[PDF] mobile development design patterns
[PDF] mobile device industry analysis
[PDF] mobile financial services companies
Bachelor thesis, 16 ECTS | Information Technology
2017 | LIU-IDA/LITH-EX-G--17/066--SE
Functional and Security test-
Sara Westberg
Supervisor : Simin Nadjm-Tehrani
Examiner : Nahid Shahmehri
hemsida http://www.ep.liu.se/.Copyright
The publishers will keep this document online on the Internet - or its possible replacement - for a period of 25 years starting from the date of publication barring exceptional circum- stances. The online availability of the document implies permanent permission for anyone to read, to download, or to print out single copies for his/hers own use and to use it unchanged for non-commercial research and educational purpose. Subsequent transfers of copyright cannot revoke this permission. All other uses of the document are conditional upon the con- sent of the copyright owner. The publisher has taken technical and administrative measures to assure authenticity, security and accessibility. According to intellectual property law the author has the right to be mentioned when his/her work is accessed as described above and versity Electronic Press and its procedures for publication and for assurance of document integrity, please refer to its www home page: http://www.ep.liu.se/. cSara Westberg
Students in the 5 year Information Technology program complete a semester-long software development project during their sixth semester (third year). The project is completed in mid- sizedgroups, andthestudentsimplementamobileapplicationintendedtobeusedinamulti- actor setting, currently a search and rescue scenario. In parallel they study several topics relevant to the technical and ethical considerations in the project. The project culminates by demonstrating a working product and a written report documenting the results of the practical development process including requirements elicitation. During the final stage of the semester, students create small groups and specialise in one topic, resulting in a bachelor thesis. The current report represents the results obtained during this specialisation work. Hence, the thesis should be viewed as part of a larger body of work required to pass the semester, including the conditions and requirements for a bachelor thesis.Abstract
A mobile application has been developed to be used for assistance in crisis scenarios. To assure the application is dependable enough to be used in such scenarios, the application was put under test. This thesis investigates different approaches to functional testing and security testing. Five common methods of generating test cases for functional testing have been identified and four were applied on the application. The coverage achieved for each method was measured and compared. For this specific application under test, test cases from a method called decision table-testing scored the highest code coverage. 9 bugs re- lated to functionality were identified. Fuzz testing is a simple security testing technique for efficiently finding security flaws, and was applied for security testing of our applica- tion. During the fuzz test, system security properties were breached. An unauthorized user could read and alter asset data, and it also affected the system"s availability. Our over- all conclusion was that with more time, creating functional tests for smaller components of the application might have been more effective in finding faults and achieving coverage.Acknowledgments
Nilsson and Filip Polbratt - for doing the excellent job of developing the system that has been put to test in this thesis together with us. Especially we want to thank our supervisor Simin Nadjm-Tehrani for the feedback and help during this project, and Mikael Asplund for the feedback during the start of the project. We also want to thank Rickard Hellenberg and OskarGustafsson for their opposition on this thesis.
vContents
Abstractiv
Acknowledgments v
Contentsvi
List of Figures viii
List of Tablesix
1 Introduction1
1.1 Aim . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2 Problem statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.3 Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.4 Delimitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.5 Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2 Software testing 4
2.1 Software testing in general . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2.2 Android testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.3 Functional testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.4 Coverage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.5 Security testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.6 Related work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
3 Test implementation 10
3.1 The application under test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
3.2 Selected frameworks and approach . . . . . . . . . . . . . . . . . . . . . . . . . . 10
3.3 Functional testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
3.4 Security testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
4 Results18
4.1 Functional testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
4.2 Coverage reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
4.3 Security testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
5 Discussion21
5.1 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
5.2 Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
5.3 The work in a wider context . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
6 Conclusion25
Bibliography27
vi A Appendix: Functional tests in Espresso - Source code 29 A.1 Authentication test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 A.2 Map test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 A.3 Contact test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39B Appendix: Fuzz test - Source code 42
B.1 Fuzz test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42List of Figures
1.1 Application view over the map activity . . . . . . . . . . . . . . . . . . . . . . . . . 2
2.1 A pyramid of the testing levels. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
3.1 Application view when creating a new pin . . . . . . . . . . . . . . . . . . . . . . . 12
3.2 Fuzz test flow chart . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
4.1 Invalid database entries from the fuzzer . . . . . . . . . . . . . . . . . . . . . . . . . 20
viiiList of Tables
2.1 Decision table - Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
3.1 Decision table - Map test, add pin . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
3.2 Decision table - Map test, remove pin . . . . . . . . . . . . . . . . . . . . . . . . . . 13
3.3 Identified test cases - Map test, add pin . . . . . . . . . . . . . . . . . . . . . . . . . 13
3.4 Identified test cases - Map test, remove pin . . . . . . . . . . . . . . . . . . . . . . . 14
3.5 Decision table - Authentication test . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
3.6 Identified test cases - Authentication test . . . . . . . . . . . . . . . . . . . . . . . . . 15
3.7 Decision table - Call test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
4.1 Bug report from functional testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
4.2 Coverage report - Whole application . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
4.3 Coverage report - Map activity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
4.4 Coverage report - Authentication activity . . . . . . . . . . . . . . . . . . . . . . . . 19
4.5 Coverage report - Service class . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
4.6 Vulnerability report for the security testing . . . . . . . . . . . . . . . . . . . . . . . 20
ix1Introduction
Smartphones are becoming more and more advanced and used in more advanced situations. Nowadays they are used not only in the entertainment sector but also in more critical do- mains. The increasing complexity makes testing mobile applications very hard [1], since the combination of possible inputs grows rapidly. Making sure that your application is de- pendable before pushing an update is not an easy task, but a necessary one. By testing the application"s functionality, faults can be found and the application will become more depend- able. However, there are many different testing methods to chose from. Therefore we will investigate how well different testing methods actually work. This will be done by performing tests on an application that has been developed along- side this thesis. The application under test (AUT) consists of an Android application, where the application depends on a server which is connected to a database. The purpose of the ap- plication is to be a mock up for a tool used by the Swedish Defence Forces for communication and information sharing during a crisis scenario. Figure 1.1 shows a view of the AUT, where pins are placed on a map, representing different tasks for the Swedish Defence Forces. when all of the faults have been found. But if the AUT passes tests that covers all of the source code with relevant variations of different input, it is a sign that most errors due to coding faults given input assumptions have been considered. Coverage is a metric that will be used in this thesis, which is often used in software testing and makes it possible to measure how much of the source code has been tested. A systematic way of performing automatic testing to cover all of the AUT"s functionality needs to be identified. This is due to the low coverage that would be achieved when only end-user testing is used. This is seen in a study with 7- user testing of some popular apps that got a result of only 30% coverage of app screens and only 6% of the apps functionality [2], which is seen as a bad results. More advanced software also leads to more vulnerabilities in a system. These vulnerabil- ities can be exploited by an attacker who wants to get access to the system [3]. This type of attack is seen as a threat to the system since the system assets would be at risk, which calls for the need to identify and remove as many vulnerabilities as possible to increase the security of a system. 11.1. Aim
Figure 1.1: Application view over the map activity1.1 Aim
In this thesis, we will test the AUT with respect to functionality and security. To test the functionality we will analyze different approaches of functional testing with the goal to gain higher coverage than 70%. When testing the security of the system we will focus on the assets and their exposure to breach of confidentiality and integrity. An unauthorized user should not be able to read or alter data. We will find and use approaches of constructing tests that make sure that the AUT is not compromising the confidentiality and integrity of the system.1.2 Problem statement
In this thesis we will:
1. Investigate how to constr uctfunctional test cases systematically to achieve a high enough coverage. 2. Identify an ef ficientway of testing a system"s confidentiality and integrity . 3. Perform functional and security tests and apply on our system, then categorize and analyze the results from the implemented tests. We will try to answer what we mean by saying "high enough" in chapter 2, but for now, our aim is to achieve 70% coverage.1.3 Approach
To identify the faults on our application"s functionality we have performed functional tests, aiming to get high enough coverage. The coverage was measured in terms of instructions run and branches taken. Random input generation tests were also performed to make a com- parison between the different functional tests based on how much coverage they achieved. 21.4. Delimitations
To identify vulnerabilities in the system we performed a simple penetration test, aiming to get access to or alter asset data as an unauthorized user. The need for an efficient method is because we have limited knowledge in security testing, thus need an easy to implement method that still can find flaws in a systems security.1.4 Delimitations
Software security is typically defined as the confidentiality, integrity and availability of a system. However, in this thesis we will only design test cases for the first two, confidentiality and integrity.1.5 Structure
The rest of the thesis is structured as follows. Chapter 2 will present background theory of testing and related work. Chapter 3 will describe the AUT as well as how the tests were constructed, and how the results were collected. Chapter 4 will present the results from the tests. In chapter 5 the results and method will be discussed. Lastly in chapter 6 the answers to the problem statement as well as our ideas for future work to be done will be given. 32Software testing
In this chapter, we will present some background theory of software testing. We will present some common methods for identifying test cases for functional testing, how to evaluate tests and cover security testing briefly. Lastly we will view related work for this thesis.2.1 Software testing in general
The definition of software testing can vary quite a lot depending on who you ask. One of the most popular definitions is Glen Myers, "Testing is the process of executing a program with the intent of finding errors". James Bach"s definition is "Testing is questioning a product to evaluate it". The latter is more relevant to modern use of software testing, since testing can be used for quality assurance of a product [4]. The stakeholders of a product must know whatquality it possesses, and to find that out empirical experiments are carried out [4].Figure 2.1: A pyramid of the testing levels.
Testing is usually performed at different levels, where the test cases have specified areas to cover. The most common levels used areunit testing, integration testing, system testingand acceptance testing[5], which can be seen in figure 2.1. A unit is the smallest piece of software that is tested. What a unit is can differ for different programming languages (for example a unit can be a class in java or c++, or a function in c), but if the tester thinks of it and tests it 4