TEST PLAN FOR MUPEMAP MAP PLATFORM

 

Version history

 

Revision	Date	Changes	Changed by
0.1		Initial version
0.2	February 6, 2006	Review version	Teemu Mäki (some edits)

 

TABLE OF CONTENTS

TEST PLAN FOR MUPEMAP MAP PLATFORM Table of figures 1. Introduction 1.1 purpose and scope 1.2 product and environment 1.3 definitions, acronyms and abbreviations 2. Environmental restrictions 2.1 hardware 2.2 software 2.3 security 2.4 tools and data 3. Staffing requirements 3.1 people 3.2 training 4. Responsibilities 4.1 integration test group 4.2 system/application test group 4.3 other possible test group(s) 5. Required outputs 6. Specialities 6.1 functionality not to be tested 7. Order and methods of testing tasks 7.1 order of tasks 7.2 components to be tested 7.3 test case classes 7.4 methods and techniques 7.5 coverage 7.6 restrictions 8. Test cases 8.1 database testing 8.2 related software testing 8.3 user interface testing Test Cases of Main Menu Test Cases of Option button 8.4 interface testing 8.5 printing testing 8.6 security testing 8.7 recovery testing 8.8 performance testing 8.9 regression testing 9. Criteria and requirements 9.1 acceptance 9.2 rejection 9.3 requirements for termination (quitting) of testing 9.4 requirements for continuing testing 9.5 requirements for ending testing 9.6 requirements for rejecting code 10. Risk management 11. Schedule and tasks Week 13 Week 14 Week 15 Week 16 Week 17 12. Acceptance 12.1 Evaluate testing and testing results 12.2 Acceptance of testing as a whole

 

Table of figures

 

Name	Chapter
Diagram 1. System internal interfaces	8.4

 

1. Introduction

1.1 purpose and scope

 

The purpose of this document is to provide test cases and test results to be expected when performing testing of MUPEMAP map platform and its example game "Rise of the Xanthya".

 

The goals of this test plan are:

 

• To ensure that the software functions as described in the specifications
• To ensure that critical errors are detected in the software components during phase (unit testing)
• To ensure that different components function correctly when put together (integration testing)
• To ensure that the database transactions function with reasonably good performance (performance testing)
• To ensure that corrected errors don't reappear later (regression testing)
• To specify the reports being produced during testing

 

1.2 product and environment

 

MUPEMAP is an Open Source world map database platform for the Multi-User Publishing Environment (MUPE) created by Nokia Research Centre. The platform will be created in standard MUPE application fashion using MUPE XML server-side scripting and Java. The server will use an SQL database for storing the world map and game data respectively. The application UI will operate on MIDP 2.0 enabled smartphones running the MUPEClient. The software product details are discussed in more detail in the ProjectPlan and the DesignDocument.

 

1.3 definitions, acronyms and abbreviations

 

Emulator	A software emulator allows computer programs to run on a platform other than the one for which they were originally written, in this project J2ME emulator
GPS	Global Positioning System, a satellite navigation system used for determining one's precise location almost anywhere on Earth
J2ME	Java 2 Platform Micro Edition, a collection of Java APIs targeting embedded consumer products such as PDAs, cell phones and other consumer appliances
Java	Java, an object-oriented programming language. Java platform, execution engine called a virtual machine, and a set of standard libraries which provide common functionality.
MIDP	Mobile Information Device Profile, a specification published by Sun Microsystems for the use of Java on embedded devices such as cell phones and PDAs.
MUPE	Multi-User Publishing Environment, an Open Source application platform for creating mobile multi-user context-aware applications.
MySQL	MySQL is an open source, multithreaded, multi-user, SQL Database Management System
NOKOS	Nokia Open Source License
SQL	Structured Query Language, the most popular computer language used to create, modify and retrieve data from relational database management systems
XML	Extensible Markup Language, markup language for creating special-purpose markup languages
JSR 179	An Optional Package that enables developers to write mobile location-based applications for resource-limited devices. The API works on the J2ME CLDC v1.1 and CDC configurations. (http://www.jcp.org/en/jsr/detail?id=179)
Tile	Map item. Describes one area of the map and is the smallest item of the map. Tile types can be for example lake, field, forest.

 

2. Environmental restrictions

2.1 hardware

 

If possible, positioning enabled smartphones will be used in the testing stage. If these are not available, the positioning data input will be emulated in software.

 

2.2 software

 

Unit testing will be performed in the Eclipse IDE using the standard JUnit plugin. As new routines are coded, unit test cases are created as seen appropriate. Created unit test cases will be store in the CVS repository for easy access allowing unit tests to be repeated as necessary.

 

System testing will be performed in the J2ME Wireless Toolkit Emulator.

 

2.3 security

 

Probably the only security issue that needs to be considered is the possibility of SQL injection. This is covered in chapter 8.1 under database testing.

 

2.4 tools and data

 

The database will be populated with world map data suitable for testing, either live data created using the update tool or specially manufactured data, depending on the purpose of the respective test case.

 

3. Staffing requirements

3.1 people

 

Due to limitations from the nature of this coursework, the testing staff will be exactly the same as the programming staff. However, to increase the effectivity of the testing and the odds of discovering new errors, the developers will, schedule allowing, test each other's components.

 

3.2 training

 

As the testing staff are already experts in the software they're testing (see above) no specific training will be organised prior to testing.

 

4. Responsibilities

 

The following responsibilities are provisional and may be changed as the implementation is finalised.

 

4.1 integration test group

 

Responsibility	Person
Example game integration	Jouni Erola
Database integration	Hannes Karhumaa
Map server integration	Sakari Tamminen

 

4.2 system/application test group

 

Responsibility	Person
System testing	Xiaoqing Meng-Pitkänen
System testing, UI	Wei-wei Zhang

 

4.3 other possible test group(s)

 

Time allowing, it may be possible to engage some people not part of the MUPEMAP team to do some less technically demanding testing, such as testing the game rules.

 

5. Required outputs

 

Our test outputs will be following: test plan (actually this document), test cases and test reports. Test cases mean detailed description of tests, that we are gtoing to perform. Test reports mean, that we will take the minutes from every testing session. The record could look like it is described here:

 

Date	Amount of run tests	Tester's name
04/04/2006	3	Ted Tester
Test no	Test case	Success
1	Building a building in Xanthya game	OK
2	Harvesting a resource in game	OK
3	Submitting tile information to MUPEMAP database	NOT OK
	Succeeded	Not succeeded
	2	1

 

The test report should also include at least the following information:

Description of the error and all error messages, how critical the error is (minimal, average, severe or critical) and who of the programmers is going to fix it.

 

One important feature of MUPE server is that it produces automatically a log file called log.txt. The server writes the XML data, that it receives and produces, to this log file. This will surely be helpful, when we will be testing our own application. These log files can also be considered as the project's test outputs.

 

6. Specialities

6.1 functionality not to be tested

 

When we think about testing a mobile application, it would be natural to test mobile-specific cases, like how application behaves when the phone receives a phone call or a SMS. But as MUPEMAP will be working on MupeClient2 application, which we don't code at all, it is not necessary to test these normal phone functionality cases. Also things like connections and data transfers between server and clients are not necessary to test, because these are things that we can not affect.

 

7. Order and methods of testing tasks

7.1 order of tasks

 

This chapter describes the order of test tasks that are described more detailed in chapter 8.

 

The tests will be performed in an approximate bottom-up order:

 

1. Database testing
2. Map server testing
3. UI testing (game app + map update tool)

 

Testing the game rules may continue in parallel with these testing activities, as they don't have a major impact in the system structure or UI. Some tests have already been made in January 2006 prior to writing this document.

 

7.2 components to be tested

 

 

• Class testing:

1. Map
2. Database

 

 

• Class testing:

 

1. Worker
2. Building
3. Game rules
4. Player
5. Game

 

• Server side testing:

1. Recovery

 

• Class testing:

 

1. Game rules
2. UI (class)

 

• User interface testing:

 

1. Menus, accessing
2. All “sub” dialogs/forms
3. Usability in general

 

• Game rules:

 

1. game rules should be tested with sample game with multiple players playing.

 

7.3 test case classes

 

Tests are classified with the following levels in importance and criticality.

 

Importance

 

Level of importance	Description
High	The test with this level must be passed
Medium	This level make normal system operationality possible
Low	The test should be passed

 

Criticality

 

Level of criticality	Description
Very critical	Error makes it impossible to use the system. Should be fixed during the testing.
Critical	Functionality is working wrong.
Average	Functionality is working partly wrong.
Not important	The error doesn't cause issues using the system.

 

7.4 methods and techniques

 

 

Final and official tests are done using empty database which is similar to the actual productional database. During the test one should make sure the database stays consistent.

 

 

System tests are executed in specified order by using the defined test cases. System tests will output report of how the test cases have passed the test.

 

 

Module testing should be done using http://junit.sourceforge.net/ which is a tool to build in-code testcases. All main methods should have enough variable and global test cases. In-code tests should be implemented parallel to the actual system code. When releases of the software are published, the system tests are run before the release.

 

7.5 coverage

 

All test cases should be run correctly before the testing can be approved. When problem occurs, the test case should be repeated twice to confirm the errorneus functionality. When error has been fixed, the test is run again twice before it can be confirmed as fixed.

 

7.6 restrictions

 

Approval tests cannot be run forever because the project has time schedule. If problems start occuring before deadline, the errors are handled in importance-criticality-order.

 

8. Test cases

8.1 database testing

The database queries should be implemented in such a way that there is no way

to do an SQL-injection attack (ie. prepare and the ?-notation). This means that

there is little or no reason to test the database against malformed input as it can

be assumed it will fail gracefully. Also, since one cannot set any measurable limits

regarding the performance of the database, it shall only be tested so that it works

reasonably fast. This requirement affects mostly select-queries. If the used

queries are too slow, the queries and indexes shall be redesigned. If some of the

slow queries are impossible to redesign to be faster with the current database-engine,

the database engine shall be changed to something else.

8.2 related software testing

The product will be implemented in Java programming language that will use a MySQL (or similar) database to store the world map. The server extension will be made in Java JDK 1.5 and the example application’s environment is J2ME, to be more specific Java MIDP 2, where the MupeClient works. MupeClient is open source software; hence it could be implemented for different platform.

With those cases, whatever the Java language has provided by SUN, MySQL will use to construct the database or MupeClient are the open source software. All of them are the accomplished software for developing those do not require us to do more tests.

 

8.3 user interface testing

Software testing should be performed at the very beginning of the software development stage and every possible piece of code is tested. In our project, the player interface will generate the XML to the MupeClient. Accesses to game, game rules, and load map and attend to location classes. This is the only way that player could communicate with the game server and other players.

Hence, player interface testing will divide into two parts – one is the components test that will test the buttons on the screen with the mobile phone keyboard. Another is the menu application testing that will check screen shown for reaction.

 

8.3.1 User interface components testing

Actually, the main button on the screen is the “Menu”, which located on the left bottom. In the “Menu” button will include “Map”, “Resource”, “Harvest”, “Building”, “Help” and “Quit” elements.

Menu button: it is designed on left that should contrapose to the predefined of the mobile phone design, which is controlled by pressing the left softkey on mobile keyboard. By pressing, it will show drop-down menu and use “up” and “down” buttons on the keyboard to select the action. To use “back” softkey to drop out of this menu.

Map button: by using “up” and “down” buttons on the keyboard to move the light bar, by pressing the left button on mobile keyboard that will continue to map selection step. The right button will lead light bar back to main menu.

Resource button: by using “up” and “down” buttons on the keyboard to move the light bar, by pressing the left softkey on mobile keyboard that will continue. The right button will lead light bar back to main menu.

Harvest button: by using “up” and “down” buttons on the keyboard to move the light bar, by pressing the left softkey on mobile keyboard that will continue. The right button will lead light bar back to main menu.

Building button: by using “up” and “down” buttons on the keyboard to move the light bar, by a long push that will continue to show the submenu that include the names/pictures of the building. Using the direction control panel to move light bar and press the left softkey to select. But at this time, using right softkey the light bar will return to main menu.

Help button: by using “up” and “down” buttons on the keyboard to move the light bar, by pressing the left softkey on mobile keyboard to continue. The right softkey will lead light bar back to main menu.

Quit button: by using “up” and “down” buttons on the keyboard to move the light bar, by pressing the left softkey it will quit out this game.

Purpose of tests for component

In order to find unnecessary designs, error situation and make the system as perfect as possible, our system needs to be tested repeatedly.

Expected results

We hope the results can be matched between game designers and players’ familiar operation with the mobile phone. That is to say, every triggered operation can let players feel this game is easy to play and enable their conscience logical thinking models.

 

And another important button is the “Option” on the right bottom that is also a pull-up menu, which includes the “Refresh”, “Trade”, “Chat” and “My Status” elements. With the physical control, it is the same as operating the main menu – to use direction keys to move “up” and “down” and use “select” and “back” keys to choose and drop out of it.

 

8.3.2 User interface components application testing

After testing physical buttons control, we move to component application testing. Each element will trigger different screen reaction.

Menu button: as same as say above, when press the button to select that will show drop-down menu. “Up” and “Down” button on the direction control button will move the light bar.

 

Map button: when player chooses Map that should trigger another interface asking player to select which kind of map. The location maybe the same player is at in the real world (detected with GPS), or it can be chosen from the map.

Resource button: when player chooses this that will show a document about resources.

Harvest button: for this button that will link to database to show how many resources that player has at that time.

Building button: when a long push (over 2 seconds) on this button, the submenu will be shown. When a long push on any building, it will show how much will need to build this building and using pressing left key to select.

Help button: it will content a brief instruction document. But if a problem appears to the player, a context-depended online help will be available for the player, and will offer sufficient help.

Quit button: only regular to quit out the game.

 

Option button: it provide a menu from update the information moment by moment.

Refresh: this is the button to refresh the information on map in time. When player select it, every new action had been done will show on the map. This will use frequent by players.

Trade: this is function allow player to do business with others.

Chat: player will allow speaking to each other that are playing the game as same time.

My Status: this function will show player’s efforts. For example how buildings, points and other on-line information she/he has.

Purpose of tests for component application

For the application testing, there are seven main test cases that are classified from their functionality. Each testing is carried out individually in order to test whether these works well within the section.

Expected results

We would like to prove that sections work well. It should work according to our design.

 

Test Cases of Main Menu

 

Test case number	Test case name
831	Map selected
832	Resource
833	Harvest document
834	Building construction
835	Help link
836	Quit

 

Name	Map selected
ID	831
Module/Component	Main menu
Description	The player can load the real map according to play she/he stays by using location system; player can select a map from database.
Required test environment	User Interface Class is implemented
Initial condition	1. The product shall operate for everyone who has place-located system function in the mobile. 2. The product shall also operate with any MIDP 2.0 enable mobile device
Pre-case sequence	1. The phone has a connection with the server
Case sequence	1. Press Map button 2. Select one kind of map
Post-case sequence	Wait for map loading
Expected results	90% of first time players successfully add a location to the database in two minutes

 

Name	Resource
ID	832
Module/Component	Main menu
Description	The staff will use to construct the buildings
Required test environment	User Interface Class is implemented
Initial condition	1. The map is loaded
Pre-case sequence	1. The basic resources are available on the map
Case sequence	1. Press select button, it will show the resources list that are available 2. After appointed building has been built, the certain resource will available in the list 3. Use the back button to go back to main menu
Post-case sequence
Expected results	Information will be right and update in time.

 

Name	Harvest document
ID	833
Module/Component	Main menu
Description	The list will show the name of resources that player get and their amount
Required test environment	User Interface Class is implemented
Initial condition	1. The map is loaded 2. Player ask the work to obtain
Pre-case sequence	1. Resource is available
Case sequence	1. Press to show list 2. If needed, player could start trade with others
Post-case sequence	Could lead to trade
Expected results	Information will be right and update in time

 

Name	Building construct
ID	834
Module/Component	Main menu
Description	The available buildings can be constructed on the map
Required test environment	User Interface Class is implemented
Initial condition	1. The map is loaded 2. Player has collected some kinds of resources
Pre-case sequence	1. May need certain building has built already
Case sequence	1. A long push on .building. button 2. Move to submenu 3. A long push on certain building 4. Show the amount of resources that need 5. Press .select. key on the mobile
Post-case sequence	1. Dissipative resources will deduct from harvest document 2. If player wants quit submenu without build, press .back. button on the mobile keyboard. 3. If player.s harvest is not enough, it will come out a warning box that will note what resources are nor enough
Expected results	Information will be right and update in time

 

Name	Help link
ID	835
Module/Component	Main menu
Description	Related information for assisting play the game
Required test environment	User Interface Class is implemented
Initial condition
Pre-case sequence
Case sequence	1. A brief indication document 2. Online help is available
Post-case sequence
Expected results	If a problem appears to the player, a context-depended online help will be available for the player, and will offer sufficient help

 

Name	Quit
ID	836
Module/Component	Main menu
Description	To quit out of game
Required test environment	User Interface Class is implemented
Initial condition
Pre-case sequence
Case sequence	1. A dialog box to notify 2. Quit out of the game 3. Save the current personal status automatically
Post-case sequence
Expected results	Actions will be successful

 

Test Cases of Option button

 

Test case number	Test case name
837	Refresh
838	Trade
839	Chat
8310	My Status

 

Name	Refresh
ID	837
Module/Component	Option Button
Description	Refresh the information on map in time and This will use frequent by players
Required test environment	1. User Interface Class is implemented 2. The connection is built
Initial condition	1. The product shall operate for everyone who has place-located system function in the mobile. 2. The product shall also operate with any MIDP 2.0 enable mobile device
Pre-case sequence	The phone has connect with server
Case sequence	Refresh the map information
Post-case sequence	Wait for map reloading
Expected results	To show all of changes compare to last refresh time

 

Name	Trade
ID	838
Module/Component	Option Button
Description	Allow player to do business with others with their harvest
Required test environment	1. User Interface Class is implemented 2. The connection is built
Initial condition	1. The product shall operate for everyone who has place-located system function in the mobile. 2. The product shall also operate with any MIDP 2.0 enable mobile device
Pre-case sequence	1. The phone has connect with server 2. Player has some resources
Case sequence	1. Press trade button 2. To select .sell. or .buy. 3. To select which resource 4. To input the how much wants to trade 5. If someone answer, trade will happen
Post-case sequence	If no one answers, player can cancel the trade
Expected results	The business is performed in the game

 

Name	Chat
ID	839
Module/Component	Option Button
Description	Player will allow speaking to each other that are playing the game as same time.
Required test environment	1. User Interface Class is implemented 2. The connection is built
Initial condition	1. The product shall operate for everyone who has place-located system function in the mobile. 2. The product shall also operate with any MIDP 2.0 enable mobile device
Pre-case sequence	Connect to other person who is playing the game as same time
Case sequence	1. Get reply from other 2. Start chat
Post-case sequence	Use .back. key and go out of the dialog box
Expected results	Player can chat in through this game

 

Name	My Status
ID	8310
Module/Component	Option Button
Description	Show player's efforts. For example how buildings, points and other on-line information she/he has.
Required test environment	1. User Interface Class is implemented 2. The connection is built
Initial condition	1. The product shall operate for everyone who has place-located system function in the mobile. 2. The product shall also operate with any MIDP 2.0 enable mobile device
Pre-case sequence	The phone has connection with server
Case sequence	Show the information that player ask
Post-case sequence	Wait for page loading
Expected results	Information will be right and update in time

 

8.4 interface testing

As same as we had mention before, the architecture among the user interface class, database and game is like in diagram 1.

Hence, there is not any hardware interfaces have been defined here. The interface testing part will be skipped in our case.

8.5 printing testing

This part has no relevance in our project.

8.6 security testing

This game is one of applications of Mupe that will be implemented on mobile phone. Hence every player can have this game on his/her mobile phone if their phones support it. Every phone number is the unique certification; it does not ask more validating processes.

8.7 recovery testing

-- If the problem occurs during playing, player can easily quit the game. The game status will keep same as last time.

-- If the server has been cut down, player can easily quit the game. The game status will keep same as last time.

8.8 performance testing

-- Response times for web page loading.

-- The map database can be updated through a network connection

-- Testing that code doesn’t produce dead-locks.

-- The status of the system is shown always.

-- Help button is always shown.

-- Because the system will be used in public places with possible human traffic and quick situations, the system can't distract the user too much and has to be straightforward to use.

-- The user can exit the system with a total maximum of two pushes of a button.

 

8.9 regression testing

-- The system is tested with several hundred simultaneous users.

-- The capacity of the database is tested with several hundred simultaneous users.

-- Match between system and the real world

-- Database update and maintenance

 

9. Criteria and requirements

The testing manager shall have the power to decide when some part(s) of the testing

are accepted or rejected. His/her decision should be based on input he/she receives

from person(s) doing the testing.

9.1 acceptance

Testing is accepted if all critical tests pass and atleast 95% of other tests pass.

9.2 rejection

Testing shall be rejected if the test cases can't be tested properly due to badly

designed test cases, required functionality is missing or some technical problem

affects testing. The testing manager shall have the power to decide when a test case

is indeed badly designed.

9.3 requirements for termination (quitting) of testing

Testing shall be stopped, if such an error is found, that there is reason to suspect that

it will affect the outcome of other tests. Testing should also be stopped if so many

bugs are found that there is a reason to suspect there is a common reason for them.

The person ultimately reasonable for testing shall have the power to

decide which cases fulfill the aforementioned criteria.

9.4 requirements for continuing testing

When testing continues after a termination, all test cases which belong to the same

module shall be tested again. The same applies to all other tests, such that there

is reason to suspect that the fixed bug or its fix may have affected the outcome of the test(s).

9.5 requirements for ending testing

Testing shall be ended when one or more of the following cases is true:

All test cases pass in row atleast twice and critical tests (TBD) pass atleast five times

The time allocated (TBD) to testing is used

The date (TBD) in which testing has to be ended is met

 

The testing manager shall decide when testing can be ended.

9.6 requirements for rejecting code

The testing manager together with the project group can decide that

all code in some or all modules shall be rejected if the code in question

fails more than 75% of the tests regarding it.

 

10. Risk management

 

Testing related risks:

Risk	Mitigation
Time and effort spent to testing is exceeded	Only the planned test cases will be performed in scheduled testing phase. Unit tests will be created systematically already during coding phase to ensure that critical errors are found.
Test plans do not correspond to the actual program functionality	Test plan is double checked after implementation is finalised.

 

11. Schedule and tasks

 

Testing phase is scheduled to occur during weeks 13-17 with the following breakdown:

 

Week 13

• Database testing
• UI testing
• Game rules testing

Week 14

• Database testing
• UI testing
• Game rules testing

Week 15

• System testing
• Game rules testing

Week 16

• System testing
• Game rules testing

Week 17

• Regression testing
• Final test report creation

 

12. Acceptance

 

12.1 Evaluate testing and testing results

 

Each of found errors will be written in clear description including location of occurrence, test steps, reproducible or not, severity and priority of errors. Based on this information, developer team will decide what to fix first and also decide time used for correction.

 

Coverage of testing

Testing for final product will be done with real phones. It will cover all functionalities and all features of SW. We can run a Basic Acceptance testing which include a few test cases for each feature and run system testing, performance testing and field testing separately or run all test sets together in a smaller scale, this will be defined later.

 

Test procedures

Before testing starts, the test specification should be provided by SW designer and test responsible person. Testers perform testing according to test specification. However, testers are allowed to update it if needed. During testing, testers should write description for found errors as soon as they are identified. After testing is complete, test results will be submitted with error list to developer team.

 

Time issues

We should try to avoid being hurry in testing to ensure better quality of test results. In order to achieve this, we need to make good plan for scheduling SW developer’s work and testers’ work and we must follow the schedule. There is always risk to have defect not found or not fixed if time is running out, therefore, we should start the work earlier to avoid this kind of situation.

 

Note: reliability of test results and suggestions for improvement will be analyzed and provided after testing is done if there is any.

 

12.2 Acceptance of testing as a whole

 

Project design team and Xiaoqing Meng-Pitkänen design test specification together. Xiaoqing will be main responsible for writing test specification and test cases. It is necessary and important to get comment and approval from project contact person in Nokia. Riku Suomela will be approver for all test documents produced by project team. Estimated date of approval is 17th or 24th March, 2006.