Cambridge AS & A Level · Part 2 · Chapter 15
Software Development
How real programs get built, checked and kept alive: life-cycle models, structure charts, state-transition diagrams, the three kinds of error, every testing method, test data and the three kinds of maintenance. Every worked example, task and exam-style question from the chapter is here, with labs to try each idea.
Guru Jazzy🎯 Learning objectives
By the end of this chapter you should be able to:
- explain the purpose of a development life cycle, and why different programs need different life cycles
- describe the principles, benefits and drawbacks of the waterfall, iterative and rapid application development (RAD) models
- explain the analysis, design, coding, testing and maintenance stages of the program development cycle
- use a structure chart to break a problem into sub-tasks and to show the parameters passed between modules; describe its purpose, construct one, and derive equivalent pseudocode from it
- explain the purpose of state-transition diagrams for documenting an algorithm
- show how faults can be exposed and avoided; locate, identify and correct syntax, logic and run-time errors
- explain the testing methods: dry-run, walkthrough, white-box, black-box, integration, alpha, beta, acceptance, stub, and choose suitable data for each
- explain why a test strategy and test plan are needed, and choose normal, abnormal and extreme/boundary test data
- explain why systems need continuing maintenance, and the difference between corrective, adaptive and perfective maintenance
- analyse an existing program and make amendments to enhance its functionality
🔄 15.01 Stages in the program development life cycle
To write any program you design a solution (in structured English, a flowchart or pseudocode, Chapters 12 and 13), write the code and test it. For large systems with many people involved, these stages become more formal. A problem has to be analysed before anyone designs a solution, and once the program is in use, issues will appear that need changes: this is maintenance.
One way to think about it
Small programs can be proved correct, but for large systems it is practically impossible to test every combination of inputs, hardware and timing. Dijkstra's point (Section 15.05) is that testing can only show that bugs are present, never that they are absent. So we aim to reduce errors to an acceptable level with good design, testing and maintenance.
The program development life cycle Figure 15.01
The cycle runs analysis → design → coding (implementation) → testing → maintenance. When maintenance can no longer keep the program fit for purpose, development starts again, which is why it is a cycle. There are several development methodologies; the syllabus names three.
🔬 Lab: put the stages in order
Use the arrows to move each stage. Then check.
💧 The waterfall model Figure 15.02
The downward arrows show that the results of one stage are the input to the next. The upward arrows show that you often have to go back to an earlier stage to finish the current one.
✅ Benefits
- Simple to understand: the stages are clearly defined.
- Easy to manage: fixed stages, each with specific outcomes.
- Stages are processed and completed one at a time.
- Works well for smaller projects whose requirements are very well understood.
⚠️ Drawbacks
- No working software until late in the life cycle.
- Not a good model for complex and object-oriented projects.
- A poor model for long and ongoing projects.
- Cannot accommodate changing requirements.
- Hard to measure progress within a stage.
- Integration is done at the very end, so technical or business problems are found late.
🔁 The iterative model
An iterative model does not start with a full requirements specification. Development starts by implementing a small subset of the requirements; repeated (iterative) reviews identify further requirements until the complete system emerges.
✅ Benefits
- A working model exists very early, so functional or design flaws are found early and corrected quickly.
- Some working functionality is developed quickly and early.
- Results are obtained early and periodically.
- Parallel development can be planned, and progress can be measured.
- Changing the scope or requirements costs less.
- Testing and debugging a small subset is easy.
- Risks are identified and resolved during each iteration; the high-risk part is done first.
- Every increment delivers an operational product, and lessons from one increment feed the next.
- Better suited to large and mission-critical projects.
- Software is produced early, so customers can evaluate it and give feedback.
⚠️ Drawbacks
- Only large projects really benefit: a small system is hard to split into smaller serviceable modules.
- More resources may be needed.
- Design issues can arise because not all requirements are gathered at the start.
- Defining the increments may need a definition of the complete system.
⚡ The Rapid Application Development (RAD) model
RAD uses minimal planning and relies on prototyping: a prototype is a working model of part of the solution. Modules are developed in parallel as prototypes and integrated into the complete product for faster delivery. There is no detailed preplanning; changes are made during development. Analysis, design, coding and testing are folded into a series of short, iterative development cycles.
✅ Benefits
- Changing requirements can be accommodated.
- Progress can be measured.
- Productivity rises with fewer people in a short time.
- Reduces development time.
- Increases reusability of components.
- Quick initial reviews; encourages customer feedback.
- Integrating from the very beginning solves many integration issues.
⚠️ Drawbacks and limits
- Only systems that can be modularised can be built with RAD.
- Needs highly skilled developers and designers.
- Suits systems that are component-based and scalable.
- Needs user involvement throughout the life cycle.
- Suits projects that need shorter development times.
🎮 Which model?
Read the statement or scenario and pick the life-cycle model it describes best.
🧩 15.02 Program design using structure charts
Another approach to modular design is to choose the sub-tasks and draw a structure chart showing how the modules relate. Each box is a module; each level is a refinement of the level above.
A structure chart also shows the interface between modules: the variables passed between them, called parameters (Section 14.10). A parameter supplying a value to a lower-level module is drawn as a downward arrow; one supplying a new value back to the module above is drawn as an upward arrow.
Averages Figure 15.03
The top box (level 0) names the module. It is refined into three level-1 sub-tasks. Number1 and Number2 come up from INPUT numbers, go down into Calculate average, and Average comes back up and is passed down to OUTPUT average. This parameter passing is the interface.
Model answer
🔀 Selection and repetition in a structure chart
Structure charts can also show control information. The number-guessing game from Section 12.06 can be modularised like this:
The diamond is a condition that is either TRUE or FALSE, so one branch or the other is followed.
Figure 15.05 is the pyramid-drawing program from Worked Example 12.10. The semi-circular arrow means the modules underneath it are repeated; the label gives the loop condition.
Model answer
Model answer
🔴 Connect 4 Figure 15.06
Structure charts help programmers see how modules relate and how they interface, which matters more as the problem grows. Figure 15.06 is the chart for the Connect 4 program (Task 13.06). Two extra symbols:
- An arrow with a solid round end (●→) shows the value passed is a flag (a Boolean).
- A double-headed arrow (←○→) shows the variable's value is updated within the module.
📜 15.03 Deriving pseudocode from a structure chart
Back to the pyramid (Figure 15.05). In Worked Example 12.10 every variable was global. Now we use local variables and parameters, so each module is self-contained and a change to a variable cannot accidentally affect a value elsewhere.
The top-level module, Pyramid, calls four modules. Each call supplies its parameters in brackets after the module identifier:
MODULE Pyramid
CALL SetValues(NumberOfSymbols, NumberOfSpaces, Symbol, MaxNumberOfSymbols)
REPEAT
CALL OutputSpaces(NumberOfSpaces)
CALL OutputSymbols(NumberOfSymbols, Symbol)
CALL AdjustValuesForNextRow(NumberOfSpaces, NumberOfSymbols)
UNTIL NumberOfSymbols > MaxNumberOfSymbols
ENDMODULE
PROCEDURE SetValues(NumberOfSymbols, NumberOfSpaces, Symbol, MaxNumberOfSymbols)
INPUT Symbol
CALL InputMaxNumberOfSymbols(MaxNumberOfSymbols)
NumberOfSpaces ← (MaxNumberOfSymbols - 1) / 2
NumberOfSymbols ← 1
ENDPROCEDURE
PROCEDURE InputMaxNumberOfSymbols(MaxNumberOfSymbols)
REPEAT
INPUT MaxNumberOfSymbols
UNTIL MaxNumberOfSymbols MOD 2 = 1
ENDPROCEDURE
PROCEDURE OutputSpaces(NumberOfSpaces)
FOR Count ← 1 TO NumberOfSpaces
OUTPUT Space // without moving to next line
NEXT Count
ENDPROCEDURE
PROCEDURE OutputSymbols(NumberOfSymbols, Symbol)
FOR Count ← 1 TO NumberOfSymbols
OUTPUT Symbol // without moving to next line
NEXT Count
OUTPUT Newline // move to the next line
ENDPROCEDURE
PROCEDURE AdjustValuesForNextRow(NumberOfSpaces, NumberOfSymbols)
NumberOfSpaces ← NumberOfSpaces - 1
NumberOfSymbols ← NumberOfSymbols + 2
ENDPROCEDURE Note that a structure chart does not say how parameters are passed: by reference or by value. Here SetValues, InputMaxNumberOfSymbols and AdjustValuesForNextRow change their parameters, so those must be passed BYREF for the changes to reach Pyramid.
Teacher's note: the chart's label says "UNTIL NumberOfSymbols = MaxNumberOfSymbols", but the pseudocode tests >. Because NumberOfSymbols grows by 2 after the row is printed, testing = would stop one row early and miss the base. Trust the pseudocode.
🔬 Lab: run the modules
Enter the symbol and a maximum. The log shows each CALL and the parameter values passed, just like walking down the structure chart. Try an even number to see InputMaxNumberOfSymbols reject it.
- Write pseudocode to implement the structure chart for the average of two numbers (Figure 15.03; the book refers to it as Figure 12.03).
- Write pseudocode to implement the structure chart for the number-guessing game (Figure 15.04; the book says Figure 12.04).
- Amend the pseudocode from Worked Example 13.05 to implement the interface shown in the Connect 4 structure chart (Figure 15.06).
Model answer 1: averages
MODULE Averages
CALL InputNumbers(Number1, Number2)
CALL CalculateAverage(Number1, Number2, Average)
CALL OutputAverage(Average)
ENDMODULE
PROCEDURE InputNumbers(BYREF Number1 : REAL, BYREF Number2 : REAL)
INPUT Number1
INPUT Number2
ENDPROCEDURE
PROCEDURE CalculateAverage(BYVAL Number1 : REAL, BYVAL Number2 : REAL, BYREF Average : REAL)
Average ← (Number1 + Number2) / 2
ENDPROCEDURE
PROCEDURE OutputAverage(BYVAL Average : REAL)
OUTPUT "The average is ", Average
ENDPROCEDURE Model answer 2: number-guessing game (one guess)
MODULE NumberGuessingGame
CALL GenerateSecretNumber(SecretNumber)
CALL InputGuess(Guess)
CALL OutputMessage(Guess, SecretNumber)
ENDMODULE
PROCEDURE GenerateSecretNumber(BYREF SecretNumber : INTEGER)
SecretNumber ← INT(RAND(100)) + 1 // 1 to 100
ENDPROCEDURE
PROCEDURE InputGuess(BYREF Guess : INTEGER)
OUTPUT "Guess a number: "
INPUT Guess
ENDPROCEDURE
PROCEDURE OutputMessage(BYVAL Guess : INTEGER, BYVAL SecretNumber : INTEGER)
IF Guess = SecretNumber
THEN
OUTPUT "Well done, you guessed it!" // OUTPUT congratulation
ELSE
OUTPUT "Sorry, the number was ", SecretNumber // OUTPUT consolation message
ENDIF
ENDPROCEDURE Outline for 3: Connect 4 interface
Worked Example 13.05 is in Chapter 13. Rewrite its top level so every module gets its data through parameters, exactly as Figure 15.06 shows:
MODULE Connect4
CALL InitialiseBoard(Board) // Board ↑
CALL SetUpGame(ThisPlayer, GameFinished) // ThisPlayer ↑, GameFinished ●↑
WHILE NOT GameFinished
CALL PlayerMakesMove(Board, ThisPlayer) // Board ↕, ThisPlayer ↓
CALL OutputBoard(Board) // Board ↓
CALL CheckGameFinished(Board, ThisPlayer, GameFinished)
IF NOT GameFinished
THEN
CALL SwapPlayer(ThisPlayer) // ThisPlayer ↕
ENDIF
ENDWHILE
ENDMODULE
PROCEDURE PlayerMakesMove(BYREF Board : ARRAY, BYVAL ThisPlayer : CHAR)
CALL PlayerChoosesColumn(Board, ColumnNumber) // ColumnNumber ↑
CALL FindFreeRow(Board, ColumnNumber, RowNumber) // RowNumber ↑
Board[RowNumber, ColumnNumber] ← ThisPlayer
ENDPROCEDURE CheckGameFinished calls CheckIfPlayerHasWon (which calls CheckHorizontalLine and CheckVerticalLine and returns the flag WinnerFound) and, if no winner was found, CheckForFullBoard.
Suggested answer
Add Check diagonal lines as a third child of Check if player has won, beside the horizontal and vertical checks. It needs Board, RowNumber, ColumnNumber (and the player's token) passed down, and returns the flag WinnerFound. Yes, it is worth refining into two sub-modules, one for each diagonal direction (↗ and ↘), because each counts matching tokens both ways from the new token.
🚦 15.04 Program design using state-transition diagrams
A computer system can be seen as a finite state machine (FSM). An FSM has a start state, and an input makes it change (transform) from one state to another. The information about the states can be set out in a state-transition table.
Table 15.01 State-transition table
| Current state S1 | Current state S2 | ||
|---|---|---|---|
| input | a | S1 | S1 |
| b | S2 | S2 | |
- In state S1, input a causes no change of state; input b transforms S1 to S2.
- In state S2, input b causes no change; input a transforms S2 to S1.
A state-transition diagram describes the same behaviour. The start state has an arrow with a dot on its tail (●→). If the FSM has a final (halting) state, it is drawn as a double circle (S1 here).
If an input causes an output, the label has a vertical bar: input | output. In Figure 15.08, in state S1 the input b produces output c and moves the FSM to S2. An FSM with outputs is also called a Mealy machine.
🔬 Lab: drive the FSM of Figure 15.08
Press the inputs. Watch the current state and the output.
Creating a state-transition diagram for an intruder detection system
A program must simulate an intruder detection system. The system has a battery power supply. It is activated when the start button is pressed; pressing the start button while it is active has no effect. To de-activate it the operator enters a PIN. It goes into alert mode when a sensor is activated, and stays in alert mode for two minutes. If it has not been de-activated within two minutes, an alarm bell rings.
Table 15.02 State-transition table for the intruder detection simulation
| Current state | Event | Next state |
|---|---|---|
| System inactive | Press start button | System active |
| System active | Enter PIN | System inactive |
| System active | Activate sensor | Alert mode |
| System active | Press start button | System active |
| Alert mode | Enter PIN | System inactive |
| Alert mode | 2 minutes pass | Alarm bell ringing |
| Alert mode | Press start button | Alert mode |
| Alarm bell ringing | Enter PIN | System inactive |
| Alarm bell ringing | Press start button | Alarm bell ringing |
The start state is "System inactive". From the table we can draw the diagram:
🚨 Simulator
This is a working simulation. The row of Table 15.02 that fires is highlighted. "2 minutes pass" is a button so you don't have to wait; or tick real timer to have it fire after 10 seconds (sped up).
Model answer in pseudocode
DECLARE State : STRING
DECLARE Event : STRING
State ← "System inactive"
REPEAT
OUTPUT "State: ", State
INPUT Event // "start", "PIN", "sensor" or "timeout"
CASE OF State
"System inactive":
IF Event = "start" THEN State ← "System active" ENDIF
"System active":
IF Event = "PIN" THEN State ← "System inactive" ENDIF
IF Event = "sensor" THEN State ← "Alert mode"
StartTime ← NOW() // begin 2-minute count
ENDIF
"Alert mode":
IF Event = "PIN" THEN State ← "System inactive" ENDIF
IF NOW() - StartTime >= 120 THEN
State ← "Alarm bell ringing"
OUTPUT "ALARM!"
ENDIF
"Alarm bell ringing":
IF Event = "PIN" THEN State ← "System inactive" ENDIF
ENDCASE
UNTIL FALSE // the system runs until the battery is removed Any event not listed for a state (for example "start" while active) leaves the state unchanged, which matches the self-loops in Figure 15.09.
Creating a state-transition diagram for a two's complement FSM
An FSM takes a positive binary integer as input, one bit at a time, least significant bit first, and converts it into its two's complement negative equivalent:
- Output the bits input up to and including the first 1.
- Output the other bits following this scheme: for each 1, output a 0; for each 0, output a 1.
Table 15.03 State-transition table with outputs
| Current state | S1 | S1 | S2 | S2 |
|---|---|---|---|---|
| Input bit | 0 | 1 | 0 | 1 |
| Next state | S1 | S2 | S2 | S2 |
| Output bit | 0 | 1 | 1 | 0 |
🔬 Lab: step through the two's complement FSM
Type a positive binary number (or generate one). The FSM reads it from the right (least significant bit first).
Input (read right to left):
Output so far:
0101?
Answer
Reading from the least significant bit: 1 → output 1 (S1→S2); 0 → output 1; 1 → output 0; 0 → output 1. Written most significant bit first the output is 1011, which is −5 in 4-bit two's complement (0101 = +5). Check it in the lab above.
Answer
Yes, because taking the two's complement is its own inverse: applying it twice gives the original number. For example 1011 (−5) goes in and 0101 (+5) comes out. The one exception is the most negative number (e.g. 1000 = −8 in 4 bits), which comes out unchanged because +8 cannot be represented in 4 bits.
🐞 15.05 Types of error
Why errors occur and how to find them
Software may not do what it should because:
- The programmer made a coding mistake
- the requirements specification was not drawn up correctly
- the software designer made a design error
- the user interface is poorly designed and the user makes mistakes
- the computer hardware fails.
If the end user finds the error, that is bad for the developer's reputation, so software must be tested before release. Research shows that the earlier an error is found, the cheaper it is to fix, so testing goes on throughout development. The purpose of testing is to discover errors. As the Dutch computer scientist Edsger Dijkstra put it, testing can show the presence of bugs, but never their absence.
Syntax errors are the easy ones: the compiler or interpreter finds them and usually gives a hint. Many editors flag them as you type. A program that compiles successfully has no syntax errors left. With an interpreter that is not true: only statements about to be executed are syntax-checked, so an untested program may still hide syntax errors (Chapter 8, Section 8.05; Chapter 20, Section 20.06).
Figure 15.11: the compiler stops when it first notices the error, and the error is often on the previous line, because it cannot tell until it reaches the next line and finds an unexpected keyword.
Much harder to find are logic errors and run-time errors. A run-time error happens when execution comes to an unexpected halt ("crash") or goes into an infinite loop and "freezes".
Both can only be found by careful testing. They may appear only in certain circumstances: a program that crashes every time is obviously wrong, but one that works for months until one data set makes it fail is much harder to catch, sometimes only after serious consequences.
🎮 Syntax, logic or run-time?
🧪 15.06 Testing methods
Stub testing Figure 15.12
You may want to test a user interface before all its facilities exist. Write a stub for each procedure: a body containing only an output statement that confirms the call was made. Each menu option then calls its stub.
Black-box testing
The programmer can see the code (white-box testing, below). A program should also be tested by people who cannot see the code and do not know how it was written. They read the specification, devise test data, work out the expected results, run the program and record the actual results. The program is a "black box". Test data usually includes normal, extreme/boundary and erroneous/abnormal values.
Where actual and expected results differ, there is a problem. Black-box testing shows that there is an error; debugging software or dry-running is then used to find the lines that need correcting (Section 15.08).
White-box testing
To check that the code itself works, choose test data that checks every path through it.
White-box testing of pseudocode
The pseudocode from Worked Example 12.02:
INPUT Number1
INPUT Number2
INPUT Number3
IF Number1 > Number2
THEN // Number1 is bigger
IF Number1 > Number3
THEN
OUTPUT Number1
ELSE
OUTPUT Number3
ENDIF
ELSE // Number2 is bigger
IF Number2 > Number3
THEN
OUTPUT Number2
ELSE
OUTPUT Number3
ENDIF
ENDIF To test it we need four sets of numbers:
- The first number is the largest
- the first is larger than the second, and the third is the largest
- The second number is the largest
- the second is larger than the first, and the third is the largest.
The exact values don't matter; what matters is that they differ so that each part of the nested IF is checked. Parts not entered for a test are greyed out, so you can see every part has been checked after all four tests.
Table 15.04 Testing the validity of the nested IF statement
| Line of algorithm | Test 1 | Test 2 | Test 3 | Test 4 |
|---|---|---|---|---|
| INPUT Number1 | 15 | 12 | 12 | 8 |
| INPUT Number2 | 12 | 8 | 15 | 12 |
| INPUT Number3 | 8 | 15 | 8 | 15 |
| IF Number1 > Number2 | TRUE | TRUE | FALSE | FALSE |
| THEN IF Number1 > Number3 | TRUE | FALSE | ||
| THEN OUTPUT Number1 | Output 15 | |||
| ELSE OUTPUT Number3 | Output 15 | |||
| ELSE IF Number2 > Number3 | TRUE | FALSE | ||
| THEN OUTPUT Number2 | Output 15 | |||
| ELSE OUTPUT Number3 | Output 15 |
🔬 Lab: white-box coverage
Enter three numbers and run. The path taken lights up, and each of the four paths ticks off once you have covered it. Can you cover all four in four runs?
✍️ Dry-running an algorithm
A good way to check that an algorithm works is to dry-run it with a trace table and different test data, also called a walk through. You write down the current contents of every variable and condition at each step.
Tracing an algorithm
The number-guessing game:
SecretNumber ← 34
INPUT "Guess a number: " Guess
NumberOfGuesses ← 1
REPEAT
IF Guess = SecretNumber
THEN
OUTPUT "You took ", NumberOfGuesses, " guesses"
ELSE
IF Guess > SecretNumber
THEN
INPUT "Guess a smaller number: " Guess
ELSE
INPUT "Guess a larger number: " Guess
ENDIF
NumberOfGuesses ← NumberOfGuesses + 1
ENDIF
UNTIL Guess = SecretNumber The trace table has one column per variable plus one for the condition Guess > SecretNumber. We don't usually rewrite values that don't change: SecretNumber is only filled in once.
Table 15.05 Trace table for the number-guessing game
| SecretNumber | Guess | NumberOfGuesses | Guess > SecretNumber | Message |
|---|---|---|---|---|
| 34 | 5 | 1 | FALSE | ...larger... |
| 55 | 2 | TRUE | ...smaller... | |
| 30 | 3 | FALSE | ...larger... | |
| 42 | 4 | TRUE | ...smaller... | |
| 36 | 5 | TRUE | ...smaller... | |
| 33 | 6 | FALSE | ...larger... | |
| 34 | 7 | ... 7 guesses |
An entry is made only when an assignment happens, and values stay until overwritten, so a blank cell means the previous value remains. Start a new row for each iteration (each time round the loop).
🎲 Lab: play the game, watch the trace table
The computer picks a secret number from 1 to 100 (or press Book: 34). Each guess adds a row exactly as the algorithm would.
Tracing an algorithm: the improved bubble sort
To test the improved bubble sort of Worked Example 13.03, dry-run it with a trace table.
01 MaxIndex ← 7 02 n ← MaxIndex - 1 03 REPEAT 04 NoMoreSwaps ← TRUE 05 FOR j ← 1 TO n 06 IF MyList[j] > MyList[j + 1] 07 THEN 08 Temp ← MyList[j] 09 MyList[j] ← MyList[j + 1] 10 MyList[j + 1] ← Temp 11 NoMoreSwaps ← FALSE 12 ENDIF 13 NEXT j 14 n ← n - 1 15 UNTIL NoMoreSwaps = TRUE
Table 15.06 in the book traces the list 5 34 98 7 41 19 25. The lab below generates exactly that table, and the same table for any list you like, using the book's conventions: line numbers in the first column, and only changed values written in.
🔬 Table 15.06 generator
ConvertFromHex('A5').
FUNCTION ConvertFromHex(HexString : STRING) RETURNS INTEGER
DECLARE ValueSoFar, HexValue, HexLength, i : INTEGER
DECLARE HexDigit : CHAR
ValueSoFar ← 0
HexLength ← Length(HexString)
FOR i ← 1 TO HexLength
HexDigit ← HexString[i]
CASE OF HexDigit
'A': HexValue ← 10
'B': HexValue ← 11
'C': HexValue ← 12
'D': HexValue ← 13
'E': HexValue ← 14
'F': HexValue ← 15
OTHERWISE HexValue ← StringToInt(HexDigit)
ENDCASE
ValueSoFar ← ValueSoFar * 16 + HexValue
NEXT i
RETURN ValueSoFar
ENDFUNCTION Model answer
| HexString | HexLength | i | HexDigit | HexValue | ValueSoFar | Return |
|---|---|---|---|---|---|---|
| 'A5' | 2 | 0 | ||||
| 1 | 'A' | 10 | 10 | |||
| 2 | '5' | 5 | 165 | 165 |
A5₁₆ = 10 × 16 + 5 = 165. Try other strings:
These testing methods are used early in development, for example as individual modules are written. Sometimes programmers use them on their own code; larger organisations employ separate software testers.
One way to think about it
Almost always, yes. The programmer tests what they expect users to do and shares the same blind spots as their code. A tester works from the specification, tries odd inputs and unexpected sequences, and is not attached to the code, which is exactly why black-box testing by other people is part of thorough testing.
Integration, alpha, acceptance and beta testing
Software is often many modules written by different programmers. Each may pass its own tests, but once they are joined, the whole program must be tested: integration testing, usually done incrementally (add one module, test again, then add the next).
Before release, in-house testers carry out alpha testing. Bespoke software (written for a specific customer) is then handed over, and the customer checks it meets their requirements: acceptance testing, usually part of hand-over; when it passes, the customer signs off the software.
Software for general sale has no single customer, so after alpha testing a version is released to a limited audience of beta testers, who use it in their own environments and report problems back so the software house can fix them: beta testing.
🎮 Name the testing method
🗺️ 15.07 Test strategy, test plans and test data
During the design stage, a testing strategy must be worked out so the software is tested rigorously from the very beginning. Decide which testing methods suit the project and produce a carefully designed test plan.
Large programs cannot be tested exhaustively, but systematic testing should find as many errors as possible. First an outline test plan, for example:
- flow of control: does the user get appropriate choices, and does the chosen option go to the correct module?
- validation of input: has all data been entered into the system correctly?
- do loops and decisions perform correctly?
- Is data saved into the correct files?
- does the system produce the correct results?
The outline is then turned into a detailed test plan. To carry out the tests we choose test data: unlike real, live data, it is chosen specifically to test different possibilities.
Table 15.07 Types of test data
| Type of test data | Explanation |
|---|---|
| Normal (valid) | Typical data values that are valid. |
| Abnormal (erroneous) | Data values that the system should not accept. |
| Boundary (extreme) | Values at a boundary or an extreme end of the range of normal data. Test data should include values just within the boundary (valid) and just outside it (invalid). |
Designing test data
The pyramid problem (code in Section 14.13) has two inputs: the number of symbols in the base and the symbol itself. Consider just the number of symbols.
Table 15.08 Test data for the pyramid problem
| Type | Example values | Explanation |
|---|---|---|
| Normal (valid) | 7 | 7 is an odd integer, so it should be accepted. Any odd positive integer would do, but it should be bigger than 1 to check the pyramid is formed correctly. Testing more than one value is a good idea. |
| Abnormal (erroneous) | −7 8 7.5 '*' | Anything that is not a positive odd integer, which needs several tests: a negative integer, an even integer, a real number, non-numeric input. Don't take shortcuts such as one negative even integer, or one negative real: if the test fails you won't know whether it failed for one reason or both. |
| Boundary (extreme) | 1 0 79 81 | The smallest pyramid is one symbol, so 1 is just within the boundary. Test data raises questions for the design: should 0 be accepted? Is 0 even? Is it outside the boundary because a pyramid of 0 symbols is not a pyramid? Is there just one boundary? A pyramid that wraps around the screen would not look like a pyramid, so the program should limit the input to the screen width: with 80 characters across, 79 is just within and 81 just outside. Testing values just within the boundary also tests normal data, at the extreme ends of the range. |
🎮 Lab: normal, boundary or abnormal?
Each round gives a validation rule and a test value. Following Table 15.07, values just inside and just outside a limit both count as boundary data.
- Design test data for the number-guessing game (Task 14.09.2).
- Design test data for the Connect 4 game (Task 14.11).
Model answer 1: number-guessing game (guesses 1 to 100)
| Type | Guess | Expected result |
|---|---|---|
| Normal | 50, then guesses either side of the secret | "smaller" or "larger" hint; counter goes up by 1 each time |
| Normal | the secret number | congratulation message with the correct number of guesses (also test getting it on the first guess: "1 guesses") |
| Boundary | 1, 100 | accepted |
| Boundary | 0, 101 | rejected, asked again, not counted |
| Abnormal | −20, 37.5, "ten", empty input | rejected with an error message; the program must not crash |
Model answer 2: Connect 4 (7 columns, 6 rows)
| Type | Test | Expected result |
|---|---|---|
| Normal | column 4 on an empty board | token lands in the bottom row of column 4; player swaps |
| Boundary | columns 1 and 7 | accepted |
| Boundary | columns 0 and 8 | rejected: "column number not valid" |
| Boundary | a 7th token into a full column | rejected: no free row |
| Abnormal | "x", 3.5, −1 | rejected without crashing |
| Flow of control | four in a row horizontally; four vertically; four ending at the board edge; 42 tokens with no winner | correct winner announced; draw detected when the board is full |
How to prevent errors
The best way to get a program that works is to prevent errors in the first place. A major cause of errors is poor requirements analysis, so understand the problem and what the user wants or needs. Use:
- tried and tested design techniques, such as structured programming or object-oriented design
- conventions such as identifier tables, data structures and standard algorithms
- tried and tested modules or objects from program libraries.
🔧 15.08 Corrective maintenance
Programs don't wear out or need lubricating like machines. Corrective maintenance is the work needed when a program does not work correctly because of a logic error or a run-time error. Some errors only show up after a long time, under rare circumstances: a part of the program that is seldom used, or data that occasionally includes extreme values. Earlier corrective maintenance can also introduce new errors.
When a problem is reported, the programmer finds the cause of the bug with debugging software or a trace table (Section 15.06).
- Design a trace table for this algorithm:
INPUT BinaryString
StringLength ← Length(BinaryString)
FOR i ← 1 TO StringLength
Bit ← BinaryString[i]
BitValue ← IntegerValue(Bit) // convert string to integer
DenaryValue ← DenaryValue + 2 + BitValue
NEXT i - Dry-run the algorithm with
'101'as the input and complete the trace table. - The result should be 5. Can you find the error in the code and correct it?
🔬 Trace it: buggy or fixed
Model answer
Assuming DenaryValue starts at 0, the trace for '101' gives 3, 5, 8, so the result is 8, not 5. There are two faults:
- Logic error:
DenaryValue + 2 + BitValueshould beDenaryValue * 2 + BitValue(shift the value so far one place left, then add the new bit). DenaryValueis never initialised. AddDenaryValue ← 0before the loop; otherwise the result depends on whatever value the variable happens to hold.
INPUT BinaryString
StringLength ← Length(BinaryString)
DenaryValue ← 0
FOR i ← 1 TO StringLength
Bit ← BinaryString[i]
BitValue ← IntegerValue(Bit)
DenaryValue ← DenaryValue * 2 + BitValue
NEXT i
OUTPUT DenaryValue 🧭 15.09 Adaptive maintenance
Programs are often changed so they can do things they were not originally designed to do. The Connect 4 game (Worked Example 13.03) lets two players, O and X, play each other. An amended version could make one player the computer, so one person can play against it.
Model answer (one good strategy)
PROCEDURE ComputerChoosesColumn(BYREF Board : ARRAY, BYREF ColumnNumber : INTEGER)
// 1. Win if possible
FOR Col ← 1 TO 7
IF ColumnHasSpace(Board, Col) AND MoveWins(Board, Col, 'X')
THEN
ColumnNumber ← Col
RETURN
ENDIF
NEXT Col
// 2. Block the opponent's winning move
FOR Col ← 1 TO 7
IF ColumnHasSpace(Board, Col) AND MoveWins(Board, Col, 'O')
THEN
ColumnNumber ← Col
RETURN
ENDIF
NEXT Col
// 3. Otherwise prefer the centre, else any column with space
IF ColumnHasSpace(Board, 4)
THEN
ColumnNumber ← 4
ELSE
REPEAT
ColumnNumber ← INT(RAND(7)) + 1
UNTIL ColumnHasSpace(Board, ColumnNumber)
ENDIF
ENDPROCEDURE MoveWins drops the token into a copy of the board (using Find free row) and calls Check if player has won. In the main loop, when ThisPlayer = 'X' call ComputerChoosesColumn instead of Player chooses column.
✨ 15.10 Perfective maintenance
The program works, but there is still room to improve it. For example, it might run faster if file handling changed from sequential access to direct access.
FUNCTION GetPositiveNumber
DECLARE n : INTEGER
OUTPUT "Enter a positive number: "
INPUT n
RETURN n
ENDFUNCTION
// main program
REPEAT
Number1 ← GetPositiveNumber
IF Number1 <= 0
THEN
OUTPUT "Not a positive number: "
ENDIF
UNTIL Number1 > 0
REPEAT
Number2 ← GetPositiveNumber
IF Number2 <= 0
THEN
OUTPUT "Not a positive number: "
ENDIF
UNTIL Number2 > 0 Model answer
The validation loop is written out twice in the main program, so any change (say, a new error message or an upper limit) must be made in two places. The function's name promises a positive number, but it doesn't deliver one. Move the validation into the function, give it a proper return type, and the main program shrinks to two lines:
FUNCTION GetPositiveNumber() RETURNS INTEGER
DECLARE n : INTEGER
OUTPUT "Enter a positive number: "
INPUT n
WHILE n <= 0
OUTPUT "Not a positive number. Enter a positive number: "
INPUT n
ENDWHILE
RETURN n
ENDFUNCTION
// main program
Number1 ← GetPositiveNumber()
Number2 ← GetPositiveNumber() Further improvements: use a constant for the message, add comments, and use meaningful identifiers if the numbers have a purpose.
🎮 Corrective, adaptive or perfective?
📌 Summary
- The program development cycle has the stages analysis, design, coding, testing and maintenance.
- Structure charts are graphical representations of the modular structure of a solution.
- A structure chart shows the interface between modules: the parameters passed between the calling module and the module called.
- Structure charts show selection (a module called only under certain conditions) and repetition (modules called repeatedly).
- A state-transition diagram is another way of documenting an algorithm.
- Testing strategies include stub testing, black-box testing, white-box testing, integration testing, alpha and beta testing, and acceptance testing.
- Logic errors and run-time errors can be located and corrected by dry-running an algorithm or using a trace table.
- Corrective maintenance fixes bugs that come to light while the program is in use.
- Adaptive maintenance alters the algorithm and data structures in response to required changes.
- Perfective maintenance enhances performance or maintainability.
🃏 Key-term flashcards
Tap a card to flip it.
✅ Quick quiz
📝 Exam-style questions
Write your answer first, then open the model answer. These are my mark-scheme-style points, not official mark schemes.
1 Consider this function:
FUNCTION Binary(Number : INTEGER) RETURNS STRING
DECLARE BinaryString : STRING
DECLARE PlaceValue : INTEGER
BinaryString ← '' // empty string
PlaceValue ← 8
REPEAT
IF Number >= PlaceValue
THEN
BinaryString ← BinaryString & '1' // concatenates two strings
Number ← Number - PlaceValue
ELSE
BinaryString ← BinaryString & '0'
ENDIF
PlaceValue ← PlaceValue DIV 2
UNTIL Number = 0
RETURN BinaryString
ENDFUNCTION | Number | BinaryString | PlaceValue | Number >= PlaceValue |
|---|---|---|---|
| 11 | '' | 8 | |
| 3 | '1' | 4 | TRUE |
| '10' | 2 | FALSE | |
| 1 | '101' | 1 | TRUE |
| 0 | '1011' | 0 | TRUE |
Return value: '1011' (correct: 11 = 1011₂).
| Number | BinaryString | PlaceValue | Number >= PlaceValue |
|---|---|---|---|
| 10 | '' | 8 | |
| 2 | '1' | 4 | TRUE |
| '10' | 2 | FALSE | |
| 0 | '101' | 1 | TRUE |
Return value: '101', which is wrong (it should be '1010').
- Binary(11) is correct because 11 is odd:
Numberonly reaches 0 after the last place value (1) has been used. - Binary(10) stops as soon as
Numberbecomes 0, after the 2s column, so the trailing 0 for the 1s column is never added. - Fix: the loop must run for every place value:
UNTIL PlaceValue = 0(or a FOR loop over 4 place values), notUNTIL Number = 0.
🔬 Trace Binary(n) yourself
Try 0, 10, 15 and 20 in the book version. 20 never reaches 0, so the loop freezes: a run-time error.
2 A procedure to output a row in a tally chart:
PROCEDURE OutputTallyRow(NumberToDraw : INTEGER)
IF Count > 0
THEN
FOR Count ← 1 TO NumberToDraw
IF (Count MOD 5) = 0
THEN
OUTPUT('\') // every 5th bar slants the other way
ELSE
OUTPUT('/')
ENDIF
NEXT Count
ENDIF
OUTPUT NewLine // move to next row
ENDPROCEDURE | Test data | Type | Expected output / justification |
|---|---|---|
| 3 | Normal | ///: fewer than 5, so no backslash |
| 5 | Normal / boundary | ////\: the first group of five is completed |
| 12 | Normal | ////\////\//: checks the MOD 5 pattern repeats |
| 4 and 6 | Boundary | either side of the 5th bar, to check the backslash appears in exactly the right place |
| 1 | Boundary | /: the smallest row that draws anything |
| 0 | Boundary | only a new line; the loop must not run |
| −3 | Abnormal | nothing drawn, only a new line, and no crash |
| a large value, e.g. 200 | Extreme | does the row fit on the screen or wrap? |
Dry-running these exposes a fault: the procedure tests Count > 0 before Count has been given a value. The condition should be NumberToDraw > 0. (Strictly the IF is not needed at all, since FOR Count ← 1 TO 0 runs zero times.) Abnormal data of the wrong type, such as 2.5 or "a", should be rejected before the call, because the parameter is an INTEGER.
3 A random number generator is to be tested to see whether all numbers from 1 to 20 are generated equally often. In structured English:
Initialise a tally for the numbers 1 to 20 Repeatedly generate numbers in range 1 to 20 For each number generated, increment the relevant count Calculate how often each number should be generated (expected frequency) Output expected frequency Output the list of numbers as a table with actual frequency
| Identifier | Data type | Explanation |
|---|---|---|
| Tally | ARRAY[1:20] OF INTEGER | 1D array storing how many times each number has been generated |
| RandomNumber | INTEGER | the number generated |
| NumberOfTests | INTEGER | how many random numbers to generate (1000 here) |
| ExpectedFrequency | INTEGER | how often any one number would appear if all were equally likely (1000/20 here) |
- A: Initialise Tally (it returns
Tallyupwards) - B: Generate random number (it returns
RandomNumberupwards) - C:
RandomNumber(passed down into Update Tally) - D:
Tally(passed down and returned updated) - E:
Tally(passed down into Output Tally)
CONSTANT NumberOfTests = 1000
DECLARE Tally : ARRAY[1:20] OF INTEGER
DECLARE RandomNumber : INTEGER
MODULE RandomNumberTally
CALL InitialiseTally(Tally)
FOR Test ← 1 TO NumberOfTests // 1000 times
CALL GenerateRandomNumber(RandomNumber)
CALL UpdateTally(RandomNumber, Tally)
NEXT Test
CALL OutputTally(Tally)
ENDMODULE
PROCEDURE InitialiseTally(BYREF Tally : ARRAY)
FOR i ← 1 TO 20
Tally[i] ← 0
NEXT i
ENDPROCEDURE
PROCEDURE GenerateRandomNumber(BYREF RandomNumber : INTEGER)
RandomNumber ← INT(RAND(20)) + 1 // 1 to 20
ENDPROCEDURE
PROCEDURE UpdateTally(BYVAL RandomNumber : INTEGER, BYREF Tally : ARRAY)
Tally[RandomNumber] ← Tally[RandomNumber] + 1
ENDPROCEDURE
PROCEDURE OutputTally(BYVAL Tally : ARRAY)
DECLARE ExpectedFrequency : INTEGER
ExpectedFrequency ← NumberOfTests DIV 20
OUTPUT "Expected frequency: ", ExpectedFrequency
OUTPUT "Number Actual frequency"
FOR i ← 1 TO 20
OUTPUT i, " ", Tally[i]
NEXT i
ENDPROCEDURE 🔬 Run it
4 A car park has a barrier at the exit. It starts lowered. To exit, the driver inserts a coin; the barrier raises and lets the car out, then lowers after the car has passed. In an emergency, staff can open the barrier with a remote control; it stays open until the remote control lowers it again. The three states are lowered, raised and open:
| Current state | Event | Next state |
|---|---|---|
| Barrier lowered | Coin inserted | Barrier raised |
| Barrier lowered | Open remotely | Barrier open |
| Barrier open | Close remotely | Barrier lowered |
| Barrier raised | Car has exited | Barrier lowered |
- The start state (top left) is Barrier lowered.
- Top right: Barrier raised. Upper arrow (lowered → raised): Coin inserted. Lower arrow (raised → lowered): Car has exited.
- Bottom: Barrier open. Arrow down (lowered → open): Open remotely. Arrow up the left side (open → lowered): Close remotely.
🚧 Barrier simulator
Guru Jazzy · Cambridge AS & A Level Computer Science · Chapter 15 Software development · Figures from the Cambridge coursebook (Langfield & Duddell); model answers are my own