← AS/A Level

CS · Chapter 26

Cambridge AS & A Level · Part 4 · Chapter 26

File Processing and Exception Handling

How a program keeps its records after it stops, whether one after another or straight to a calculated address, and how it survives run-time errors without crashing. Every worked example, task and exam-style question from the chapter is here, with labs to try each idea.

Guru Jazzy
Chapter 26 opener: program code that reads from a file

🎯 Learning objectives

By the end of this chapter you should be able to:

  • write code to perform file-processing operations:
    • open a file in read, write or append mode
    • Read a record from a file
    • Write a record to a file
    • On serial, sequential and random files
  • show understanding of an exception and the importance of exception handling
  • show understanding of when it is appropriate to use exception handling
  • Write code to use exception handling

🧱 26.01 Records

A record is a user-defined type: you decide which fields it has and the type of each field. Records were introduced in Chapter 16 (Section 16.01), and Section 13.02 has more. In this chapter they are what we store in files.

Worked Example 26.01 · Using records

A car manufacturer and seller wants to store details about cars. The details can be stored in a record structure. One field, VehicleID, is unique to each car, so it becomes the record key.

TYPE CarRecord
    DECLARE VehicleID          : STRING    // unique identifier and record key
    DECLARE Registration       : STRING
    DECLARE DateOfRegistration : DATE
    DECLARE EngineSize         : INTEGER
    DECLARE PurchasePrice      : CURRENCY
ENDTYPE

DECLARE ThisCar : CarRecord                    // one record
DECLARE Car : ARRAY[1:100] OF CarRecord        // details of 100 cars

The last line declares an array of records: 100 cars, each with all five fields. Car[7].EngineSize is the engine size of the seventh car.

The same record in Python, VB.NET and Java

Python and Java have no record type, so a class with only a constructor does the same job. Chapter 27 covers classes properly. VB.NET has Structure.

class CarRecord:                    # a class with only a constructor
    def __init__(self):             # constructor: sets the initial values
        self.VehicleID = ""
        self.Registration = ""
        self.DateOfRegistration = None
        self.EngineSize = 0
        self.PurchasePrice = 0.00

ThisCar = CarRecord()               # instantiates a car record
ThisCar.EngineSize = 2500           # assign a value to a field
Car = [CarRecord() for i in range(100)]   # a list of 100 car records
Car[1].EngineSize = 2500            # a field of the 2nd car in the list
Watch the index. The pseudocode array is ARRAY[1:100], so the 2nd car is Car[2]. Python lists and Java arrays start at 0, so the 2nd car is Car[1] in Python. The book's Java line car[2] is really the 3rd car. VB.NET's Dim Car(100) makes 101 elements, 0 to 100.

🔬 Lab: build a CarRecord

Fill in the fields and press Assign. Each field has a type, so the lab rejects a value that does not fit, just as a typed language would.

ThisCar is empty.

📁 26.02 File processing

In Chapter 13 (Section 13.06) we used text files to store and read lines of text. A text file only lets us write strings, in a serial or sequential way, and we can append strings to the end of the file.

To store records we create a binary file (see Chapter 16, Section 16.02). Records can be stored serially or sequentially, or each record can go to its own address in a random file, where it can be reached directly.

Random file: a file that stores records at specific addresses that can be accessed directly.

Table 26.01 · Operations for file processing

Structured EnglishPseudocode
Create a file and open it for writingOPENFILE <filename> FOR WRITE
Open a file in append modeOPENFILE <filename> FOR APPEND
Open a file for readingOPENFILE <filename> FOR READ
Open a file for random accessOPENFILE <filename> FOR RANDOM
Close a fileCLOSEFILE <filename>
Write a record to a filePUTRECORD <filename>, <identifier>
Read a record from a fileGETRECORD <filename>, <identifier>
Move to a specific disk address within the fileSEEK <filename>, <address>
Test for end of fileEOF(<filename>)
Serial, sequential or random? (teacher's note)
  • Serial: records are stored one after another in the order they arrive, with no sorting. New records are appended at the end. Good for logs and transaction files.
  • Sequential: records are stored one after another in order of the key field. Good for batch processing such as a monthly payroll, where every record is read in key order.
  • Random (direct access): a hashing function turns each key into an address, so a single record can be read or updated without reading the ones before it. Good when you need fast look-up of one record.

Both serial and sequential files are read from the start, record by record; only a random file can jump straight to a record.

🎮 Which mode?

Pick the OPENFILE mode each job needs.

💾 Sequential file processing

If a program holds an array of records, the data is lost when the program quits. So before it quits, the program can open a binary file and write the records one after another. Next time it runs, it reads them back into the array.

Worked Example 26.02 · Processing records in a sequential file

Table 26.02 shows the pseudocode for storing the car records from Worked Example 26.01 in a sequential file, and for reading them back.

Saving contents of array

OPENFILE CarFile FOR WRITE
FOR i ← 1 TO MaxRecords
    PUTRECORD CarFile, Car[i]
NEXT i
CLOSEFILE CarFile

Restoring contents of array

OPENFILE CarFile FOR READ
FOR i ← 1 TO MaxRecords
    GETRECORD CarFile, Car[i]
NEXT i
CLOSEFILE CarFile

Table 26.02 · Pseudocode for processing records

Processing records sequentially in Python, VB.NET and Java

import pickle      # this library is required to create binary files
Car = [CarRecord() for i in range(100)]

CarFile = open('CarFile.DAT', 'wb')    # open file for binary write
for i in range(100):                   # loop for each array element
    pickle.dump(Car[i], CarFile)       # write a whole record to the binary file
CarFile.close()                        # close file

CarFile = open('CarFile.DAT', 'rb')    # open file for binary read
Car = []                               # start with empty list
while True:                            # keep reading until end of file
    try:
        Car.append(pickle.load(CarFile))   # append record from file to end of list
    except EOFError:                   # CORRECTION: the book's loop has no way out;
        break                          # pickle.load raises EOFError at the end
CarFile.close()
Corrections to the book's code
  • Python: the book reads with while True: Car.append(pickle.load(CarFile)), which has no exit. At the end of the file pickle.load raises EOFError and the program crashes. Section 26.03 admits this. The version above catches it.
  • Java: the book loops for (int i = 1; i < 100; i++), which writes only 99 records and skips car[0]. The reading code also used car[i].vehicleID = … without ever creating car[i], which throws a NullPointerException. Both are fixed above, and the object version writes whole objects so they can be read back with one readObject() each.
  • Read in the same order you wrote: a binary file has no field labels. If you write engineSize before dateOfRegistration, you must read them back in that order too.

🔬 Lab: the file simulator

This is a pretend file on disk. Open it in a mode, then try PUTRECORD, GETRECORD and EOF. Each command you run appears in the log as pseudocode. Try the mistakes too: reading a file opened for WRITE, or reading past the end.

CLOSEDpointer: –ThisCar: –
CarFile.DAT is on the disk, closed and empty.

✏️ Task 26.01

  1. Write a complete program to save several car records to a sequential file.
  2. Write another program to read the file and display the contents on screen.
// Program 1: save several car records to a sequential file
DECLARE ThisCar : CarRecord
DECLARE NumberOfCars, Count : INTEGER
OUTPUT "How many cars? "
INPUT NumberOfCars
OPENFILE "CarFile.DAT" FOR WRITE
FOR Count ← 1 TO NumberOfCars
    INPUT ThisCar.VehicleID
    INPUT ThisCar.Registration
    INPUT ThisCar.DateOfRegistration
    INPUT ThisCar.EngineSize
    INPUT ThisCar.PurchasePrice
    PUTRECORD "CarFile.DAT", ThisCar
NEXT Count
CLOSEFILE "CarFile.DAT"

// Program 2: read the file and display every record
OPENFILE "CarFile.DAT" FOR READ
WHILE NOT EOF("CarFile.DAT")
    GETRECORD "CarFile.DAT", ThisCar
    OUTPUT ThisCar.VehicleID, " ", ThisCar.Registration, " ",
           ThisCar.EngineSize, " ", ThisCar.PurchasePrice
ENDWHILE
CLOSEFILE "CarFile.DAT"
import pickle

class CarRecord:
    def __init__(self):
        self.VehicleID = ""
        self.Registration = ""
        self.DateOfRegistration = ""
        self.EngineSize = 0
        self.PurchasePrice = 0.00

# ---------- Program 1: save ----------
NumberOfCars = int(input("How many cars? "))
CarFile = open('CarFile.DAT', 'wb')
for Count in range(NumberOfCars):
    ThisCar = CarRecord()
    ThisCar.VehicleID = input("Vehicle ID: ")
    ThisCar.Registration = input("Registration: ")
    ThisCar.DateOfRegistration = input("Date of registration: ")
    ThisCar.EngineSize = int(input("Engine size (cc): "))
    ThisCar.PurchasePrice = float(input("Purchase price: "))
    pickle.dump(ThisCar, CarFile)       # one record after another
CarFile.close()

# ---------- Program 2: display (needs the same CarRecord class) ----------
CarFile = open('CarFile.DAT', 'rb')
while True:
    try:
        ThisCar = pickle.load(CarFile)
    except EOFError:                    # no more records
        break
    print(ThisCar.VehicleID, ThisCar.Registration,
          ThisCar.EngineSize, ThisCar.PurchasePrice)
CarFile.close()

The second program must define the same CarRecord class, because pickle stores the class name and rebuilds objects of that class. Strictly, this file is serial (records are in the order typed). To make it sequential, sort the records by VehicleID before writing them.

🎯 Random-access file processing

Instead of keeping records in an array, we can store each record in a binary file as soon as it is created. A record can then be updated in situ: read it, change it and save it back in the same place.

This only works for fixed-length records, because every record slot must be the same size. A hashing function calculates an address from the record key, and the record is stored at that address in the file. It is just like a hash table (Chapter 23, Section 23.11). As with a hash table, two keys can give the same address. When that collision happens, the record goes in the next free record space.

Worked Example 26.03 · Processing records in a random-access file

Table 26.03 shows the pseudocode for storing a car record from Worked Example 26.01 in a random-access file, and for getting it back.

Saving a record

OPENFILE CarFile FOR RANDOM
Address ← Hash(ThisCar.VehicleID)
SEEK CarFile, Address
PUTRECORD CarFile, ThisCar
CLOSEFILE CarFile

Retrieving a record

OPENFILE CarFile FOR RANDOM
Address ← Hash(ThisCar.VehicleID)
SEEK CarFile, Address
GETRECORD CarFile, ThisCar
CLOSEFILE CarFile

Table 26.03 · Pseudocode for random-access file operations

SEEK moves a pointer to the given record address. PUTRECORD and GETRECORD then access the record at that address. After either command, the pointer points to the next record in the file.

Processing random-access records in Python, VB.NET and Java

The book's Python version:

import pickle    # this library is required to create binary files
ThisCar = CarRecord()

CarFile = open('CarFile.DAT', 'rb+')   # open file for binary read and write
Address = hash(ThisCar.VehicleID)
CarFile.seek(Address)
pickle.dump(ThisCar, CarFile)          # write a whole record to the binary file
CarFile.close()                        # close file

# to find a record from a given VehicleID:
CarFile = open('CarFile.DAT', 'rb')    # open file for binary read
Address = hash(VehicleID)
CarFile.seek(Address)
ThisCar = pickle.load(CarFile)         # load record from file
CarFile.close()

A version that really works (teacher's correction, explained below):

import struct, os

# Every record must be the SAME length, so pack the fields into fixed-size bytes:
# VehicleID 6 chars, Registration 10, Date 10, EngineSize int (4), Price double (8)
FMT = struct.Struct('<6s10s10sid')      # FMT.size = 38 bytes per record
SLOTS = 100

def Hash(VehicleID):                    # our own hash: gives the same slot every run
    Total = 0
    for ch in VehicleID:
        Total = Total + ord(ch)
    return Total % SLOTS                # slot number 0 to 99

if not os.path.exists('CarFile.DAT'):   # set up the file with 100 empty records
    with open('CarFile.DAT', 'wb') as f:
        f.write(bytes(FMT.size * SLOTS))

# saving a record
CarFile = open('CarFile.DAT', 'r+b')    # open for binary read and write
CarFile.seek(Hash(ThisCar.VehicleID) * FMT.size)   # byte offset = slot x record size
CarFile.write(FMT.pack(ThisCar.VehicleID.encode(), ThisCar.Registration.encode(),
                       ThisCar.DateOfRegistration.encode(),
                       ThisCar.EngineSize, ThisCar.PurchasePrice))
CarFile.close()

# finding a record from a given VehicleID
CarFile = open('CarFile.DAT', 'rb')
CarFile.seek(Hash(VehicleID) * FMT.size)
Fields = FMT.unpack(CarFile.read(FMT.size))
CarFile.close()
print(Fields[0].rstrip(b'\x00').decode(), Fields[3], Fields[4])

In Python and VB.NET the hash function must allow for the record size in bytes. If every record is 58 bytes, the 2nd record slot starts 58 bytes into the file, and the nth slot starts (n − 1) × 58 bytes in.

Corrections to the book
  • Byte positions start at 0. The book says the 2nd slot "starts at position 59" and the nth at "(n − 1) × 58 + 1". That counts bytes from 1. But seek() in Python and Position in VB.NET count from 0, so the 1st record is at 0, the 2nd at 58, and the nth at (n − 1) × 58. Seeking to 59 would land 1 byte inside the 2nd record.
  • Python's built-in hash() is the wrong tool. For a string it returns a huge number that changes every time Python starts (hash randomisation). A record saved today could not be found tomorrow. Write your own Hash function that returns slot × record size.
  • pickle records are not fixed length. A pickled record's size depends on its data, so one record can overwrite the start of the next. The corrected version uses struct to pack every record into exactly 38 bytes.

🔬 Lab: a hash file with collisions

This file has 10 record slots, each 58 bytes long. The hash function is Address ← CustomerID MOD 10, a small version of the book's MOD 1000. If a slot is taken, the record goes in the next free slot, wrapping round to slot 0.

All 10 slots are empty (CustomerID = 0).

🧮 Generator: where does record n start?

Byte positions count from 0. Type the byte offset you would pass to seek().

✏️ Task 26.02

Write a complete program to save several car records to a random-access file. Write another program to find a record in the random-access file using the record key, and display the record data on screen.

// Program 1: save several records to a random-access file
OPENFILE "CarFile.DAT" FOR RANDOM
FOR Count ← 1 TO 3
    INPUT ThisCar.VehicleID
    INPUT ThisCar.Registration
    INPUT ThisCar.DateOfRegistration
    INPUT ThisCar.EngineSize
    INPUT ThisCar.PurchasePrice
    Address ← Hash(ThisCar.VehicleID)
    SEEK "CarFile.DAT", Address
    PUTRECORD "CarFile.DAT", ThisCar
NEXT Count
CLOSEFILE "CarFile.DAT"

// Program 2: find one record using its key
OUTPUT "Vehicle ID to find: "
INPUT SearchID
OPENFILE "CarFile.DAT" FOR RANDOM
Address ← Hash(SearchID)
SEEK "CarFile.DAT", Address
GETRECORD "CarFile.DAT", ThisCar
IF ThisCar.VehicleID = SearchID
    THEN
        OUTPUT ThisCar.Registration, " ", ThisCar.EngineSize, " ", ThisCar.PurchasePrice
    ELSE
        OUTPUT "No record for ", SearchID
ENDIF
CLOSEFILE "CarFile.DAT"

Checking ThisCar.VehicleID = SearchID matters: the slot may be empty, or hold a different car whose key gave the same address. A full answer would then try the next slot, as in Exam question 1 b(iii). For a Python version, use the corrected struct code above inside a loop for the saving program.

⚠️ 26.03 Exception handling

Run-time errors happen for many reasons: division by zero, an invalid array index, or trying to open a file that does not exist. Run-time errors are called exceptions. They can be handled by an error subroutine called an exception handler, instead of letting the program crash.

In pseudocode, the error-handling structure is:

TRY
    <statementsA>
EXCEPT
    <statementsB>
ENDTRY

Any run-time error during <statementsA> is caught and handled by running <statementsB>. There can be more than one EXCEPT block, each handling a different type of exception. Sometimes a FINALLY block follows the handlers; it runs whether or not there was an exception.

VB.NET treats exceptions as abnormal, unpredictable situations. Python also uses exception handling as a normal flow-control structure, which is why the end of a pickled file is detected with EOFError. That is why Worked Example 26.02 needs exception handling: without it, reaching the end of the file crashes the program.

Some Python exceptions

IOErrora file cannot be opened, for example
ImportErrorPython cannot find the module
ValueErroran argument has an inappropriate value
KeyboardInterruptthe user presses Ctrl+C or Ctrl+Del
EOFErrora file read meets an end-of-file condition
ZeroDivisionErrora division by zero has been attempted

Some Java exceptions

IOExceptiona file cannot be opened, for example
ArithmeticExceptionan arithmetic error, such as division by zero
ArrayIndexOutOfBoundsExceptionthe program tries to use an array element outside the boundary

In Python 3, IOError is another name for OSError; FileNotFoundError is the specific one for a missing file. A list index out of range raises IndexError.

Worked Example 26.04 · A simple exception handler

A program asks the user to key in an integer, but the user could type something that is not an integer. That should not crash the program.

NumberString = input("Enter an integer: ")
try:
    n = int(NumberString)
    print(n)
except:
    print("This was not an integer")
Good habit: a bare except: catches everything, including Ctrl+C and your own typing mistakes in the code. Name the exception you expect (except ValueError:) so other bugs still show up.

More than one handler, plus FINALLY (teacher's example)

try:
    DataFile = open('Scores.txt', 'r')
    Total = 0
    for Line in DataFile:
        Total = Total + int(Line)      # ValueError if a line is not a number
    print("Total:", Total)
except FileNotFoundError:              # a more specific kind of IOError
    print("Scores.txt is missing")
except ValueError:
    print("The file contains a line that is not a whole number")
finally:
    print("Finished")                  # runs whether or not there was an exception

🔬 Lab: trace an exception

Type what the user enters, then press Step to run one line at a time. Try 2, abc, 0, 5 and -1. Watch which except catches each one, and notice that finally always runs.

Screen

Press Step to start.

🎮 Name that exception

✏️ Task 26.03

Add exception-handling code to your programs for Task 26.01 or Task 26.02. Test that your code handles exceptions without the program crashing.

import pickle

FileName = input("File to read: ")
try:
    CarFile = open(FileName, 'rb')
except FileNotFoundError:
    print("There is no file called", FileName)
else:                                  # only runs if open() worked
    Count = 0
    while True:
        try:
            ThisCar = pickle.load(CarFile)
        except EOFError:
            break                      # normal end of the file
        except pickle.UnpicklingError:
            print("That is not a file of car records")
            break
        Count = Count + 1
        print(ThisCar.VehicleID, ThisCar.Registration)
    CarFile.close()
    print(Count, "records read")

Test plan: a file that exists (normal data); a filename that does not exist (expect the "no file" message); an empty file (expect "0 records read"); a file that is not pickled records, e.g. a text file (expect the "not a file of car records" message). In the saving program, also catch ValueError when the engine size or price is typed as text.

💭 Reflection Point

Can you explain the difference between serial, sequential and random-access files? Give an example of when each is appropriate.

🎮 Serial, sequential or random?

📌 Summary

  • Records are user-defined types.
  • Records can be stored in files in a serial, sequential or random (direct-access) manner.
  • Exception handling is advisable to avoid program crashes due to run-time errors.

🃏 Key-term flashcards

Tap a card to flip it.

✅ Quick quiz

📝 Exam-style questions

Write your answer first, then open the model answer. These are my mark-scheme-style answers, not official mark schemes.

1 A company stores details about their customers in a binary file of records.

  • The key field of a customer record is the customer ID (a number between 100001 and 999999).
  • The name of the customer is stored in a 30-character field.
  • The customer's telephone number is stored in a 14-character field.
  • The total value of orders so far is stored in a currency (decimal) field.
TYPE CustomerRecord
    DECLARE CustomerID  : INTEGER    // key field: 100001 to 999999
    DECLARE Name        : STRING     // 30 characters
    DECLARE Telephone   : STRING     // 14 characters (a string: may start with 0 or +)
    DECLARE TotalOrders : REAL       // currency value of orders so far
ENDTYPE

DECLARE CustomerData : ARRAY[0:999] OF CustomerRecord
class CustomerRecord:
    def __init__(self):
        self.CustomerID = 0          # 0 means "empty slot"; real IDs are 100001-999999
        self.Name = ""               # up to 30 characters
        self.Telephone = ""          # up to 14 characters
        self.TotalOrders = 0.0       # currency

CustomerData = [CustomerRecord() for i in range(1000)]   # index 0 to 999

Marks for: the record header and end; each of the four fields with a sensible type; the telephone number as a string (it can start with 0 or +, and you never do arithmetic on it); the array with bounds 0 to 999 of the record type.

def Hash(CustomerID):
    Address = CustomerID % 1000
    return Address

def AddRecord(Customer):
    Index = Hash(Customer.CustomerID)
    Start = Index
    while CustomerData[Index].CustomerID != 0:     # slot already in use: collision
        Index = (Index + 1) % 1000                  # try the next slot, wrapping to 0
        if Index == Start:                          # gone all the way round
            print("Hash table is full")
            return
    CustomerData[Index] = Customer

def FindRecord(CustomerID):
    Index = Hash(CustomerID)
    Start = Index
    while CustomerData[Index].CustomerID != CustomerID:
        if CustomerData[Index].CustomerID == 0:     # empty slot: record was never added
            return -1
        Index = (Index + 1) % 1000
        if Index == Start:                          # searched every slot
            return -1
    return Index

The same FindRecord in pseudocode:

FUNCTION FindRecord(CustomerID : INTEGER) RETURNS INTEGER
    DECLARE Index, Start : INTEGER
    Index ← Hash(CustomerID)
    Start ← Index
    WHILE CustomerData[Index].CustomerID <> CustomerID
        IF CustomerData[Index].CustomerID = 0
            THEN
                RETURN -1                     // empty slot: not in the table
        ENDIF
        Index ← (Index + 1) MOD 1000
        IF Index = Start
            THEN
                RETURN -1                     // whole table searched
        ENDIF
    ENDWHILE
    RETURN Index
ENDFUNCTION

Marks for: calling Hash; testing whether the slot is empty; moving to the next slot on a collision; wrapping from 999 back to 0 (MOD 1000); stopping when the table is full; storing the record. For FindRecord: comparing the key in the slot; stopping at an empty slot or after a full loop; returning the index, or a rogue value such as −1 when the record is not found.

import pickle

CustomerFile = open('CustomerData.DAT', 'wb')      # binary file, write mode
for Index in range(1000):                          # every slot, empty ones too,
    pickle.dump(CustomerData[Index], CustomerFile) # so each record goes back to its slot
CustomerFile.close()
OPENFILE "CustomerData.DAT" FOR WRITE
FOR Index ← 0 TO 999
    PUTRECORD "CustomerData.DAT", CustomerData[Index]
NEXT Index
CLOSEFILE "CustomerData.DAT"

Write all 1000 slots, empty ones included, in index order. Then reading them back in order restores every record to the same slot, so FindRecord still works without re-hashing.

  • The file must be created first, filled with 1000 empty (dummy) records so that every address exists.
  • Every record must be fixed length (30-character name, 14-character phone) so each slot is the same size.
  • Open the file FOR RANDOM instead of using the array.
  • Hash still gives the slot; the byte address is slot × record size. Use SEEK to move there.
  • AddRecord: GETRECORD to see if the slot is empty; if not, move to the next slot (wrap at the end of the file); then PUTRECORD the customer at that address.
  • FindRecord: SEEK and GETRECORD from the file, comparing keys and probing as before.
  • Part (c) is no longer needed: each record is already saved on disk the moment it is written.
  • Close the file when finished, and add exception handling for file errors.
FileOpened = False
while not FileOpened:
    FileName = input("Which file do you want to use? ")
    try:
        DataFile = open(FileName, 'r+b')   # binary read/write, like FOR RANDOM
        FileOpened = True
    except IOError:                        # includes FileNotFoundError
        print("That file does not exist. Please try again.")
DECLARE FileOpened : BOOLEAN
FileOpened ← FALSE
REPEAT
    OUTPUT "Which file do you want to use? "
    INPUT FileName
    TRY
        OPENFILE FileName FOR RANDOM
        FileOpened ← TRUE
    EXCEPT
        OUTPUT "That file does not exist. Please try again."
    ENDTRY
UNTIL FileOpened = TRUE

Marks for: prompt and input; the open statement inside try; an except for the file error; a helpful message; a loop so the user can try again until the file opens.

Guru Jazzy · Cambridge AS & A Level Computer Science · Chapter 26 File Processing and Exception Handling · Based on the Cambridge coursebook (Langfield & Duddell); model answers are my own