Evaluating & Refining Solutions

AQA 4.13.1 — Evaluating and refining computational solutions

Retrieval: the previous lesson tested a working overlap-checking function against a genuine test plan. Passing every planned test is necessary, but — as this lesson shows — it is not the same question as "is this a good solution to the original problem."

Key vocabulary

• Efficiency — how well a solution's resource use (time, memory) scales as the problem grows.
• Robustness — how well a solution handles unexpected, invalid or extreme input without failing badly.
• Maintainability — how easily a solution can be correctly understood and modified later, by yourself or someone else.
• Usability — how well a solution serves the people who actually have to use it, not just whether it's technically correct.

Understand — evaluation returns to the requirements, not just the tests

A solution that passes every test in its own test plan can still be a poor solution, if the test plan itself didn't reflect everything the original specification (Lesson 1) actually required. Evaluation asks a broader question than testing: does this solution genuinely meet the stated requirements and constraints, including the non-functional ones (efficiency, robustness, maintainability, usability) that a pass/fail test plan doesn't naturally capture?

Evaluate it — the badminton system, against a changed constraint

The original specification assumed a fixed, small number of courts (4) and implicitly a small number of bookings. Suppose the sports centre now expands to 40 courts across three sites, with online booking open to thousands of members.

• Efficiency: a solution that checks every existing booking one-by-one (linear search) for every overlap check was entirely fine at a handful of bookings; at genuine scale, this is exactly Sequence 5/13's algorithm-performance reasoning applied for real — an approach whose cost grows linearly with the number of bookings starts to matter once "the number of bookings" is actually large.
• Robustness: an in-memory Python list of bookings that resets every time the program restarts was a reasonable simplification for a single-session prototype; a real, persistent, concurrently-used system genuinely needs the durability a database provides (Sequence 16) — this is a case where the original design decision should now be revisited, not just re-implemented at greater scale.
• Maintainability: if court-specific logic (opening hours, maintenance schedules) is about to grow, bundling a court's data and behaviour into a class (Sequence 11) becomes more justified now than it was for the original small, simple version — an example of a paradigm choice that was reasonably deferred earlier and is now worth revisiting.

Notice that "evaluate against a changed constraint" doesn't automatically mean "the original design was wrong" — it means explicitly re-checking whether the original justification (Lesson 2) still holds under the new conditions, and changing the design only where it genuinely no longer does.

Justify it — recommend a refinement, with evidence

A different classmate's badminton-booking solution stores every booking as a plain list, checked with a linear search on every overlap check, and works correctly at the current scale of 4 courts. Justify whether this specific implementation detail is worth changing right now, versus after the expansion described above, referring explicitly to at least one of efficiency, robustness or maintainability.

(A reasonable justified answer: not worth changing right now, on efficiency grounds specifically - at 4 courts and a small number of bookings, a linear search's cost is negligible regardless of Big-O, and refactoring now adds complexity and development time with no measurable benefit at the current scale. It becomes worth changing once genuine scale (efficiency) or persistence-across-restarts (robustness) requirements are actually present - evaluation should be evidence-based against the ACTUAL current requirements, not a general "more advanced is always better" instinct. An answer arguing FOR changing it now is also acceptable if it identifies a genuine, currently-present requirement - not just future-proofing for its own sake - that justifies the change immediately.)

Common mistake

Treating "could be made more efficient/robust/advanced" as automatically meaning "should be changed." Every refinement has a cost (development time, added complexity, new opportunities for bugs) — a genuine evaluation weighs that cost against evidence of an actual current or clearly-imminent requirement, exactly the same non-forced-superiority reasoning Sequence 17 established for paradigm choice, now applied to refinement decisions generally.

Check your understanding

A solution correctly passes every planned test but takes visibly longer to respond as more bookings are added. Which non-functional quality does this evaluation concern relate to, and what evidence (beyond "it passed its tests") would you need to gather to properly evaluate it? (This concerns efficiency. Evidence needed beyond passing tests: how the response time actually changes as the number of bookings grows (e.g. timing the function at increasing data sizes) - genuine evaluation of efficiency requires evidence about scaling behaviour, not just confirmation that the current, small-scale tests pass.)

Challenge

For any coding challenge you completed earlier in this course, evaluate it against the four qualities introduced in this lesson (efficiency, robustness, maintainability, usability), and justify one specific refinement you would make if you were revisiting it now.

Why this matters for the NEA

AQA's NEA evaluation section is marked on exactly this skill — evaluating a finished solution against its own original requirements with genuine evidence, and justifying real refinements rather than simply asserting the project "works well."

This sequence's problem-solving methodology — analyse, decompose, select and design, implement and test, evaluate and refine — is now complete. Five synoptic assessments apply it for real: combining Advanced Data Structures with Advanced Algorithms, Databases with Advanced Data Structures, Programming Paradigms with Functional Programming, an NEA-readiness assessment spanning exception handling, OOP and abstraction, and a whole-course capstone spanning the full two-year curriculum.

Looking ahead: Sequence 20 (Examination Preparation) shifts from building solutions to demonstrating everything you know under timed examination conditions.