Learning objectives
Describe how the control unit, the arithmetic logic unit and the registers PC, IR, MAR, MDR and accumulator work together, and state what the address, data and control buses each carry
Distinguish single-core, multi-core and co-processor designs, explain why the many parallel cores of a GPU suit graphics, simulation and machine learning, and (HL) compare the CPU with the GPU and explain how the two share work
Explain the roles of registers, L1, L2 and L3 cache, RAM and ROM, and use cache hit and cache miss to explain how the memory hierarchy keeps the processor supplied
Describe the fetch–decode–execute cycle with the register and bus used at each step, and (HL) explain how pipelining and independent parallel cores raise throughput
Key definitions
Control unit (CU)
The part of the CPU that decodes the instruction held in the instruction register and sends the timing and control signals that direct memory, the ALU and the registers.
Arithmetic logic unit (ALU)
The part of the CPU that carries out arithmetic such as addition and subtraction, logical operations such as AND, OR and NOT, and comparisons on binary data.
Program counter (PC)
The register that holds the memory address of the next instruction to be fetched.
Instruction register (IR)
The register that holds the instruction that has just been fetched while the control unit decodes and executes it.
MAR and MDR
The memory address register holds the address of the location about to be read or written and places it on the address bus. The memory data register holds the data or instruction just read from memory, or waiting to be written to it.
Accumulator (AC)
A register that holds the value the ALU is working on and receives the result of each calculation.
Cache hit and cache miss
A cache hit occurs when the instruction or data the processor needs is already in the cache. A cache miss occurs when it is not, so it must be fetched from a slower level such as RAM.
Explanation
Inside the CPU. The central processing unit runs a program by fetching, decoding and executing its machine-code instructions one after another. The control unit coordinates everything: it decodes each instruction and, in step with the clock, sends the control signals that tell memory, the ALU and the registers what to do. The arithmetic logic unit performs arithmetic, logical operations and comparisons, and its result normally goes into the accumulator. Around them sit the registers, tiny stores inside the processor that are faster than any other memory. The guide names five: the program counter (address of the next instruction), the instruction register (the instruction being decoded and executed), the memory address register (the address about to be used), the memory data register (the value travelling to or from memory) and the accumulator (the working value and results).
Three buses. A bus is a set of parallel wires shared by several components. The address bus carries an address from the MAR to memory, in one direction only; its width fixes how many locations can be reached, since n lines give 2ⁿ different addresses (a 32-line bus reaches 2³² = 4 294 967 296 locations). The data bus carries the instruction or data itself between memory and the MDR, and it is two-way, because values are both read and written. The control bus carries control and timing signals such as memory read, memory write, the clock and interrupt requests; the CPU sends some of these and devices send others back, so every component knows which operation is happening and when. The width of the data bus sets how many bits move in one transfer.
The fetch–decode–execute cycle. Fetch: the address in the PC is copied into the MAR, which places it on the address bus; the control unit sends a memory-read signal on the control bus; the instruction stored at that address travels along the data bus into the MDR and is copied into the IR; the PC is incremented so that it points to the next instruction. Decode: the control unit splits the instruction in the IR into its opcode (what to do) and its operand (the value or address to use) and prepares the signals needed. Execute: the operation is carried out. For LOAD 250, the address 250 goes to the MAR and onto the address bus, a read signal is sent, and the value at 250 arrives in the MDR along the data bus before being copied into the accumulator. For ADD, the ALU adds the fetched value to the accumulator. For STORE, the accumulator’s value is placed in the MDR, the address in the MAR, a write signal is sent and the data bus carries the value to memory. A jump instruction simply overwrites the PC. The cycle then repeats.
Single-core, multi-core, co-processors and the GPU. A single-core processor has one set of control unit, ALU and registers, so it executes one instruction stream at a time, although it can still multitask by switching rapidly between programs. A multi-core processor places two or more independent cores on one chip; each fetches and executes its own instructions, so several threads really do run at the same moment, provided the software splits its work into threads. A co-processor takes one kind of work off the CPU — historically floating-point arithmetic, today graphics, audio signal processing or machine learning — leaving the CPU free for general control. The graphics processing unit is the most important co-processor. It contains hundreds or thousands of small, simple cores that apply the same operation to many data items at once. Rendering suits it perfectly: each of the 2 073 600 pixels of a 1920 × 1080 frame can be shaded independently. So do large simulations (weather, fluids, particles, where every cell obeys the same rule) and machine learning, whose training is dominated by multiplying large matrices. A GPU is built for throughput, the amount of work finished per second, and has its own high-bandwidth memory so that its cores are not starved of data.
(HL) CPU and GPU compared, and working together. Design philosophy: a CPU is latency-oriented, making one sequence of instructions finish as fast as possible, while a GPU is throughput-oriented. Cores: a CPU has a small number of powerful, complex cores with branch prediction, large caches and high clock speeds, which suit the operating system, decision-heavy code and work that must happen in order; a GPU has thousands of simpler cores that are individually slower. Memory access: a CPU relies on large caches and low-latency access to system RAM, while a discrete GPU uses its own very high-bandwidth memory and hides the long delay of each access by switching between thousands of threads. Processing power: the GPU delivers far more arithmetic per second on data-parallel work but is poor at sequential, branching code. Energy efficiency: on parallel workloads the GPU completes many more operations per watt, although its total power draw is high; for small or serial tasks the CPU is the more efficient. Working together, the CPU runs the program and divides the task, copies the data into GPU memory (data sharing, usually across the PCIe bus), launches the parallel part on the GPU and synchronises with it before collecting the results (coordinating execution). Because copying data costs time, handing work to the GPU pays off only when the parallel part is large.
Primary memory, cache hits and cache misses. The processor can execute only instructions held in primary memory, organised as a hierarchy in which each level is larger and cheaper per byte but slower. Registers are fastest and smallest. Cache is small, fast memory between the registers and RAM, in levels: L1 is the smallest and fastest, private to each core and often split into instruction and data caches; L2 is larger and a little slower, often private to each core; L3 is larger and slower again and is usually shared by all the cores. RAM is volatile main memory holding the operating system, the running programs and their data. ROM is non-volatile and read-only in normal use; it holds firmware such as the start-up program that tests the hardware and loads the operating system. When the CPU needs an item it checks L1, then L2, then L3. Finding it is a cache hit, served within a few clock cycles. Not finding it in any level is a cache miss: the item is fetched from RAM, which takes far longer, and a block of neighbouring bytes is copied into the cache at the same time. Most requests hit because of locality: loops reuse the same instructions and programs use data stored close together. A simple model gives the average access time as h × t(cache) + (1 − h) × t(RAM), where h is the hit rate.
(HL) Pipelining and parallel cores. Without pipelining, a core finishes one instruction completely before starting the next. Pipelining divides the work into stages — fetch, decode, execute and write-back, which stores the result in a register or memory — and lets every stage work on a different instruction in the same clock cycle: while instruction 1 is written back, instruction 2 executes, instruction 3 is decoded and instruction 4 is fetched. Each instruction still needs four cycles, but once the pipeline is full one instruction completes every cycle, so n instructions take 4 + (n − 1) cycles instead of 4n. The gain is lost when a branch changes the PC, because the instructions already fetched must be discarded, or when an instruction has to wait for the result of the one before it. In a multi-core processor each core has its own pipeline, registers and L1 cache, so the cores fetch, decode and execute separate instruction streams at the same moment, none waiting for another. Four cores can complete up to four times as many instructions per second, but only when the work divides into independent threads, and the cores still compete for the shared L3 cache and the memory bus.
Formulae
Number of addressable locations = 2ⁿ for an address bus of n lines
Average access time = h × t(cache) + (1 − h) × t(main memory), where h is the hit rate
Fetch: PC → MAR → address bus; read signal on the control bus; data bus → MDR → IR; PC incremented
(HL) Clock cycles for n instructions in a k-stage pipeline = k + (n − 1); without pipelining = k × n
Worked examples
q: Memory location 100 holds the instruction LOAD 250, location 250 holds the value 17, and the PC contains 100. Describe each step of the fetch–decode–execute cycle for this instruction, naming the register and the bus used at each step.
steps: Fetch 1: the address 100 is copied from the PC into the MAR, and the MAR places it on the address bus.
Fetch 2: the control unit sends a memory-read signal along the control bus.
Fetch 3: the instruction LOAD 250 travels from location 100 along the data bus into the MDR, and is then copied into the IR.
Fetch 4: the PC is incremented to 101, so it points at the next instruction.
Decode: the control unit splits the instruction in the IR into the opcode LOAD and the operand 250, which is an address.
Execute: 250 is copied into the MAR and placed on the address bus, a read signal is sent on the control bus, and the value 17 travels along the data bus into the MDR.
Execute (finish): 17 is copied from the MDR into the accumulator. The next cycle begins with the PC holding 101.
In context
Laptop and phone chips now combine performance cores, efficiency cores, a GPU and a neural processing unit on one piece of silicon, so each kind of work goes to the unit that does it with the least energy.
Machine-learning models are trained on racks of GPUs rather than CPUs because training is mostly matrix multiplication, which thousands of GPU cores carry out in parallel.
Common misconceptions
Students often say the program counter holds the current instruction. In fact it holds the address of the next instruction; the instruction itself is copied into the instruction register.
Students often think a multi-core processor makes every program run faster. In fact a program gains only if its work is split into threads that different cores can run at once; a purely sequential program uses one core.
Students often describe a GPU as simply a faster CPU. In fact each GPU core is slower and simpler; the GPU is faster only on data-parallel work where the same operation is applied to many items.
Students often say cache raises the clock speed. In fact the clock is unchanged; cache cuts the time the processor spends waiting for memory, so more cycles do useful work.
Study tips
In a fetch–decode–execute answer, name the register (PC, MAR, MDR, IR, accumulator) and the bus (address, data, control) at every step; "the CPU fetches the instruction" earns nothing.
State bus directions: the address bus is one-way from the CPU, the data bus is two-way, and the control bus carries signals such as read, write and the clock.
(HL) For a CPU versus GPU comparison, write paired sentences on each named criterion — design, cores, memory access, processing power, energy efficiency — rather than two separate descriptions.
In an average-access-time calculation, show the hit term and the miss term separately before adding them.
Reasoning
The processor and memory described here are managed by the operating system in Operating systems, control systems and translation, and the GPU returns as training hardware in Machine learning types, applications and hardware.
Summary
The CU decodes and directs, the ALU calculates, and the PC, IR, MAR, MDR and accumulator hold addresses, instructions and data.
Address bus: one-way, n lines give 2ⁿ locations. Data bus: two-way. Control bus: read, write, clock and interrupt signals.
Fetch PC → MAR, read, data bus → MDR → IR, increment PC; decode opcode and operand; execute; repeat.
Registers, L1, L2, L3, RAM: each level larger and slower; a hit is served by the cache, a miss goes to RAM.
GPUs have thousands of simple cores for data-parallel work; (HL) pipelining and extra cores raise throughput, not the speed of one instruction.
Check your understanding
Question 1: During the fetch stage, the address in the program counter is first copied to the…
1. instruction register
2. memory address register
3. accumulator
Question 2: Which bus carries the memory-read signal?
1. The address bus
2. The data bus
3. The control bus
Question 3: Which level of cache is usually shared by all the cores of a multi-core processor?
1. L3
2. L1
3. L2