Modern CPUs process multiple instructions simultaneously.
Example (Three-Stage Pipeline):
- Fetch unit gets instruction $n+2$.
- Decode unit decodes instruction $n+1$.
- Execute unit executes instruction $n$.
All three stages work simultaneously like an assembly line.
Challenges:
- Instruction must be executed even if previous instruction was a taken conditional branch.
- Increases complexity for compiler/OS writers.

More advanced than pipelining.
Multiple execution units present (e.g., integer arithmetic, floating-point, Boolean operations).
Multiple instructions fetched, decoded, and placed in a holding buffer.
Execution units grab instructions from buffer when available.
Result: Instructions often executed out of order.
Hardware ensures results match sequential execution.
Some complexity pushed onto the operating system.

What it does:
- CPU holds state of two different threads.
- Switches between threads on a nanosecond timescale.
- If one thread waits for memory, CPU switches to the other.
- Reduces thread-switching time to nanoseconds.
Not true parallelism: Only one thread runs at a time, but switching is extremely fast.
Impact on OS:
- Each thread appears as a separate CPU to the OS (e.g., 2 physical CPUs with 2 threads each = OS sees 4 CPUs).
- OS must schedule carefully to avoid running 2 threads on the same CPU while another CPU is idle.

Modern approach: 4, 8, or more complete processors/cores on one chip.
Each core is an independent CPU.
High-end processors: 50+ cores (Intel Xeon, AMD Ryzen).
Requires a multiprocessor operating system.
Cache organization choices:
- Shared L2 cache among all cores (needs complex cache controller).
- Separate L2 cache per core (keeping consistent is harder).

Layer	Access Time	Capacity	Persistent?	Cost per Bit
Registers	$<1$ nsec	$<1$ KB	No	Highest
Cache	1-8 nsec	4-8 MB	No	Very High
Main Memory	10-50 nsec	16-64 GB	N