Application of The Concept
The memory hierarchy in most computers is:
- Processor registers – the fastest possible access (usually 1 CPU cycle), only hundreds of bytes in size
- Level 1 (L1) cache – often accessed in just a few cycles, usually tens of kilobytes
- Level 2 (L2) cache – higher latency than L1 by 2× to 10×, usually has 512 KiB or more
- Level 3 (L3) cache – higher latency than L2, usually has 2048 KiB or more
- Main memory – may take hundreds of cycles, but can be multiple gigabytes. Access times may not be uniform, in the case of a NUMA machine.
- Disk storage – millions of cycles latency if not cached, but can be multiple terabytes
- Tertiary storage – several seconds latency, can be huge
Note that the hobbyist who reads "L1 cache" in the computer specifications sheet is reading about the 'internal' memory hierarchy .
Most modern CPUs are so fast that for most program workloads, the bottleneck is the locality of reference of memory accesses and the efficiency of the caching and memory transfer between different levels of the hierarchy. As a result, the CPU spends much of its time idling, waiting for memory I/O to complete. This is sometimes called the space cost, as a larger memory object is more likely to overflow a small/fast level and require use of a larger/slower level.
Modern programming languages mainly assume two levels of memory, main memory and disk storage, though in assembly language and inline assemblers in languages such as C, registers can be directly accessed. Taking optimal advantage of the memory hierarchy requires the cooperation of programmers, hardware, and compilers (as well as underlying support from the operating system):
- Programmers are responsible for moving data between disk and memory through file I/O.
- Hardware is responsible for moving data between memory and caches.
- Optimizing compilers are responsible for generating code that, when executed, will cause the hardware to use caches and registers efficiently.
Many programmers assume one level of memory. This works fine until the application hits a performance wall. Then the memory hierarchy will be assessed during code refactoring.
Read more about this topic: Memory Hierarchy
Famous quotes containing the words application of the, application of, application and/or concept:
“Courage is resistance to fear, mastery of fear—not absence of fear. Except a creature be part coward it is not a compliment to say it is brave; it is merely a loose application of the word. Consider the flea!—incomparably the bravest of all the creatures of God, if ignorance of fear were courage.”
—Mark Twain [Samuel Langhorne Clemens] (1835–1910)
“Science is intimately integrated with the whole social structure and cultural tradition. They mutually support one other—only in certain types of society can science flourish, and conversely without a continuous and healthy development and application of science such a society cannot function properly.”
—Talcott Parsons (1902–1979)
“We will not be imposed upon by this vast application of forces. We believe that most things will have to be accomplished still by the application called Industry. We are rather pleased, after all, to consider the small private, but both constant and accumulated, force which stands behind every spade in the field. This it is that makes the valleys shine, and the deserts really bloom.”
—Henry David Thoreau (1817–1862)
“The new concept of the child as equal and the new integration of children into adult life has helped bring about a gradual but certain erosion of these boundaries that once separated the world of children from the word of adults, boundaries that allowed adults to treat children differently than they treated other adults because they understood that children are different.”
—Marie Winn (20th century)