• Unit 6: The Memory Hierarchy

    In prior units, you have studied elementary hardware components, e.g., combinational circuits and sequential circuits; functional hardware components, such as adders, arithmetic logical units, data buses; and computational components, such as processors.

    This unit will address the memory hierarchy of a computer and will identify different types of memory and how they interact with one another. This unit will look into a memory type known as cache and will discuss how caches improve computer performance. This unit will then discuss the main memory, DRAM (or the Dynamic Random Access Memory), and the associated concept of virtual memory. You will take a look at the common framework for memory hierarchy. The unit concludes with a review of the design of a cache hierarchy for an industrial microprocessor.

    Completing this unit should take you approximately 8 hours.

    • 6.1: Elements of Memory Hierarchy and Caches

      • Indian Institute of Technology, Delhi: Anshul Kumar's "Memory Hierarchy: Basic Idea" Page

        Watch this lecture. Previously, you have focused on processor design to increase performance. Now, you will turn to memory. This video introduces various methods of improving processor performance through the use of memory hierarchy. The lecture discusses memory technologies, which vary in cost and speed. We have to assume that memory is flat, but with current technology, flat memory does not meet performance demands placed on it. You will take a look at hierarchical memory and the use of cache. This video will also discuss analysis of memory hierarchies and cache performance with respect to miss rates and block size. Finally, the lecturer considers cache policy.

      • Eijkhout, Chow, and van de Geijn's "Introduction to High-Performance Scientific Computing, Chapter 1: Sequential Computer Architecture" URL

        Read section 1.2 of Chapter 1 on pages 14-23 to learn about memory hierarchies, and read section 1.4 on pages 25-29 to learn about locality and data reuse. Subsections 1.2.6 and 1.2.7 apply to the topics outlined below in subunits 6.2 and 6.3. These readings supplement the memory topics discussed in Kumar's video.

    • 6.2: Cache Architectures and Improving Cache Performance

      • Indian Institute of Technology, Delhi: Anshul Kumar's "Memory Hierarchy: Cache Organization" Page

        Watch this lecture on memory hierarchy design with caches. This lecture discusses the impact that memory operations have on overall processor performance and identifies different cache architectures that can improve overall processor performance. In the memory hierarchy, from top to bottom, there are: processor registers, cache, main memory, and secondary memory. The order goes from faster, smaller capacity, and higher cost to slower, higher capacity, and lower cost. Data moves from the main memory to cache. To do this requires a mapping from addresses in the main memory to cache addresses. How this is done affects performance. Performance analysis of cache and memory organization involves miss rate and miss penalty. Policies for reading, loading, fetching, replacing, and writing memory also affect cost and performance.

      • Eijkhout, Chow, and van de Geijn's "Introduction to High-Performance Scientific Computing, Chapter 1: Sequential Computer Architecture" URL

        Read section 1.5 of Chapter 1 on pages 30-41. This reading discusses the relationship of pipelining and cache to programming.

    • 6.3: Main Memory and Virtual Memory

      • Indian Institute of Technology, Delhi: Anshul Kumar's "Memory Hierarchy: Virtual Memory" Page

        Watch this lecture, which addresses the subject of virtual memory. This topic pertains to the relationship of main memory and secondary memory and has similarities and differences to the relationship of cache and main memory. Differences arise due to the speed of the various memories and their capacities. This lecture discusses virtual memory, mapping from virtual memory addresses to physical addresses in main and paging techniques used with the main memory. The lecture discusses the use of page tables in translating virtual addresses to physical addresses. Issues that arise with page tables include structure, location, and large size.

    • 6.4: Performance Tuning

      • Blaise Barney's "Introduction to Parallel Computing, Chapters 1-5" URL

        Read Chapters 1-5. Before reading these chapters, list the factors that you can think of that can affect performance, e.g., memory performance, cache, memory hierarchy, multi-cores, etc. and what you might suggest as ways to increase performance. After reading these chapters, what might you add, if anything, to your list?

    • Unit 6 Assessment

      • Unit 6 Assessment Quiz

        Please take this assessment to check your understanding of the materials presented in this unit.

        Notes:

        • There is no minimum required score to pass this assessment, and your score on this assessment will not factor into your overall course grade.
        • This assessment is designed to prepare you for the Final Exam that will determine your course grade. Upon submission of your assessment you will be provided with the correct answers and/or other feedback meant to help in your understanding of the topics being assessed.
        • You may attempt this assessment as many times as needed, whenever you would like.