Introduction

In my architectural history of the Pentium line, I characterized the P6 architecture as crown jewel of Intel’s x86 processor line. From its first incarnation as the Pentium Pro to its latest incarnation as the Pentium M, the P6 core has been an enormous success by both commerical and performance standards. Intel has the P6 core to thank for a large part of its market dominance over the past decade.

AMD owes a similar debt to the K7 core, the main architecture on which the company has staked its processor business since the launch of the Athlon line in 1999. Like the P6, the Athlon line ? and the K7 architecture on which it is based ? has been a commercial and performance success story. The K7 vaulted AMD to the top of the performance ladder, enabling the company to take the performance lead from Intel and to hang on to that lead for some significant stretches of time.

Given the Athlon’s continuous performance scaling, it came as no surprise that when AMD hatched the idea of bringing x86 into the 64-bit realm, they decided to build on the K7 architecture instead of creating an entirely new design from scratch. With a few signficigant changes and an untold number of tweaks, Hammer brings both the K7 architecture and the x86 instruction set into the future.

In my previous work, I’ve looked in some detail at the x86-64 instruction set, but I’ve never talked about the microarchitecture of processors that implement it. The present series of articles fills that gap by taking a close look at the architecture that underlies AMD’s Hammer-based Opteron and Athlon 64 lines. In Part I of this series, we’ll take a look at Hammer’s overall design approach. We’ll also discuss some of the problems with the x86 instruction set that Hammer’s front end is designed to solve, before examining in detail how the processor handles instruction fetching and decoding.

Overall approach

Much like x86-64 is an evolutionary ? not revolutionary ? improvement over x86, the Hammer’s architecture is an evolutionary improvement over the Athlon. AMD took the Athlon’s basic design and built on it, adding a number of tweaks and improvements that help Hammer to better the Athlon’s already solid performance.

To understand the general outlines of Hammer’s approach, it is useful think of the CPU using a producer-consumer model in which main memory is the producer of instructions and data and the CPU’s execution core is the consumer. Instructions travel from main memory into the processor’s front end, where they’re buffered and translated into a format that the CPU can understand before being sent off the CPU’s execution core to be executed. Data moves from main memory directly into the processor’s back end, where the execution units process it in accordance with the instructions that make up the incoming instruction stream.

Figure 1: Data and instruction flow

A close inspection of the Hammer’s architecture will reveal that when it came time to turn the Athlon into the Hammer, AMD took a three-fold approach that clearly stresses the following design goals:

• Production: Increase the flow of instructions and data from main memory into the processor.
• Conversion: Increase the flow of instructions through decode/translation phases of the front end and into the back end.
• Consumption: Increase the ability of the processor to consume instructions.

Before delving into the details of how these design goals are met, let’s first look at a brief overview that starts at the bottom of the above list and work our way up.

Consuming code and data: the execution core

In turning the Athlon into the Hammer, AMD left the number and arrangement of the execution units in the processor’s execution core the same: there are three integer units, three address generation units, three floating-point units, and a load-store unit. AMD reasoned that these 10 execution units represent enough potential code and data “consumption” capacity per clock cycle to carry the architecture forward, so there was no need to rearrange and/or widen the execution core to increase the basic number of execution units.

Instead, AMD focused all of their efforts into making better and more efficient use of the existing 10 execution units. Thus, a wider instruction window and more rename registers provide for better code scheduling, and more load/store buffers provide for increased data transfer capacity between the execution core and main memory. Both of these features enable Hammer’s execution units to stay busier on each clock cycle.

With regard to the processor as a whole, the Hammer’s deeper pipeline and higher clockspeeds increase consumption capacity of the processor as a whole, enabling it to grind through more code and data per nanosecond.

Download the PDF
(This feature for Premier subscribers only.)

Converting code: the front end

Increases in the execution core’s ability to efficiently schedule, and therefore to consume, more code per cycle are worthless if the front end can’t keep the execution core fed. Hammer reorganizes the fetch and decode phases of the original Athlon pipeline in order to increase the average number of instructions per cycle that the front end can deliver to the execution core.

Hammer’s front end also features an improved branch predictor, which helps to ensure that the front end stays full of instructions in the event of a change in the flow of the code stream. Keeping Hammer’s more deeply pipelined front end supplied with instructions is essential if the front end’s improved decoding hardware is going to be put to good use.

Producing code and data: caching and memory

Hammer sports three features aimed at getting code and data from main memory to the processor as quickly as possible: an on-die memory controller, improved caches, and a so-called “large workload” translation look-aside buffer (TLB). I’ll talk in more detail about these features later in the article, but for now it’s sufficient to note that they’re all aimed at complementing the Hammer’s increased consumption capacity by increasing main memory’s effective production capacity.

Front end and pipeline detail

The following diagram shows instruction flow in the Hammer processor. If you’ve seen diagrams of the Athlon, then most of this diagram should be familiar. The main differences between this diagram and a similar one of the Athlon lie in the processor’s front end. I noted previously that Hammer reorganized the Athlon’s front end for more efficient decoding, and in this section we’ll take a look at exactly how that reorganization was done.

Figure 2: The Hammer’s Architecture

Let’s take a look at a chart comparing Hammer’s pipeline to that of the Athlon. Don’t worry if you don’t recognize all of the stages, because we’ll walk through them in a moment.

You’ll notice that the Hammer’s pipeline is longer than that of the Athlon, a feature that will allow the new processor to scale to higher clockspeeds. The changes to the Hammer’s pipeline occur mainly in the front end, and in this section we’ll take a look at how the K7’s front end has been reorganized to produce the K8.

Hammer’s fetch and decode phases: general considerations

Fetch and decode are the first two stages of the classic RISC four-stage pipeline. Note that in my writing, I normally speak of fetch and decode “phases,” because most modern pipelines take more than just two stages to fetch and decode instructions. On the Hammer, the fetch phase is broken up into two individual pipeline stages, and the decode phase can be said to cover either two, four, or five stages, depending on which stages you decide to include.

The function of the fetch and decode stages, and in fact the processor’s front-end hardware in general, is to keep the execution core supplied with a constant stream of instructions. Thus the front end contains sets of buffers for collecting batches of instructions and decoding them in parallel, so that there are always instructions on hand to dispatch to the execution core.

In this respect, you can think of the processor’s front end like a large hot water heater that first fills its internal reservoir with water, then heats the water, and finally pumps the hot water into an residential building. As people in the building use the hot water, it’s important that the heater has a constant supply of incoming fresh water so that its reservoir never becomes fully drained. It’s also important that this stored water can be heated quickly enough to keep up with demand.

To unpack the analogy, the hot water heater’s internal reservoir is analogous to the front end’s internal buffers, and the act of heating the water for use by residents corresponds to the act of decoding instructions for use by the processor’s execution units. As the processor’s execution units consume program code, they drain the front end’s buffers just like our hypothetical building’s residents drain the hot water heater by washing clothes, doing dishes, etc. Thus the front end needs access to a constant supply of code, with a few stops and hiccups in that supply as possible, in order to keep the execution core supplied with decoded instructions.

Aside from the simple acts of fetching and decoding instructions, one of the most important things that takes place over the course of these two phases is branch prediction. Branches in the code stream must be identified and predicted as early in a processor’s pipeline as possible, so that the processor’s front end can be quickly filled by instructions from the new location in the code stream.

To return to the hot water analogy, imagine that, due to a water shortage, the water company in the area is in the habit of cutting off the water at certain times. When this happens, the building supervisor has to run quickly outside and connect the water heater to some other source ? like an outside well. If he takes a long time to get outside and connect the heater to the well, the heater’s reservoir will empty and the building’s residents will be without hot water for as long as it takes to refill the reservoir and heat a fresh batch of water. So the earlier he can detect a water outage and respond to it by switching water sources, the better able he is to minimize the amount drained from the hot water heater and to prevent a hot water outage inside the building.

By identifying and predicting branches in the code stream early in the pipeline’s initial fetch and decode phases, a processor can determine quickly that it’s going to have to start fetching code from a new location. Thus it can move into action and begin filling its front end with code from the new location, before the execution core drains the front end’s buffers and runs out of code to execute.

Note that I’m not going to talk in detail about Hammer’s branch prediction unit until much later in the article. But as we’ll see when we walk through the front-end functions of each of the processors described below, branch prediction considerations have an impact in how a processor’s front end is structured. Modern processors are designed with a number of front-end features whose purpose is to provide options for handling branches as early in the pipeline as possible.

Fixed-length vs. variable-length instructions: the problem with x86

The process of fetching, decoding, and branch prediction is handled differently on different processors, and in a moment we’ll look at how the Hammer and the Athlon organize the fetch and decode phases of their respective pipelines. But before you can appreciate the reasoning behind the design decisions embodied in either of these two processors, you must first understand the peculiar difficulties that attend instruction fetching and decoding on x86 processors.

One of the more headache-inducing features of x86 is that its instructions are variable-length, meaning that an x86 instruction can consist of anywhere from one to fifteen bytes. In contrast, the instructions in a RISC ISA (e.g. MIPS and PowerPC) are all the same number of bytes in length. The variable-length versus fixed-length aspect of an ISA makes a big difference in how a processor’s front end is designed, and it involves some tradeoffs that may not be immediately obvious. Let me explain.

Fetching and decoding on the PowerPC 601

In this section, we’ll take a quick look at the front end of the PowerPC 601. I’ve chosen the 601 instead of a more recent processor like the 970 as a baseline against which to compare both the Athlon and the Hammer, because the 601’s front end is simpler and is paradigmatic of the PPC approach. The 970 does some peculiar things, like ISA translation and grouping, which perhaps make it more interesting from a comparative standpoint but less useful from an explanatory standpoint.

PowerPC instructions are all four bytes in length, regardless of whether they really “need” to be or not. This simplifies instruction fetching and decoding to a great degree, because instructions can be fetched four bytes at a time and passed directly into an instruction queue for decoding and branch detection.

Figure 3: The PowerPC 601’s front end

Up to 8 instructions per cycle are be fetched directly into the 601’s eight-entry instruction queue (IQ), where they are decoded before being dispatched to the execution core.

The instruction queue is used mainly for detecting and dealing with branches. The 601’s branch unit scans the bottom four entries of the queue, identifying branch instructions and determining what type they are (conditional, unconditional, etc.). In cases where the branch unit has enough information to resolve the branch right then and there (e.g., in the case of an unconditional branch, or a conditional branch whose condition is dependent on information that is already in the condition register) then the branch instruction is simply deleted from the instruction queue and replaced with the instruction located at the branch target.

This branch-elimination technique, called branch folding, speeds performance in two ways. First, it eliminates an instruction (the branch) from the code stream, which frees up dispatch bandwidth for other instructions. Second, it eliminates the single-cycle pipeline bubble that usually occurs immediately after a branch. So branch folding allows the processor to assemble in the IQ the portion of the code stream that will be speculatively executed and to push this stream right through the decoding hardware to the execution core.

If the branch unit determines that the branch is not taken, then it allows the branch to propagate to the bottom of the queue, where the dispatch logic simply deletes it from the code stream. The act of allowing not-taken branches to “fall” out of the instruction queue is called fall-through.

Non-branch instructions and branch instructions that have not been folded sit in the instruction queue while the dispatch logic examines the four bottommost entries to see which three of them it can send off to the execution core on the next cycle.

The simple, relatively straightforward fetch/decode scheme described above is made possible at every stage by the fact that PowerPC instructions are all the same length. When a PowerPC chip fetches a group of 16 bytes, it knows exactly how many instructions it’s getting (i.e., four instructions); and quickly scanning a fetched group of instructions in order to identify branches is easy because the processor doesn’t have to figure out where one instuction ends and the next begins.

Fixed-length instructions do have one drawback, however, in the area of code density. Some instructions in the PowerPC ISA could concievably take up fewer than four bytes, so the four-byte mandatory length wastes some space in both memory and in the cache hierarchy. Variable-length instructions, on the other hand, are only as long as they need to be, so they can take up less space.

As it turns out, though, x86’s variable-length instructions add so much complexity to the fetch and decode phases of a processor that the slightly higher code density isn’t worth it. And in fact, both Athlon and Hammer actually give up some of this code density advantage in order to smooth out the fetch and decode phases a bit. As I’ll describe in more detail below, each processor’s instruction cache stores extra data that helps ease the decoding process later on.

Fetching and decoding on modern x86 processors

A RISC instruction set’s fixed-length instruction format does more than just simplify processor fetch and decode hardware; it also simplifies dynamic scheduling, making the instruction stream easier to reorder in the execution core.

In addition to being fixed-length, RISC instructions are also atomic in that each instruction tells the computer to perform one specific and carefully delimited task (e.g. multiply, divide, load, store, shift, rotate, etc.). A single x86 instruction, in contrast, can specify a whole series of tasks, e.g. a memory access followed by an arithmetic instruction, a multi-step BCD conversion, a multi-step string manipulation, etc.

This non-atomic aspect of x86 instructions renders them pretty well impossible for the execution core to reorder as-is. So in order for an x86 processor’s instruction window to be able to rearrange the instruction stream for optimal execution, x86 instructions must first be converted into an instruction format that’s uniform in size and atomic in function. This conversion process is called instruction set translation, and all modern x86 processors do some form of it.

AMD’s Athlon and Hammer translate x86 instructions into sequences of small, RISC-like instructions called MacroOps. A MacroOp consists of either one or two parts; single-part MacroOps can be arithmetic operations or memory accesses, while two-part MacroOps consist of an arithmetic operation and memory access (a load or load-store). Note that two-part MacroOps are split at execution time, with the arithmetic operation going to the appropriate ALU and the memory access going to an AGU.

In general, x86 instructions can be categorized based on how many MacroOps they break down into. Most x86 operations break down into one or two MacroOps, while a small minority break down into more than one MacroOp. The Athlon has two types of decoders: a hardware decoder for single- or two-MacroOp instructions and a microcode decoder for all the rest. We’ll talk more about these two decoders when we talk about each processor’s decode phase.

The Athlon’s fetch and decode phases

Before you can understand the front-end changes that Hammer brings to the table, you must first understand how the Athlon’s fetch and decode hardware is organized. The Hammer takes the basic organization described below and builds on it.

The Athlon’s instruction cache and predecode hardware

When code is first moved into the Athlon’s L1 instruction cache, the processor’s predecode logic examines the newly cached lump of code in order to detect individual instruction boundaries, and it marks those boundaries with a small amount of “metadata” so that the front end has less work to perform. The predecode logic also marks static branches.

This predecoding process moves some of the front-end work to an earlier portion of the pipeline, speeding the actual fetch and decode phases later. The drawback is that the extra metadata eats up valuable L1 I-cache space, and so gives back some of the code density gains that you normally get with a CISC instruction set like x86.

The Athlon’s fetch, scan, align, and decode stages

On the Athlon, the fetch/decode phase takes up a total of six pipeline stages. It’s useful to walk through each of these, because you’ll see not only how variable-length encoding creates certain problems, but how both the Athlon solves them. This will prepare you for a later discussion of how the Hammer improves on this solution.

Fetch

A group of 16 bytes, containing an unknown number of instructions, is fetched into a set of buffers in the processor’s front end. This batch of fetched bytes is generally referred to as a “fetch group.”

Note that on the PowerPC processor that we looked at, the 16-byte fetch group would contain exactly four instructions. But but because the Athlon has no way of knowing how many instructions make up each 16-byte fetch group, the fetch group must be placed in a set of buffers so that the number and types of instructions can be identified. This identification step (i.e., the scan stage), which we’ll look at more closely below, represents an additional phase of front-end instruction processing that’s not necessary on RISC architectures.

The Athlon’s pipeline, continued

Scan

The entire fetch group is scanned and the boundaries of each instruction in the group, boundaries that were marked in the predecode phase described above, are identified. Instructions are then sorted into one of two types: those that will go to the direct decoder (a.k.a., Fastpath, a.k.a., Direct Path) and those that will go to the microcode ROM (a.k.a., Vector Path).

Take a look at the diagram below, which shows x86 instructions entering the Scan unit and being sorted by instruction type. You’ll want to keep this diagram handy, so that you can refer to it throughout the rest of our discussion of both the Athlon’s and Hammer’s decoding hardware.

Figure 4: The Athlon’s decoding hardware

After instructions leave the Scan unit, what happens next depends on whether the instructions are headed for the microcode engine or the direct decoding hardware. As I noted above, simpler instructions ? i.e., instructions that translate into one or two MacroOps ? go to the faster hardware decoder. The more complex instructions go to the microcode decoder.

The microcode decoder works by means of a table look-up scheme, where the uop sequences that translate various x86 instructions are stored in a small ROM. The microcoder translates an x86 instruction by looking up the address of that instruction’s uop sequence in ROM, retrieving the sequence, and passing the uops three at a time on to the next stage.

Both decoders can produce up to three MacroOps per cycle, but the hardware decoder decodes x86 instructions at a rate of up to three per cycle, while the microcode decoder can decode only one x86 instruction per cycle. This means that the microcode decoder can quickly become a bottleneck if the instruction stream is made up of too many large instructions. Luckily, the majority of instructions can go through the higher-throughput hardware decoder.

Align1 and Align2

In these two stages, the first part of the actual instruction translation takes place. Fastpath instructions are positioned in a quad-word queue, precisely aligned, and then fed, three at a time, directly to the Athlon’s Fastpath decoding hardware.

Instructions that have taken the microcode track are fed into the microcode ROM and translated in these two stages.

Decode1 and Decode2 (or EDEC and IDEC)

In these two stages, MacroOps are generated by both the hardware and the microcode decoders. Some final decoding is then done on these ops before they’re fed at a rate of up to 3 per cycle into the Athlon’s instruction control unit (ICU). The ICU is essentially a reorder buffer (ROB), so if you’ve read my previous work you’ll be familiar with what a ROB does.

At the conclusion of the decode2 stage, MacroOps are sent to the execution core’s schedulers, where they’re scheduled for execution.

Conclusions

From the above description, you can see that the scan and align stages of the original Athlon’s pipeline sit between the fetch and decode stages in order to “clean up” and “organize” the variable-length x86 instructions into something that the decoder can handle efficiently. We’ll see below that Hammer handles this “clean-up” type work in much the same way, but with a few important changes that increase decode bandwidth and reduce decode-related bottlenecks.

Hammer’s front end: the fetch and decode phases

Now that we understand the peculiarities of instruction fetching and decoding on x86 processors, we’re better equipped to understand how Hammer approaches these to critical front-end functions.

The Hammer architecture improves on the Athlon mainly by sending more instructions through the Fastpath decoder. Hammer also makes a few other changes, which I’ll detail below.

Hammer’s instruction cache and predecode hardware

Like the Athlon, Hammer predecodes instructions that are fetched into the I-cache. Instruction boundaries are identified and marked, and branches are identified and marked for type. Marking the branches in the I-cache helps the front end quickly pick them out of the fetch group when they’re fetched in. That way, the branch predictor can start working on the branches as early as possible, so that if the front end needs to be redirected to a different point in the instruction stream this can happen quickly. Remember the point I made above with the hot water heater analogy ? the processor’s front end must be redirected as quickly as possible after a branch is detected and predicted, so that its buffers aren’t drained by the execution core.

Hammer’s fetch, scan, align, and decode stages

Fetch1 and Fetch2

In these two stages, Hammer fetches instructions sixteen bytes (equivalent to about five x86 instructions) at a time from the L1 instruction cache into a set of buffers in its front end.

Hammer’s fetch phase has been broken up into two stages (compare the Athlon’s single-cycle fetch stage). This two-cycle fetch is a feature of many processors, because it helps to decouple the clockspeed from L1 cache access latency.

Pick (or “Scan”)

Hammer moves these sixteen bytes into a 32-byte buffer, appending the newly-fetched 16-byte group onto the previous 16-byte fetch group. The processor then scans this 32-byte batch of instructions in order to pick out instruction boundaries, after which it aligns the instructions. Finally, instructions are sorted into one of two types ? those that can be decoded by the direct decoder, and those that must be decoded by the microcode engine ? and then aligned to be sent to the decode stage.

Like the Athlon, Hammer can do all of this scanning and aligning in one cycle because it “cheats” by using the predecoded instruction boundary data that was added when the instructions were fetched into the L1.

Decode1 and Decode2: Fastpath and Microcode decoding

Hammer’s decoding hardware is quite similar to that of the Athlon. Both processors have two types of decode hardware: the so-called Fastpath decoder, which is a direct decoder, and the microcode engine. Both of these decoders translate x86 instructions into an internal, RISC-like format that’s easier for the processor’s execution core to manage and schedule.

As was the case with the Athlon, Hammer’s Fastpath decoder handles the x86 instructions that translate into at most two upos. In other words, this decoder handles the simplest and shortest x86 instructions. The Fastpath decoder can translate up to three x86 instructions at a time into up to three uops per cycle.

The microcode decoder handles instructions that translate into more than two uops. This decoder can work on only one x86 instruction at a time, though it too can produce up to three uops per cycle.

Needless to say, because it can work on only one x86 instruction at a time, the microcode decoder has much less decode bandwidth (i.e., a lower decode rate) than the Fastpath decoder. For this reason, the Hammer has beefed up the Fastpath decoder to be able to translate a wider range of instructions. So some of the instructions that on the Athlon were translated using the microcode engine are on the Hammer translated directly. This is especially true for floating-point and SSE instructions, of which a significant number have been moved to Fastpath on the Hammer.

Moving instructions from microcode to Fastpath has the effect of increasing the Hammer’s overall average decode bandwidth, which is essential because of the other design changes mentioned above. Specifically, Hammer aims improve performance by having lower memory latencies and higher execution unit utilization than the Athlon. The decoder sits between memory and the execution units, so if there are more instructions coming into the chip from memory on average then an inadequate decoder would be a bottleneck that would prevent the increased instruction flow from reaching the execution core.

In sum, Hammer’s decode1 and decode2 stages are mostly analogous to the Athlon’s align1 and align2 stages. In these two stages, x86 instructions are aligned and then translated into MacroOps by either the Fastpath hardware or the microcode ROM.

The Hammer’s pipeline, continued

Pack, Pack/Decode, and Dispatch

The pack and pack/decode stages are roughly analogous to the decode1 and decode2 stages on the Athlon, but with a twist. Once Hammer’s decoders have produced a set of Macroops, these ops are packed three-at-a-time into what I’ll call a “dispatch group” consisting of three fixed dispatch slots. (Note that AMD calls these “issue slots,” but because I consistently use the terms “dispatch” and “issue” in a specific manner in my articles I’m going to call them “dispatch slots” to maintain consistency.)

Once the instructions have been grouped, some final decoding is done on them before they’re sent a group at a time to the Hammer’s reorder buffer (called the instruction control unit in Hammer lingo).

I wish I knew more about how this grouping affected Hammer’s internals, but from what I can tell it’s not nearly as significant as the 970’s grouping mechanism. In other words, Hammer’s dispatch slots may feed instructions into fixed lanes in the ROB, but instructions don’t appear to be tracked as groups in the ROB. Furthermore, instructions can switch lanes to alleviate dependencies, a feature that gives the Hammer a little more flexibility in code scheduling versus the Athlon.

Finally, MacroOps are dispatched to Hammer’s execution core, where they’re scheduled for execution.

Conclusions on Hammer’s front end

Hammer’s front end has been organized so that it can process incoming instructions at a higher average rate than that of the Athlon. By moving part of the work of predecoding, instruction alignment, and branch detection higher in the pipeline, Hammer simplifies the later decoding stages and allows the actual decoding hardware to work faster. This redistribution of labor, in conjunction with a stronger decoder, allows the Hammer’s overall front-end instruction throughput to exceed that of the Athlon. Thus the front end doesn’t act as a bottleneck between Hammer’s wide back end and its improved cachine and memory subsystem.

Branch prediction on the Hammer

The Hammer’s improved branch prediction unit represents a major expenditure of transistors in the service of increased execution unit utilization. Recall my hot water heater analogy above, which I used to show that the earlier in the pipeline a branch can be dealt with, the better able the processor will be to keep the execution core fed with code and data.

Like the Athlon, Hammer has two main branch prediction structures: a local history table and a global history table. The local history table keeps track of the execution history of each individual branch in order to predict whether the branches will be taken or not taken. The global history table keeps track of the history of a large number of branches, and predicts which way individual branches will go by correlating them to the execution histories of the other branches in the table. A branch selector table chooses between the two tables on a branch-by-branch basis, based on which of the two prediction schemes has produced the most accurate results for each branch. Thus both of these tables work in conjunction to provide the kind of extremely accurate branch prediction that’s essential on superpipelined machines like the Hammer.

Hammer doubles the number of branch selectors versus the Athlon, and it quadruples the size of the global history table to over 16K entries. Both of these large tables help give the Hammer a 5-10% improvement in branch prediction accuracy over the Athlon.

The other essential structure in both the Hammer’s and Athlon’s branch prediction units is the branch target buffer (BTB, also called the “branch target array” in most AMD literature). When the branch predictor decides that a particular branch should be “taken,” it needs to know the address to which it should jump (i.e., the branch target). That address can be specified in the branch instruction itself, or, as is more often the case, it can be the result of an address calculation. In cases where the the branch target is the result of a calculation, the Hammer stores the calculated target address in the BTB. That way, when later on in the course of program execution that particular branch is encountered again and is predicted as “taken” again, Hammer doesn’t have to recalculate the branch target; it just pulls the correct target address from that branch’s entry in the BTB.

Hammer’s BTB holds information on 2,048 branches, which is the same size as the Athlon’s BTB. The BTB is backed up by a branch target address calculator (BTAC). The BTAC is a new feature with the Hammer, and it shortens the mispredict penalty when the BTB is holding the wrong branch target for a branch. Under certain conditions, it’s possible for the BTB to spit out the wrong target address for a branch. The BTAC calculates the branch target and compares it to the branch target produced by the BTB; if the two don’t match, then branch predictor immediately calls a mispredict and thereby saves on mispredict penalty cycles.

Lowering memory latency

One of the performance-limiting factors with which Hammer contends is the increase in memory latencies relative to CPU speeds. In short, as CPU clock speeds increase at a much faster rate than memory access latencies decrease, main memory begins to look slower and slower from the point of view of the CPU. This is because the CPU is running faster, while main memory is not.

The result of all of this is that, as main memory gets relatively slower, it becomes harder to keep the processor fed with code and data. Hammer sports a few features that are aimed at alleviating this problem.

Hammer’s on-die memory controller

The first and most prominent of Hammer’s new latency-reducing features is its on-die DDR memory controller. AMD’s Kevin McGrath, in his Stanford presentation, attributed up to 20% of Hammer’s performance improvment over the Athlon to this controller. The on-die controller lowers memory latency by literally shortening the distance that memory traffic has to travel when moving between the processor and RAM. With a regular CPU + northbridge combination, all memory traffic is routed through the northbridge, so that the route that such traffic takes is CPU ??? northbridge ??? RAM. The Hammer takes the northbridge out of the path altogether, so that memory traffic travels directly between the CPU and RAM. This means that requests for data reach RAM more quickly and the requested data is returned from RAM faster.

Because the memory controller is integrated onto the CPU die, it also benefits from the processor’s frequency scaling. As Hammer clocks faster, so will its memory controller. This will help reduce a Hammer system’s absolute memory latency even further, and memory’s relative latency as seen from the CPU will grow at a slower rate as clockspeeds scale.

Another benefit of using an on-die memory controller is that as you add more CPUs to a Hammer system the overall aggregate memory bandwidth available to the system grows. Each of the CPUs in a multiprocessor Hammer system have a dedicated bank of DRAM attached to them, so they’re not all competing for access to the same shared DRAM bank. They’re also not all sending their memory traffic through the same northbridge and over the same memory bus.

The main drawback to this scheme is that at some point one CPU is going to need data that’s in a DRAM bank attached to another CPU. This is where Hammer’s on-die Coherent HyperTransport (HT) links come in. In such instances, the CPU whose DRAM bank is hosting the needed data can route it via the HT link to the requesting CPU, and in that sense one of the CPUs acts a sort of “northbridge” for the other. And in the case where the requesting CPU is not linked directly to the hosting CPU, there’s an extra hop involved as the request and response passes through the intervening CPU.

The other drawback to an on-die memory controller is that you can’t upgrade it when a newer, faster memory technology hits the market. So if DDR-II or some other next-generation memory technology were to become prominent, then you’d have to upgrade your Hammer CPU along with your motherboard in order to take advantage of it. And along similar lines, Hammer’s memory controller supports up to eight registered DIMMs at three speed grades (PC1600, PC2100, and PC2700). Again, if you wanted to move beyond this number of DIMMs or these three speed grades, then you’ll need a new CPU.

Hammer’s caches

Both Athlon and Hammer have a 128KB, two-way set associative, on-die L1 cache. The caches differ mainly in the amount of predecoding and branch prediction work that’s done in the Hammer’s I-cache (see above).

The Hammer’s maximum L2 size goes up to 1MB, which is double the Athlon’s maximum L2 size of 512K. And like more recent Athlon’s the Hammer’s L2 is a “victim” cache. If you read my article on caching, then you know that in most cache hierarchies each cache subsets the cache below it. Or, in other words, the L2 cache has all the same data as the L1, plus some extra, and the L3 cache has all the same data as the L2 (and, by extension, the L1) plus some extra.

The Hammer’s L2, in contrast, does not share any data in common with the L1. Rather, cache blocks that are evicted from the L1 go into the L2, which is why it’s called “victim cache.” This strategy effectively expands the size of the L1, so that the L1 and L2 act in concert as a very large L1 cache with a smaller, faster region (the actual L1) and a larger, slower region (the L2).

Hammer’s L2 cache also saves the predecode information that was added to the instructions when they originally entered the L1. This way, that information doesn’t have to be recreated when blocks move from the L2 back into the L1.

The TLB

One of the ways that Hammer lowers memory latencies is through the use of what AMD calls “large workload TLBs.” In order to properly understand how a translation look-aside buffer works, you first have to understand the basics of virtual memory. I’ve put together a brief overview of how virtual memory works, which I recommend to you before going any further.

When a processor performs a memory access (e.g., a load or a store), it must first translate the logical address specified in the memory access instruction into a physical address that tells the processor where in RAM the needed data is actually located. Because the process of address translation involves accessing a special data structure (the page table) that’s stored in main memory, a single memory access theoretically involves two memory accesses: one to fetch proper page table entry (PTE), and a second to actually read or write the data.

In order to prevent each memory access from requiring a page table access, a translation look-aside buffer (TLB) speeds the process of address translation by caching the results of previous address translations. So when the processor has to perform an address translation, it first checks the TLB to see if that translation has recently been performed. If it has, then it can get the physical address that it needs directly from the TLB, instead of having to fetch the proper page table entry from memory and the perform an address calculation. Thus the TLB cuts way down on average memory access latencies by reducing the number of memory accesses that require an extra memory access in order to be completed.

TLB continued, and conclusions

Hammer improves on the Athlon’s TLB by modifying it to work better with multiple concurrently running processes, and by doubling the size of the L1’s and L2’s TLBs so that they hold more entries. The latency of both TLBs has also been lowered so that they can be accessed more quickly.

So basically, the speed and hit rate of the Hammer’s TLBs have been improved, so that 64-bit applications that keep very large files in memory and/or use a high number of concurrent threads of execution won’t suffer as badly from TLB misses. This keeps average memory latencies down, and fits with Hammer’s overall design strategy of feeding more code and data to its execution core.

0 Comments