The MOS 6502 and the Best Layout Guy in the World

Posted on Monday, January 3, 2011.

The MOS 6502 was ubiquitous in its day. The 6502 and its slight variants were at the heart of the Apple II, the Atari 2600, the BBC Micro, the Commodore 64, and the Nintendo Entertainment System, among others. It's amazing to think that all five—each a very influential system in its own right—were all built around the same chip.

The 6502 was the brainchild of Chuck Peddle, at the time an engineer with Motorola. Peddle was one of the engineers who worked on the Motorola 6800, and one of his jobs was to, well, peddle the 6800 to customers. The customers loved everything about it except its $300 price tag. Peddle tried to convince management at Motorola to create a lower-cost microprocessor, but Motorola didn't want to have such a chip cut into their not insubstantial profits from the 6800, and they told him in no uncertain terms that they wouldn't build such a chip. In response, Peddle and a handful of other 6800 engineers left Motorola and built one themselves. It was the MOS Technology 6502 and sold for $25. Even though both the 6800 and 6502 had a clock rate of 1 MHz, the 6502 had a minimal instruction pipeline that overlapped the fetch of the next instruction with the execution of the current one when possible, giving it a significant performance boost. And of course it sold for ten times less. So it ended up everywhere.

The story of the 6502 makes up the first chapter of Brian Bagnall's On the Edge: the Spectacular Rise and Fall of Commodore. My favorite part of the description of the development of the 6502 is the actual chip layout. These days, you can't design and lay out a computer chip without a computer. An Intel Core 2 chip has hundreds of millions of transistors. The 6502 had 3,510, and an engineer—a person, not a computer—had to draw each one by hand to lay out the chip. Mainly it was a single engineer, Bill Mensch.

But it gets better. Once the layout was completed and double-checked—a process that meant months using a ruler!—it still had to be converted into a Rubylith photomask that would etch the right patterns onto the silicon. The photomask for the 6502 was the size of a large table—large enough that the engineers crawled around on top of it to perform the job of cutting the layout out of the mask, all the while being careful to wear clean socks with no holes, so that stray toenails didn't insert traces in the mask where they didn't belong.

The most amazing part about the whole process is that they got the 6502 right in one try. Quoting On the Edge:

Bil Herd summarizes the situation. “No chip worked the first time,” he states emphatically. “No chip. It took seven or nine revs [revisions], or if someone was real good they would get it in five or six.”

Normally, a large number of flaws originate from the layout design. After all, there are six layers (and six masks) that have to align with each other perfectly. Imagine designing a town with every conceivable layer of infrastructure placed one on top of another. Plumbing is the lowest layer, followed by the subway system, underground walkways, buildings, overhead walkways, and finally telephone wires. These different layers have to connect with each other perfectly; otherwise, the town will not function. The massive complexity of such a system makes it likely that human errors will creep into the design.

After fabricating a run of chips and probing them, the layout engineers usually have to make changes to their original design and the process repeats from the Rubylith down. “Each run is a couple of hundred thousand [dollars],” says Herd.

Implausibly, the engineers detected no errors in [Bill] Mensch's layout. “He built seven different chips without ever having an error,” says Peddle with disbelief in his voice. “Almost all done by hand. When I tell people that, they don't believe me, but it's true. This guy is a unique person. He is the best layout guy in the world.”

The first chapter of On the Edge is posted on Bagnall's web site in HTML or PDF. The chapter says that Peddle “created a concept called pipelining,” which could be interpreted as saying that the 6502 was the first pipelined processor and that Peddle invented it. Does anyone know? [Note: See update at end of post.]

Fast forward thirty years. Computers are now old enough that there can be significant interest in maintaining the history of these old, venerable designs. The actual paper designs of the 6502 are long gone.

A team of three people—Greg James, Barry Silverman, and Brian Silverman—accumulated a bunch of 6502 chips, applied sulfuric acid to them to strip the casing and expose the actual chips, used a high-resolution photomicroscope to scan the chips, applied computer graphics techniques to build a vector representation of the chip, and finally derived from the vector form what amounts to the circuit diagram of the chip: a list of all 3,510 transistors with inputs, outputs, and what they're connected to. Combining that with a fairly generic (and, as these things go, trivial) “transistor circuit” simulator written in JavaScript and some HTML5 goodness, they created an animated 6502 web page that lets you watch the voltages race around the chip as it executes. For more, see their web site visual6502.org.

Oh, and it actually works. They applied the same technique to build the transistor map for an Atari 10444D TIA chip, which connected the 6502 to the television in the original Atari 2600, and then they simulated both chips together and were able to run actual Atari 2600 games. So the transistor map is probably (very close to) correct. More impressively, they got to that point without debugging. Their SIGGRAPH 2010 abstract explains that there were only 8 errors in the map of 20,000 components, and all the errors were spotted during the vectorization process. History repeats itself.

Michael Steil took the circuit information and started looking closely at how the chip did what it did. At last week's Chaos Communication Congress 27C3 conference, he gave a 50-minute talk that introduces the 6502 architecture and then uses the circuit diagram to explain various details and undocumented features of the chip. The original announcement is at Steil's blog. There is a version of the talk on YouTube, and in the blog comments you'll find links to higher-resolution copies. It's well worth watching, in any form, and it's fun to see how much Peddle, Mensch, and the rest of the 6502 team packed into that tiny number of transistors.

Update, January 2015.

The links above to Chapter 1 of On the Edge on the publisher site no longer work, but there is a copy of the HTML version at the Internet Archive.

As noted in the comments below, the Manchester Atlas, the CDC 6600, and the IBM 7030 (Stretch) all used pipelining at least a decade before the 6502 was created. Also, what the 6502 did was not really pipelining in the modern sense. It overlapped instruction execution with the fetch of the next instruction in some cases, as did the Intel 8080 (which beat the 6502 to market) and other microprocessors of its era. Bagnall's claim that Peddle created pipelining for the 6502 seems to be completely without justification.

Also, in the linked chapter, Bagnall explains that “As Motorola publicly unveiled the 6800, Chuck Peddle and seven coworkers from the engineering and marketing department left Motorola to pursue their own vision. The team included Will Mathis, Bill Mensch, Rod Orgill, Ray Hirt, Harry Bawcum [sic], Mike James [sic], Terry Holt, and Chuck Peddle.” Later, Bagnall writes, “The layout team consisted of two main engineers: Bill Mensch and Rod Orgill. A third engineer, Harry Bawcum [sic], aided the layout artists. It was their task to turn an abstract block diagram into a large-scale representation of the surface of the microprocessor. Orgill was responsible for the 6501 chip, Mensch the 6502.” There is little mention of others again. Mensch gets all the credit. In January 2015, I received the following note via email from Harry Bawcom:

Just an FYI.

Bill Mensch did not layout the 6502, I did, with the help of two others.

In truth we all worked as a team but Bill didn't draw a single line on the first version of the 6502. My initials were on the die.

I wasn't there for the later versions.

Interesting. There were two other circuit designers involved, one senior to Bill. They deserve credit, too.

An EE Times article from August 1975 shows a picture of the 6502 team, along with a caption listing Harry Bawcom as a “layout designer” and Bill Mensch as a “design engineer,” which seems to corroborate Bawcom's note. (Note that the photo printed in the article has been mirror-image reversed.) I was right to be skeptical of the book's claim that Chuck Peddle and the 6502 introduced pipelining. I was probably wrong not to be skeptical of the rest of the story I quoted. It was too good to be true. The Visual 6502 story remains a nice ending, however.

P. S. If you liked this, you might also like Computing Machines. Also, Barry Silverman and Brian Silverman (mentioned above) were two of the people behind the PDP-1 simulator written in Java that runs Spacewar!