What Happened to the Capacitors in 2002?
Starting in the early 2000s, reports emerged of abnormal numbers of PC motherboards with leaking or even popping capacitors. A victim said in a newspaper interview that he turned on his computer one morning and suddenly heard a loud POP, like a distant firework. Other reports mention a weird, fishy smell in the air. The OS goes black and the computer fails to reboot. Most inexperienced people have no idea what to do except to bring it to a repair shop. Open it up, and inside you might find that a little cylinder on the board had swollen or even burst.
This happened a lot. Why? In this video, we look at the infamous Capacitor Plague. We will never know what exactly happened, but let's try. ## Beginnings So let's start with the basics. A capacitor. What is it? A capacitor stores electricity kind of like a battery. But unlike a battery, the capacitor discharges its electrical energy in a very short period of time. Sometimes in only a few microseconds. The basic structure of a capacitor has two electrically conductive plates called electrodes - the anode and cathode - separated by a material called the dielectric.
Eagle-eyed viewers of the channel - I hope there are a few of you out there - might recognize the term "capacitor" from videos on DRAM. And indeed, it is conceptually similar, subject to the same laws of physics. ## E-caps Electrolytic capacitors are often called E-caps. And I am going to call it that too because I hate saying the word "electrolytic". The name refers to the electrolyte, which serves as the capacitor’s cathode or negative electrode. This electrolyte can be a solid, gel, or liquid. Simply put, an E-cap is made up of two strips of aluminium foil with another very thin separator layer of porous paper or tissue in between. The paper
and the gap between the foil strips are saturated with this electrolyte. The positive electrode or anode foil is coated with another layer of aluminium oxide. This oxide is the dielectric. The foils are attached to tabs which connect to the circuit board. The whole
thing is rolled up in a sealed protective housing, usually also made from aluminium. The seal is to keep the electrolyte from drying out. There may be a vent at the top so to release built-up gas and preempt a messy explosion. While there are many types of capacitors and even electrolytic capacitors - like those made with Tantalum - Aluminium E-caps are commonly used because they offer pretty good capacitance - as in they hold a lot of charge - for their size and cost. A capacitor's ability to hold charge is correlated to the surface area of its electrodes, and inversely correlated to the distance between said electrodes.
So with an aluminium E-cap, the thin separator paper lets the dielectric get super thin, so that the electrodes can be as close together as possible. Manufacturers will also make them work even better by etching the aluminium foil to increase the electrodes' surface area. Nifty. ## Uses A PC motherboard might have about 60 E-caps on it. Like ... why? What are they used for? A big use case is filtering. A power supply module receives AC power from the plug and
turns it into steady DC power. AC power going into the power supply hits a thing called a rectifier, which converts it into DC power. The DC power that comes out is no longer alternating, but it is also not steady. We need to make it steady, so we use a capacitor to help smooth out the ups and downs. When voltage
peaks, the capacitors charge up. When voltage dips, the capacitor discharges to compensate. Capacitors are also used to help smooth out power supply fluctuations going to chips like the microprocessor - a process called "decoupling". The capacitor will charge up to absorb voltage spikes and discharge to boost voltage dips.
I mentioned before that these decoupling capacitors are like drainage ponds for during floods or dry periods. And by the way, it is not just PC motherboards that have these E-caps, but also boards for camcorders, VCRs, television, stereos, and LCD screens. The Aluminium e-cap industry is a multi-billion dollar industry. In 2002 it was worth about $3 to 3.5 billion. Today, various market analysts say it is worth about $5 to 6 billion.
E-cap suppliers are mostly Japanese, Taiwanese, or Chinese and they produce literally billions of units each year. In 2002, 22.5 billion E-caps were made in Taiwan, 30% of the market. ## Failure Perhaps because they are electrochemical, electrolytic capacitors age and corrode. And they have shorter natural life spans in general than other types of capacitors - about 10-20 years. The best way to see how far gone the capacitor is to look at a metric known as Equivalent Series Resistance. As the capacitor ages, that starts to rise. The capacitance will also slowly decline before dropping off at the end. This is a parametric failure, because it involves parameters.
Several factors can accelerate this aging process, the most significant being humidity and heat. Heat can cause the electrolyte in the E-caps to dry up, so it is generally recommended that we keep them cool. Sometimes though capacitors can fail in an extraordinary way: A catastrophic failure. A bulged capacitor has no capacitance, causing a short circuit on the board. An exploded capacitor
might even cause the thing to catch on fire. And that is what started happening in the early 2000s. ## Hints of Defects The reports of bulged or even exploded capacitors largely started in late 2002. But it seems like they began a bit earlier than that, mid-year. An electronics repairman in Utah named Gary Headlee was quoted saying that he had replaced some 40,000 leaky capacitors on almost 1,500 motherboards in a year starting in the summer of 2002. He said, "Over 10 a day sometimes and they just keep on coming". Headlee also claimed to see a significant increase in faulty E-caps in Mitsubishi televisions, JVC VCRs, and Sony camcorders. One of the journalists covering these failures early on was Carey Holzman, who in October 2002 started posted online about leaking capacitors in various Taiwan-made motherboards like ABIT, Asus and so on.
Holzman has a popular YouTube channel. And in one video he covers his experience in finding and bringing this stuff to light. I think it's worth a watch. ## The Rubycon Story The story behind these failures was then reported by Dennis Zogbi in the September/October 2002 issue of the "Passive Component Industry" magazine. He wrote that a materials scientist working for the Japanese electronics company Rubycon Corporation left to go work for the Taiwanese company Luminous Town Electric's factory in China.
The scientist replicates the formula for Rubycon's P-50 water-based electrolyte. Luminous Town then uses it for their aluminium E-caps, and it seems to work fine for them. But then some of that scientist's staffers defected from Luminous Town with the electrolyte formula. They then either sell it to Taiwanese E-cap companies directly or to an electrolyte provider who then resells it. The problem was that the defecting staffers did not have the whole formula. There were
missing additives that caused the E-caps to suffer hydrogen gas build-up until they rupture or break. In the next issue of the magazine, a rather amusing clarification reiterates that Luminous Town did not make any of the bad electrolyte circulating through the industry. Luminous Town did not deny the story of stealing electrolyte formulation from Rubycon though. Which is weird. Zogbi's story quotes anywhere from 5 to 11 Taiwanese producers being affected. In a 2003 interview with the Toronto Star he said: "I think anybody who uses contract manufacturers that outsource to Taiwan was affected by the problem". Zogbi doesn't note in the original magazine article when this whole thing occurred. But in a 2003 interview with the Santa Fe New Mexican newspaper,
he says that it happened some time in the middle of 2002. I’ve seen mentions of 2001 and even 1999 in other publications, but 2002 looks to me most accurate. The IEEE Spectrum magazine did a story on it at the time, and noted that the broken E-caps bore generally unknown brands like "Tayeh", "Choyo", or "Chhsi", or are unmarked. A few showed the name of Jackcon Capacitor Electronics. Jackcon's managing director
replied that they had not made capacitors for motherboards in the past 2 years. Which implies that someone used the wrong E-cap for the job. In a later newspaper interview, Zogbi says that he was tipped off on this story from a Japanese contact and confirmed it with other anonymous industry sources. IEEE Spectrum wrote that a well-placed Taiwanese source "largely" confirmed the story too. I reached out to Zogbi via email for a chat but didn’t get a response.
## Fallout Anyway Zogbi's report noted at the end several computer-makers commenting that they had indeed seen some issues with Taiwanese E-cap failures. IBM - which back then still made PCs - was willing to go on the record. IBM spokesperson Ray Gorman said that a small number of desktop PCs were returned for repair after short-circuits induced by a capacitor failure. He added the problem was "minuscule", affecting less than 1% of computers. Maybe true, but less than 1% can still add up to a lot. Gartner said that the world PC industry shipped 148.1 million PCs in 2002. Less
than 1% of that can still cause major disruption to workshops and the like. A second manufacturer to admit to an E-cap problem was the Taiwanese motherboard maker ABIT. They told the press that they were switching from Taiwanese suppliers to Japanese suppliers as a response. As a thank you for their honesty, ABIT got sued for selling defective capacitors as early as 1999. And so far as I can tell, that is why most people peg the start of the
capacitor plague to 1999. Though I have not seen real reports of such being that early. Anyway, ABIT settled and repaired the bad boards. The cost of doing so may have contributed to their later exit from the motherboard business in 2006 and practical liquidation thereafter. Interestingly, the IEEE Spectrum story even named the electrolyte company accused of buying the flawed and stolen formula: Lien Yan in Taichung.
Lien Yan denied it. They said that the accusations heavily damaged their business and pointed out that the suppliers of several bulged capacitors never bought their electrolyte. They called the Rubycon story FUD spread by the Japanese to win back business from Taiwan. One of Taiwan's biggest capacitor suppliers Luxon issued a statement anyway blaming Lien Yan or "Lenyan" for all the drama, emphasizing that they never bought any electrolyte from Lien Yan. They added:
> Luxon definitely understands that the electrolyte is one of the most important materials for aluminum electrolytic capacitors. In order to ensure our reliability and innovative technology Luxon always develops the electrolyte by ourselves. I did a few Google searches and couldn't find out what Lien Yan is up to today. But without the Chinese name, tracking them down can be hard. Lien Yan can mean anything. I bet they changed their name or went out of business. Probably the latter.
## The Electrolyte Luxon’s remarks reinforce the point that the electrolyte is not just a simple mix that you pump into the capacitor at the last second. Rather, the electrolyte is a critical trade secret that sits at the core of the whole product. It determines the capacitor's operating conditions in temperature, voltage, safety and more. These are complex formulations with many chemical components. The Rubycon story indicated that something had been missing from the formulation. And in 2004, the Center for Advanced Life Cycle Engineering at the University of Maryland was asked by a capacitor maker to do an analysis for publication. Their report - which I found rather challenging to read - indicated that the electrolytes of the bulging capacitors were missing a critical additive generally called a "depolarizer".
Depolarizers can help reduce the generation of gas within the E-cap. In this case, the lack of a depolarizer caused the electrolyte in the bad E-caps to be more alkaline or basic than it should be. Due to this and the fact that the aluminium oxide dielectric lacked other known protections like phosphate ions, the dielectric dissolved into the alkaline electrolyte. This caused the middle dielectric layer to get thinner, which in turn causes the capacitance to get unnaturally high - a key leading indicator of bulging. This exposed the underlying aluminium foil to reactions, where impurities in the foil can react to create spare electrons. And those electrons then in turn react with the hydrogen ions in this water-based electrolyte to create hydrogen gas.
## Industry-wide Talk of the Capacitor Plague tends to have the Rubycon industrial espionage story attached to it. We will never really know the truth, of course. There is nothing about it that makes me think it can't be true. In fact, I’m pretty sure something like it actually happened. Industrial espionage is a real thing.
However, I am skeptical that the whole "Plague" can be blamed on a single defector from Rubycon. I am more apt to believe that this was an extended series of unconnected challenges faced by the capacitor industry as a whole. Why? ## Too Widespread First, these were quite widespread. And people are too easily attributing every capacitor failure to the Plague and thus the Rubycon Story. In an earlier HackerNews discussion about the Capacitor Plague, there was a mass of "me too" comments, testifying that the Plague hit their audio equipment, Samsung monitors, and famously, Xboxes. And that makes me skeptical. This other stuff we can't trace, but we can with the original XBox, which was released in November 2001. That console is known for a bad capacitor that blows about 6-7
years in. And there are articles teaching users how to preemptively remove it on their own. People are apt to blame this bad capacitor - used for a clock - on the Plague. But this capacitor is a PowerStor Aerogel made by Cooper Industries, an American company based in Texas. Not only is this cap American, it is not even an aluminium E-cap. Per a July 2000 story in Everyday Practical Electronics magazine, the PowerStor Aerogel is a "supercapacitor", with its electrodes made from carbon aerogel foam, not aluminium foil. I doubt that it uses an aluminium oxide dielectric but feel free to comment if I am wrong. Considering how new it was, I am not surprised that these didn't last very long. So we have an unrelated case of bad capacitor-ship. But since the timing seems to match up,
people blame it on the poor dude at Rubycon anyway. I think this happened a lot more than we think. And speaking of timing ... ## Weird Timelines Second, the timeline is all weird. The Rubycon thing happened in mid-2002 per Zogbi's recollection. Again, there are some unsourced statements that it might have happened in 2001 as well. I buy that less. Even if we are being charitable and assume it took months to discover the bad batch, the common sentiment at the time was that a bad E-cap would reveal itself within a year. Perhaps
even less. Intel in the original 2002 report said a poor electrolyte E-cap would fail in as little as 250 hours of operation. So how can such "sleeper" E-caps keep blowing up into 2007? In late 2005, another round of bulging capacitors hit the news. Apple and Hewlett-Packard computers were affected, but Dell most of all. From May 2003 to July 2005, Dell shipped some 11.8 million OptiPlex PCs to mainstream and corporate customers with faulty capacitors that were 10 times more likely to bulge.
Dell people apparently knew of this and tried to hide the issue from customers. And thus they got sued for it in 2007 and spent $300 million on replacements. All of the bad capacitors came from a single manufacturer - Nichicon in Japan. HP put them on blast. When the news first came out about this in September 2005, Dennis Zogbi wrote that industry insiders told him that the Nichicon bad products were made by a Taiwanese sub-contractor. Okay. But three years earlier during the 2002 bulging capacitor incidents, Nichicon claimed that they had no plants in Taiwan. Sure,
it’s not the same but something about the timing and circumstances doesn't sit right with me. ## High Power So if it is not the Rubycon Scientist, then why are all these capacitors failing? Going back to Dennis Zogbi's 2005 report about the new round of capacitor bulging, he wrote: > My two electrolytic scientist colleagues in the industry ... felt that using an electrolyte that has been on the shelf since 2002 in Taiwan was unlikely, and that there may be a problem with using water based electrolytes to decouple new high-speed microprocessors This makes sense to me. These new Low-ESR aluminium e-caps with water-based electrolytes were only introduced in the late 1990s.
Such caps were brought in to handle increasingly power-hungry computers. The early 2000s were also a time of Intel and AMD aggressively pushing single-core clock rates. But with the end of Dennard scaling, these chips were consuming a lot of power and getting very, very hot.
I note that the failed Optiplex PCs from Dell were running Pentium 4 chips, which were infamous for this. So my head canon is this: Increasingly hotter microprocessors at the turn of the century heated up these new capacitors to such an extent that their water-based electrolytes simply boiled, creating gas that popped their containers. When AMD and Intel transitioned to multi-core CPUs starting in 2005, the internal temperatures scaled down. At the same time, capacitor-makers got the hang of
making these new-fangled aqueous electrolytes. Together, this finally ended the Capacitor Plague. ## Counterfeits One of the problems Zogbi pointed out in 2003 was the increasing complexity of the supply chain. He talked about brand name companies outsourcing parts like E-caps to no-name suppliers offering the cheapest prices. A valid concern, but there is another thing to consider: Counterfeits.
Even in the best of times, it could take several weeks to get a set of E-caps from manufacturers abroad. But when supply gets disrupted for whatever reason, that puts pressure on the OEM. Because what customer is okay with waiting so long for a tiny thing that costs less than $5 each? So the OEMs turns away from an authorized distributor and buys something off the market with only a quick check of the brand. The issue is that the capacitor is a counterfeit, and it won't last anywhere near as long as the real thing. One intriguing 2014 study of a set of counterfeit Nichicon E-caps - discovered after the March 2011 earthquake interrupted the legitimate supply chain - found that the counterfeit water-based electrolyte lacked enough ethylene glycol. This lowered the electrolyte's boiling point, making
it unstable at higher temperatures. Makes me feel like this sort of error is not all that uncommon. ## Conclusion All in all, this is a weird story. And was a challenging one to tell. I have seen a number of online commenters insist that all the companies' official statements are lies, that nothing can be trusted, and that the stolen formula story is the truth. If that's your insistence, there is nothing I can say to convince you otherwise.
But I reckon that the Rubycon Incident story is just a fun urban legend riding along with a real technical issue being worked out over the years by the capacitor industry. Aluminium E-caps with water based electrolytes are still around, and there are still longevity concerns. But they still offer the best capacitance for their size and price. And it seems like the vendors have largely worked out the kinks.
The little round can on the circuit board can go back to being unnoticed.
2025-03-20 02:56
