Tweet
tl;dr: 230kv power line, buried 15 ft under LA needed a repair... sending huge amp pulses down the powerline with so much power that it can be heard from street level to locate the fault, using liquid nitrogen to freeze oil plugs in place to allow work to happen, etc.
Infrastructure is a pretty cool thing.
This is a string of emails from 1989. They were posted to an engineering board back in 2003, and have been floating around ever since.
The next day:
Infrastructure is a pretty cool thing.
This is a string of emails from 1989. They were posted to an engineering board back in 2003, and have been floating around ever since.
[Not quite the right mailing list but close. If you don't care about megawatts, bus bars bigger than your wrist, things that cause ground loops out to Hawaii, or big hairy construction projects hit D now.]
Several days ago a very large number of trucks and men from the Los Angeles Department of Water and Power descended on my neighborhood. They removed large sections of Pershing drive to a depth of 15 feet or so over a stretch of about a city block. I assumed they had a problem with a water main or something.
When they started building semi-permanent structures over the holes I knew something really big was up. When the large trucks full of strange power tools, mega-welding machines, breathing equipment, and racks of test equipment came I started wondering. Driving by a couple nights ago (11 PM), I noticed that the pace hadn't slowed - they were at it 24 hours a day.
My curiosity got the best of me yesterday when they brought in the giant tanks full of liquid nitrogen. LN-2 for the DWP? I parked my car and played the lookie loo.
It turns out they have a problem with an underground wire. Not just any wire but a 230 KV, many-hundred-amp, 10 mile long coax cable. It shorted out. (Lotta watts!) It feeds (fed) power from the Scattergood Steam Plant in El Segundo to a distribution center near Bundy and S.M. Blvd.
To complicate matters the cable consists of a copper center conductor living inside a 16 inch diameter pipe filled with a pressurized oil dielectric. Hundreds of thousands of gallons live in the entire length of pipe. Finding the fault was hard enough. But having found it they still have a serious problem. They can't afford to drain the whole pipeline - the old oil (contaminated by temporary storage) would have to be disposed of and replaced with new (pure) stuff which they claim takes months to order (in that volume). The cost of oil replacement would be gigantic given that it is special stuff. They also claimed the down time is costing the costing LA $13,000 per hour. How to fix it and fast?
That's where the LN-2 comes in. An elegant solution if you ask me. They dig holes on both sides (20-30 feet each way) of the fault, wrap the pipe with giant (asbestos-looking) blankets filled with all kind of tubes and wires, feed LN-2 through the tubes, and *freeze* the oil. Viola! Programmable plugs! The faulty section is drained, sliced, the bad stuff removed, replaced, welded back together, topped off, and the plugs are thawed. I was amazed.
When they started building semi-permanent structures over the holes I knew something really big was up. When the large trucks full of strange power tools, mega-welding machines, breathing equipment, and racks of test equipment came I started wondering. Driving by a couple nights ago (11 PM), I noticed that the pace hadn't slowed - they were at it 24 hours a day.
My curiosity got the best of me yesterday when they brought in the giant tanks full of liquid nitrogen. LN-2 for the DWP? I parked my car and played the lookie loo.
It turns out they have a problem with an underground wire. Not just any wire but a 230 KV, many-hundred-amp, 10 mile long coax cable. It shorted out. (Lotta watts!) It feeds (fed) power from the Scattergood Steam Plant in El Segundo to a distribution center near Bundy and S.M. Blvd.
To complicate matters the cable consists of a copper center conductor living inside a 16 inch diameter pipe filled with a pressurized oil dielectric. Hundreds of thousands of gallons live in the entire length of pipe. Finding the fault was hard enough. But having found it they still have a serious problem. They can't afford to drain the whole pipeline - the old oil (contaminated by temporary storage) would have to be disposed of and replaced with new (pure) stuff which they claim takes months to order (in that volume). The cost of oil replacement would be gigantic given that it is special stuff. They also claimed the down time is costing the costing LA $13,000 per hour. How to fix it and fast?
That's where the LN-2 comes in. An elegant solution if you ask me. They dig holes on both sides (20-30 feet each way) of the fault, wrap the pipe with giant (asbestos-looking) blankets filled with all kind of tubes and wires, feed LN-2 through the tubes, and *freeze* the oil. Viola! Programmable plugs! The faulty section is drained, sliced, the bad stuff removed, replaced, welded back together, topped off, and the plugs are thawed. I was amazed.
The next day:
Last night the DWP held a curbside chat to allay the neighborhood's fears that they were going to accidentally blow us all up. Apparently all the vapor clouds from all the LN-2 blowoff had caused a great deal of concern.
Interesting bits:
The feeder was laid 17 years ago and was designed to have an MTBF of 60 years. There are other similar feeders in use around California, in the Pacific North West, and some on the east coast. This was the first failure in the western US. No one out here had any idea how to fix it so they brought in experts from the east. (NYC has had some faults.)
This link is a very critical part of the LA power grid. Last night the city engineers verified the $13,000 per hour power cost figure quoted the day before. (I guess that means they are being forced to buy power off the grid somewhere else.)
There are actually three center conductors (they had a cross sectional model to show us). Each is about 3" in diameter with a one inch solid copper core. Each is wrapped with hundreds of layers of a special paper. That, in turn, is sheathed with copper and then each one is spiral-wrapped its entire segment-length with a 1/4 inch bronze "wire". The three conductors are then twisted together during the pulling process. The bronze spiral wraps form a kind of linear bushing with minimal contact area with the inside of the pipe so it's "easy" to pull each segment. Ha.
Each of the three legs in the feeder carrys 600-800 Amps (depending on demand) of 230KV three phase power. The ground return is the Santa Monica Bay. Down at the Scattergood Steam Plant and up in Santa Monica they have a giant copper anchors out in the bay.
They lay these things in 2000 foot segments. 2000' is the longest segment they can pull through the steel pipe. The pipe is laid first and then the internal cable(s) are pulled through. Tensile forces must be enormous. At each segment joint (splice) there is a very large and expensive ($100K) underground vault. Future technology may allow them to go 3000 feet, reducing the number of vaults needed per run, thereby saving money.
After the feeder was originally built (and the cable pulled) it was thoroughly evacuated to both leak test and remove any contaminants. It was flushed with dry nitrogen and then reevacuated. Golden Bear High Tension Oil was then slowly added while still maintaining a vacuum so as to "pull" any residual gas contaminants out of the oil and the cables in the pipe. The pipe, full of oil, is then pressurized to about 200 PSI for some period of time before it gets powered up. 200 PSI is maintained during operation to keep any bubbles from forming and to drive insulating oil into the paper.
At both ends of the pipeline they have 6000 gallon tanks of Golden Bear lightly pressurized under a blanket of dry nitrogen. There are pumps at both ends. There is about 100K gallons in the entire pipeline, not including the 6K gal tanks. Every six hours they reverse the pumps so the oil oscillates back and forth in the pipe. The pumps only run at 3 gallons per minute but that is enough, over 6 hours, to get the oil in each 2000 foot segment to go at least a segment or two length in either direction. This eliminates hot spots in the copper conductors and spreads the heat out over several thousand feet. A little competitive pressure is always maintained between the pumps to get the 200 PSI.
They learned the hard way that you simply don't reverse the pumps lest you get the Golden Bear equivalent of water hammer. The last hour of every 6 hour cycle is spent slowly reducing the oil velocity down to zero before you reverse it and then slowly ramp back up in the other direction.
In between segments, in the vaults, are temperature sensors embedded in the pipe. These monitor the oil temperature. These are wired to a computer downtown. Because the oil oscillates, the DWP can track the temperature gradient along the pipe and get an early indication of the location of any hot spot problems. They have regularly spaced flow rate and pressure monitors for the same purpose - detecting and isolating faults.
Every vault also has a nipple which allows sampling of the pipe oil. They said you withdraw the oil through a thick membrane with a syringe (?). This happens monthly on all feeders in the LA area. The samples are analyzed downtown by a staff of chemists who can relate the presence of things like acetylene, butane, and benzene in the oil to arcing, coronas, and so forth. Apparently the oil chemistry is a very good indicator of the health of the segments.
One of their worst fears, after they open up the pipe, is having a blowout of the freeze plugs. If they ever run out of nitrogen during the repair process they'll lose one side of the pipe (or both). Right now they've got the pipe on each side of the fault dropped down to 80 PSI. They are afraid that if they go any lower in oil pressure any gas in the oil will come out of solution and cause an explosive expansion. Not only that, but since there is so much oil embedded in the paper insulation, any sort of gas bubbling (oil foaming) would shred the insulation, rendering the entire feeder useless. They say it could take months to safely let the pressure off to zero. (That is the other reason ($13k/hr) they cannot afford to drain the whole pipe.)
Even at 80 PSI, if they lose a freeze plug they will have a really big mess outside the pipeline. The holes they've dug cannot hold 100K gallons and they're operating on a hill near the beach anyway... (Big pollution threat for LA basin.) Potentially fatal for anyone around. Right now they have LN-2 companies on call from San Diego to San Francisco with contingency plans of all sorts in case there is a major traffic problem with trucks getting in.
Interesting bits:
The feeder was laid 17 years ago and was designed to have an MTBF of 60 years. There are other similar feeders in use around California, in the Pacific North West, and some on the east coast. This was the first failure in the western US. No one out here had any idea how to fix it so they brought in experts from the east. (NYC has had some faults.)
This link is a very critical part of the LA power grid. Last night the city engineers verified the $13,000 per hour power cost figure quoted the day before. (I guess that means they are being forced to buy power off the grid somewhere else.)
There are actually three center conductors (they had a cross sectional model to show us). Each is about 3" in diameter with a one inch solid copper core. Each is wrapped with hundreds of layers of a special paper. That, in turn, is sheathed with copper and then each one is spiral-wrapped its entire segment-length with a 1/4 inch bronze "wire". The three conductors are then twisted together during the pulling process. The bronze spiral wraps form a kind of linear bushing with minimal contact area with the inside of the pipe so it's "easy" to pull each segment. Ha.
Each of the three legs in the feeder carrys 600-800 Amps (depending on demand) of 230KV three phase power. The ground return is the Santa Monica Bay. Down at the Scattergood Steam Plant and up in Santa Monica they have a giant copper anchors out in the bay.
They lay these things in 2000 foot segments. 2000' is the longest segment they can pull through the steel pipe. The pipe is laid first and then the internal cable(s) are pulled through. Tensile forces must be enormous. At each segment joint (splice) there is a very large and expensive ($100K) underground vault. Future technology may allow them to go 3000 feet, reducing the number of vaults needed per run, thereby saving money.
After the feeder was originally built (and the cable pulled) it was thoroughly evacuated to both leak test and remove any contaminants. It was flushed with dry nitrogen and then reevacuated. Golden Bear High Tension Oil was then slowly added while still maintaining a vacuum so as to "pull" any residual gas contaminants out of the oil and the cables in the pipe. The pipe, full of oil, is then pressurized to about 200 PSI for some period of time before it gets powered up. 200 PSI is maintained during operation to keep any bubbles from forming and to drive insulating oil into the paper.
At both ends of the pipeline they have 6000 gallon tanks of Golden Bear lightly pressurized under a blanket of dry nitrogen. There are pumps at both ends. There is about 100K gallons in the entire pipeline, not including the 6K gal tanks. Every six hours they reverse the pumps so the oil oscillates back and forth in the pipe. The pumps only run at 3 gallons per minute but that is enough, over 6 hours, to get the oil in each 2000 foot segment to go at least a segment or two length in either direction. This eliminates hot spots in the copper conductors and spreads the heat out over several thousand feet. A little competitive pressure is always maintained between the pumps to get the 200 PSI.
They learned the hard way that you simply don't reverse the pumps lest you get the Golden Bear equivalent of water hammer. The last hour of every 6 hour cycle is spent slowly reducing the oil velocity down to zero before you reverse it and then slowly ramp back up in the other direction.
In between segments, in the vaults, are temperature sensors embedded in the pipe. These monitor the oil temperature. These are wired to a computer downtown. Because the oil oscillates, the DWP can track the temperature gradient along the pipe and get an early indication of the location of any hot spot problems. They have regularly spaced flow rate and pressure monitors for the same purpose - detecting and isolating faults.
Every vault also has a nipple which allows sampling of the pipe oil. They said you withdraw the oil through a thick membrane with a syringe (?). This happens monthly on all feeders in the LA area. The samples are analyzed downtown by a staff of chemists who can relate the presence of things like acetylene, butane, and benzene in the oil to arcing, coronas, and so forth. Apparently the oil chemistry is a very good indicator of the health of the segments.
One of their worst fears, after they open up the pipe, is having a blowout of the freeze plugs. If they ever run out of nitrogen during the repair process they'll lose one side of the pipe (or both). Right now they've got the pipe on each side of the fault dropped down to 80 PSI. They are afraid that if they go any lower in oil pressure any gas in the oil will come out of solution and cause an explosive expansion. Not only that, but since there is so much oil embedded in the paper insulation, any sort of gas bubbling (oil foaming) would shred the insulation, rendering the entire feeder useless. They say it could take months to safely let the pressure off to zero. (That is the other reason ($13k/hr) they cannot afford to drain the whole pipe.)
Even at 80 PSI, if they lose a freeze plug they will have a really big mess outside the pipeline. The holes they've dug cannot hold 100K gallons and they're operating on a hill near the beach anyway... (Big pollution threat for LA basin.) Potentially fatal for anyone around. Right now they have LN-2 companies on call from San Diego to San Francisco with contingency plans of all sorts in case there is a major traffic problem with trucks getting in.
Comment