Powering the Internet
The Need for a DC Powering Option
For additional information, contact any of the members of this group of volunteers:
|
Lars Björkström |
John Parsons |
|
Cass Brown |
Jacques Poulin |
|
Gilles Esprit |
Klaus Rambold |
|
John Hawkins |
Chris Riddleberger |
|
Bob Jurewicz |
David Sinclair |
|
Heinz Kiehne |
Wilfried Schulz |
|
Bo Lindemark |
Joseph Thottuvelil |
|
Dan McMenamin |
Katsuichi Yotsumoto |
|
Seiichi Muroyama |
John Åkerlund |
Table of Contents
1. INTRODUCTION
1.1. Problem Statement1.2. Vision
1.4. Organisation
2.1. Addition of Datacom Equipment to Telecom Facilities2.2. Power for Traditional Telecom Equipment
2.3. Power for Datacom Equipment
2.4. Problems with Present Powering of Datacom Equipment
2.4.1. Reserve Time2.4.2. Earthing (Grounding)
2.4.3. Distribution
2.4.4. Harmonic Distortion
2.4.5. Reliability
2.4.6. Safety
2.4.7. Maintenance
2.5.1. Technical Simplicity2.5.2. Modularity and Maintenance
2.5.3. Redundancy and Battery Charging
2.5.4. Established World Norms (Standards) and Safety
2.5.5. Installed Base
2.5.6. Distribution and Isolation
3. CASE STUDIES
4.1. North America, Europe and Japan4.3. Trends
5.1. System Configurations5.2. Models for Unavailability Analysis
5.3. Unavailability Calculations
5.3.1. DC System 15.3.2. DC System 2
5.3.3. AC System
5.4. Conclusions
6.1. Central Office6.2. Remote Office
9. CONCLUSIONS AND RECOMMENDATIONS
10. REFERENCES
1. INTRODUCTION
1.1. Problem Statement
Telecommunications networks worldwide are evolving rapidly into systems carrying Internet and other data traffic in addition to traditional voice telephony. Internet traffic is handled by servers, routers and modems, which are furnished by vendors of the computer/datacom industry. These vendors also supply Asynchronous Transfer Mode (ATM), gigabit Ethernet and many other broadband communications equipment units.
These equipment units typically follow the practices common to the datacom industry, operating from the AC mains (commercial AC power) with Uninterruptible Power Systems (UPSs) providing several minutes of battery reserve power. The other equipment at a telecom facility, however, is powered by the traditional 24-, 48-, or 60-volt DC power plant that typically has several hours of battery reserve power. When datacom equipment is installed at telecom facilities that have an engine-generator set, the short-term battery reserve of the UPS may appear adequate at first sight. As the networks evolve, however, the datacom equipment is being installed in smaller telecom facilities. The reliability and reserve time of the UPS for this equipment is then not compatible with the need to provide continuity of service during mains outages. The public, however, expects the reliability of telephony for all services associated with telecommunications. The need for an extended interval DC powering option for datacom equipment is therefore urgent.
1.2. Vision
We expect every vendor of equipment used by telecom network operators to offer an option of four to eight hours operation during AC mains outages.
We believe that the most reliable, safest and cost-effective way to accomplish this is to power all equipment from the telecom DC power plant, particularly the 48-volt versions.
We seek the more efficient and less complex solution that replaces switch-mode rectifiers in datacom equipment with DC-DC converters or, in a distributed power architecture, powers 48-volt input converters or board-mounted power modules from the telecom DC power plant.
We urge all end users and specifiers of Internet and data communications products to purchase and install DC-powered equipment as a first choice.
1.3. Objective of this Paper
The objective of this paper is to develop the bases for recommendations that meet the goals of the vision statements in Section 1.2 above. The present situation is discussed, and then case studies illustrate the types of problems that have been encountered. AC mains power availability is reviewed. Reliability analyses, cost analyses and market estimates are presented. Progress toward goals is reviewed and conclusions and recommendations are presented.
1.4. Organisation
This paper is organised into nine major sections and a section containing references. A qualitative basis for the recommendations can be obtained from Sections 1, 2 and 8. Sections 4 through 7 provide data to support the recommendations.
2. PRESENT SITUATION
2.1 Addition of Datacom Equipment to Telecom Facilities
Telecom operators are adding large quantities of equipment for Internet service to their networks. These additions consist primarily of datacom equipment that typically is not available with a 48-volt DC power supply. Historically, most equipment for Internet service, operator service positions, operations support systems and surveillance systems operates from AC power and requires a UPS to provide service during mains outages. This situation creates the problems discussed in this paper.
The new equipment is often collocated in the same room, cabinet or rack as conventional telecom equipment. There are differences, however, in terms of battery reserve times, earthing and maintenance. The resulting problems are described below:
Different battery reserve times for system elements, which from operational and maintenance points of view are connected in series, leads to poor availability for service and to maintenance problems. Different system elements along the same chain, on national or global levels, may have differences in battery reserve times. These times can range from four to eight hours or more to as little as five to ten minutes. The lower reserve time sets the limit for the series-connected elements, resulting in unacceptable values for availability of service. The reserve times of the different elements are often unknown, which complicates the analysis of failure situations. Another consequence of multiple battery reserve power supplies is an increase in the number of batteries used and in the number of maintenance points in the systems. The quality of the batteries in different systems often varies and maintenance cannot be performed in a systematic manner.
DC systems are not earthed (grounded) in the same way as AC systems. The different norms (standards) result in practical problems in the implementation of safe and proper earthing of equipment. There are potentially significant safety hazards, for both personnel and equipment. There is a greater risk of accidents and fire. Maintenance becomes more difficult and mean-time-to-repair (MTTR) longer, due to the complexity of the installation and the difficulties in performing preventive maintenance.
Costs will be higher since two (or more) different power supply systems must be installed and maintained.
These problems and several others are discussed further in Section 2.4 below.
2.2. Power for Traditional Telecom Equipment
Power systems for a traditional telecom installation can be characterised as shown in Table 2-1 below.
Table 2-1. Traditional Telecom Power
Power System Battery Autonomy Reserve Energy System Comments 1a. Permanent Engine-Generator Set and 48-volt DC Power Plant (10-20 kW or more)
Typically 3 hours
Diesel engine-generator
1b. Permanent Engine-Generator Set and 48-volt DC Power Plant (10-20 kW or more)
Typically 1 hour
Dual diesel engine-generators
2. 48-volt DC Power Plant only (5-10 kW)
Typically 8 hours
Receptacle for mobile generator set
Back-up for the climate control system must be considered
3. New Site (typically <5 kW)
Up to 8 hours depending on services and location
Receptacle for mobile generator set
Back-up for the climate control system must be considered
All types of traditional telecom installations require highly reliable power systems. The requirements imposed by the telecom system on the power supply system and the portion of unavailability that may be allocated to the power supply system of a telecom installation is 5_10-7, which is 0.26 minutes or 15 seconds of service disruption per year. The systems in the above table meet this requirement.
2.3. Power for Datacom Equipment
Power systems for a telecom installation with datacom equipment can be characterised as shown in Table 2-2 below.
Table 2-2. Datacom Equipment Power
Power System Battery Autonomy Reserve Energy System AC Power for Datacom Equipment 1a. Permanent Engine-Generator Set and 48-volt DC Power Plant
Typically 3 hours
Diesel engine-generator
UPS or Inverter
1b. Permanent Engine-Generator Set and 48-volt DC Power Plant
Typically 1 hour
Dual diesel engine-generators
UPS or Inverter
2. 48-volt DC Power Plant only
Typically 8 hours
Receptacle for mobile generator set
Inverter or UPS (10 min.)
3. New Site
Up to 8 hours depending on services and location
Receptacle for mobile generator set
Inverter or UPS (10 min.)
In the above table, installations with UPSs do not meet the telecom unavailability requirement of 5_10-7, as will be shown in Section 5.
New installations consist mainly of local area networks (LANs) and wide area networks (WANs) on customers’ premises. They will be installed in large quantities to provide connections to the Internet. They will be located in industrial, office, commercial and residential buildings, and will be owned by the property owners or tenants. The telecom operators will install their interface equipment in the same rooms where the owners and tenants install their network equipment. Figure 2-1 shows the types of equipment that might be installed in such rooms.
Figure 2-1. Equipment Rooms
The power supply equipment in the room should be able to feed the equipment of both parties. A common standard for the power supply equipment, including high availability of power for the Internet connection, is therefore very desirable. It is in rooms like these that the meeting of the telecom and datacom industries, with their different cultures, will become clearly visible and tangible.
Voice will become an integral part of the Internet and connected LANs, under the name of Internet Protocol (IP) telephony. User expectations will be the same as for voice service over the public switched telephone network (PSTN), or even higher. Consequently, the traditional high availability of telephony, including operation during mains outages, will be requested for data networks.
2.4. Problems with Present Powering of Datacom Equipment
2.4.1. Reserve Time
Telecom systems powered by a 48-volt DC plant typically have battery reserve times of four to eight hours according to a universal standard, unless a backup diesel engine-generator set is provided. Datacom equipments powered by UPSs usually have reserve times of five to ten minutes. The differences in reserve times will be a great problem in installations without engine-generator sets, and will entail high costs for operators if the differences are eliminated by increasing the number of batteries in the UPSs or by installing engine-generator sets.
2.4.2. Earthing (Grounding)
Standards, rules and regulations for the earthing of telecom systems differ among vendors, operators and countries. Different priorities are given to different safety and operational aspects such as protection against injury, lightning strokes and transient overvoltages, and interference in signalling systems.
There is no whole truth or complete set of rules for simple guidance in the best way of earthing telecom equipment. Often, conflicting norms and regulations cover both telecom and datacom equipment. It is hardly a good idea to mix different technical solutions and earthing principles in one installation. Doing so will bring confusion and difficulties to the installation and maintenance organisations in their attempts to get an overall view and understanding of the design and function of the facility. It will adversely affect the analysis of faults. Another result of such a mixture is that different configurations may exist at different times during the life of a telecom facility, which may cause additional uncertainty and confusion.
A major risk of mixing DC- and AC-powered equipment within the same system is that the distribution conductors’ cross-sections are dimensioned in different ways. A short-circuit in a DC system can generate large fault currents to earth that may affect the operation of, or even damage, the electronic systems powered from AC feeders. Such failures are often very hard to detect, and their causes and effects are hard both to find and to explain. There is therefore an advantage in powering all the sub-systems from DC.
2.4.3. Distribution
Comparisons of DC and AC systems should consider their capabilities of protecting the connected telecom or datacom equipment from disturbances on the mains. Examples of such disturbances are high switching voltages, transients, lightning strikes, harmonic distortion and interference from other equipment. With the 48-volt DC system, the telecom and datacom equipments are always galvanically isolated from the mains. The large system battery also works as a filter against possible transients or harmonics passing through the rectifier. This isolation and filtering yields an almost total elimination of problems with disturbances passing from the mains to the DC distribution system.
Short-circuits in a distribution system cause transient voltages with great variations, depending on the resistances and inductances in the distribution system. In a 48-volt DC installation, a well-defined and predictable distribution system may be arranged, with control of both resistance and inductance. In such a system, voltage transients caused by short-circuits and fuse clearing are under control, and do not spill over to loads served by other feeders. The battery and electrolytic capacitors in secondary distribution panels are capable of providing enough current to clear the fuses quickly. The fuses and circuit breakers generate an alarm when cleared or tripped, thereby reducing the MTTR.
A unique transient limiting distribution system, shown in Figure 2-2, has been available for electronic switching systems for over 25 years.
Figure 2-2. Transient Limiting Distribution System
What happens during the period of short-circuit current is just the ordinary voltage division in the circuit. The voltage drop is distributed between the source resistance Ri and the resistance Ry of the distribution line with the short-circuit. Thus the voltage drops a maximum of 10% or 5 volts at the fuse panel. The DC-DC converters and board-mounted power modules in the loads easily handle this change in voltage.
In by-pass operation with a UPS, the telecom and datacom equipments are directly exposed to the mains, without any protection against disturbances. Obviously, the by-pass line can be fitted with various kinds of protectors and filters, but doing so adds cost. In on-line operation, with all the power passing through the rectifier and the inverter, it may be necessary to shift to by-pass operation to clear the fuse associated with a short-circuit in the secondary distribution system. The short-circuit current available from the UPS inverter may not be sufficient to clear a fuse. A mandatory condition to obtain fuse clearing is the use of automatic bypass equipment.
For secondary distribution, normal AC distribution practices are followed. With this type of distribution, it is hard to say how other parallel loads will be affected by a short-circuit and fuse clearing. There is another problem with distributing AC over long distances in a building; abnormal voltage drops can occur and cause undue stresses on switch-mode rectifiers (SMRs). One way of circumventing this problem is to use decentralised UPSs, one for each object of importance. This approach, however, can give problems with harmonisation of the battery reserve times and cause higher maintenance costs. There are evidently several factors of uncertainty in the UPS systems architecture that must be taken into account if it is to be used in telecom or datacom installations with high availability performance requirements.
Another very problematic phenomenon in all AC distribution systems is the occurrence of harmonic distortion. Harmonics are a very tangible problem with UPSs since they are not low-impedance power sources. For these problems to be mastered, the UPS and its load must be well known and preferably matched at the planning stage. There is limited room for uncontrolled upgrading and connection of new datacom equipment. The action usually taken to manage fuse clearing and harmonics-related problems is to over-dimension the UPS. This approach entails high costs, both for the UPS and for the mains power distribution system, which will have to be dimensioned accordingly. Another measure is to add new equipment, such as active harmonic filters. This approach adds cost, and may lead to a higher frequency of failures that causes a lower availability performance.
2.4.4. Harmonic Distortion
Harmonics are the result of loads that are non-linear. The common datacom load consists of a number of parallel-connected SMRs that have capacitors at the input stage. When they are connected to a sinusoidal voltage source, they do not draw a sinusoidal current. The voltage crest factor remains near ÷2, but the current crest factor can exceed 2.5, causing voltage drop in feeders. This voltage drop, however, is typically not a problem in small installations.
In three-phase UPS installations the most powerful harmonics are the third and fifth. The third harmonic can cause a serious problem in three-phase systems, since these currents do not cancel in the neutral conductor and may therefore overload it. This conductor is often not dimensioned for high currents, since the fundamental currents cancel in the neutral. If it comes to the worst, the overload may cause a fire. However, what usually happens is that circuit breakers trip or fuses clear for no apparent reason, causing unexplained disturbances in operation.
2.4.5. Reliability
As noted in Section 2.2, the annual unavailability allocated to the power supply system of a telecom installation is 15 seconds. This value is generally applicable to all types and sizes of nodes in a telecom system. It is also a value that has been in force and served as guidance in the design of power supply systems for telecommunications for a long time. All power supply systems must comply with this value in order to be compared, judged and evaluated on equal terms for service in public telecom networks.
According to comparative reliability studies (see Section 5), 48-volt DC systems show availability rates more than twenty times higher than comparable UPS systems.
Batteries of UPS systems generally have four to ten times as many cells connected in series as 48-volt DC systems, which brings the fault frequency of the UPS battery strings up to four to ten times that of the batteries in the 48-volt DC systems. The result is lower availability and more service-affecting incidents such as fires or post meltdowns. Higher ripple currents and deeper discharges can cause higher maintenance and replacement costs for batteries in UPS installations. Batteries furnished with UPSs typically have a lower life expectancy than batteries used in telecom DC power plants and must therefore be replaced more often.
Different telecom systems, powered by different power systems, with different availability rates, may without further consideration be connected in series with respect to reliability. With the present mix of DC and AC systems, the total unavailability of a telecom installation caused by its power supply will correspond to the predominant rate, since the rates are summed. In the case at hand, the lower rate of availability of AC systems will bring the availability of the overall system far below the current standard applied to telecommunications systems.
2.4.6. Safety
A complex mix of power supply systems in telecom exchanges is more likely to give rise to faults and accidents. Operating, maintenance and installation organisations face a more complex situation that can be difficult to grasp as a result of this mix. The risk of faults due to human error is clearly increased. Major and minor service disruptions in telecom systems occur largely in connection with work by people at the telecom exchanges. It is therefore vitally important that the telecom systems have simple and uniform designs to reduce the risk of these service disruptions.
UPSs are typically high-impedance sources of AC power and therefore do not have the same capabilities as the AC mains for clearing fuses and suppression of voltage harmonics. This characteristic must be taken into account when dimensioning the power distribution system from a UPS, in order to meet the requirements for co-ordination of fuses so that only the fuse closest to the fault will clear. There is the risk that different electricians may install additions or modifications without due regard for the capabilities of a UPS. Overloads in neutral conductors may be hard to detect and, in the extreme, may cause a fire.
2.4.7. Maintenance
The operators of facilities with a mix of 48-volt DC power plants and AC UPSs will incur higher costs for the maintenance of two battery systems and two different types of power electronics, double storage of spare parts, and the like. To these costs may be added the need for an increase in competence and/or a new category of maintenance technician, or the costs of double maintenance contracts with two different suppliers, since 48-volt DC and UPS installations are usually provided by different suppliers. The two power systems have significantly different mean time between failures (MTBF) rates and may require a different frequency and intensity of maintenance. This situation will complicate the planning and optimisation of the maintenance systems and organisations in large-scale operations.
2.5. Advantages of DC Power
2.5.1. Technical Simplicity
A 48-volt DC power supply system is characterised by its technical simplicity. It consists of a number of paralleled rectifiers that connect to two or more battery strings that are also connected in parallel. In the event of a mains outage or rectifier failure, the load continues to operate from the batteries without switching or interruption. The distribution of power to the loads originates at the point where the battery strings are paralleled, with only fuses or circuit breakers interposed. The electronic equipment in the load has built-in DC-DC converters or board-mounted power modules that interface to the battery.
2.5.2. Modularity and Maintenance
It is relatively easy to connect rectifiers and batteries in parallel for reliable load sharing, since the voltages are low and there is no need to consider phasing. These attributes pave the way for power supply systems with a modular design of the rectifiers, batteries and the conductors that interconnect them. Modular systems are imperative for simple and inexpensive maintenance of installations by persons with limited training on power systems, a need shared by telecom operators with other companies and organisations. We do note a trend for UPS systems to increase modularity by providing "hot-swaps" of battery and inverter modules.
2.5.3. Redundancy and Battery Charging
During operating conditions, the parallel rectifiers provide the current consumed by the load, the float current for the batteries and the additional current for recharging the batteries after a mains outage. Redundant rectifiers fill two needs: battery recharging after a mains outage and continued operation if one rectifier fails.
2.5.4. Established World Norms (Standards) and Safety
The battery voltage of most telecom switching equipment is -48 volts. This voltage is a universal standard for telecommunications equipment, and is well defined by both the European Telecommunications Standards Institute (ETSI) and the American National Standards Institute (ANSI). For AC power, there are 14 different voltage and distribution systems defined across the world. For single-phase UPSs, however, at least one vendor can cover the world with two products.
One of the motives behind the universal 48-volt DC standard is that it allows work on a live conductor with minimum risk for personal injury and without special safety measures. Dealing with live circuits is a practical advantage when craftspersons work at a distance from the voltage source and therefore can not disconnect it.
2.5.5. Installed Base
In the existing infrastructure of the telecommunications industry, there are millions of highly dependable 48-volt DC installations already in use by the operators. Economically and technically, it would be an absolute mistake to rebuild these installations to equip them with a new system for no-break AC power.
2.5.6. Distribution and Isolation
The DC distribution system is well defined and predictable since both resistance and inductance can be specified and controlled. In such a system, voltage transients caused by short-circuits and fuses clearing are under control, and do not spill over to loads served by other feeders. In one system, large capacitors in power distribution frames provide fuse-clearing currents. In another, the transient limiting distribution system discussed previously in Section 2.4.3 above, the battery alone provides the current to clear the fuses quickly.
Additionally, the galvanic isolation in the rectifiers and the large battery prevent mains disturbances from reaching the loads.
3. CASE STUDIES
The following list illustrates the types of problems that telecom operators have encountered with their AC-powered systems.
A central office/data centre suffered an unnecessary loss of service during a mains outage. Due to a minor control problem, the standby diesel engine-generator started but failed to transfer the entire load. The facility is equipped with a card access security system backed up by a small UPS. By the time an electrician arrived to correct the problem, the small UPS batteries had discharged, so the card access system had no power to open the doors for entry. Had the card access system been powered by the 48-volt battery there would have been no such problem.
A central office was equipped with a cluster of modems and channel service units used by several banking institutions. The cluster was powered by a small UPS, but small increases in load as the facility grew gradually "outgrew" the UPS. During a mains outage, the equipment failed after only five minutes of operation.
A technician opening a Power Distribution Unit (PDU) to take inventory information bumped a main breaker, knocking out power to a major subset of an operator service system and causing widespread service delays.
A technician was removing a circuit breaker panel faceplate prior to adding a branch circuit to a Power Distribution Cabinet. The faceplate slipped and turned off the main breaker to a 42-pole panel-board, causing the loss of several critical subsystems. Typically, DC secondary distribution systems are designed for orderly growth with very little intrusive work.
A telephone company business office, with approximately 250 service representatives, had telephones and computer systems that were fed from the AC mains and backed up by a diesel engine-generator. Unfortunately, all telephone calls to these representatives were disconnected when a mains outage occurred since the engine-generator did not furnish "no-break" power. As a further embarrassment, seemingly no power engineering went into the design for these facilities. As built, a small transformer assembly powered each cluster of 25 telephones. Each transformer was plugged into a multi-outlet strip, daisy-chained from other multi-outlet strips. Ultimately, one 15-ampere AC wall receptacle provided power for the entire operation. Of course, this was changed and inverters added to improve system reliability, but during the interim, a major marketing facility was dependent on a wall outlet that shared a circuit breaker and branch circuit with other receptacles. One janitor with a floor buffer could cripple the operation.
The failure of a static switch in a 50 kVA UPS took down a cluster of approximately 100 telephone operator consoles for several hours, resulting in network delays for operator-assisted calls. This event is one of many similar failures over the years. Since most operator service consoles now are based on personal computers (PCs) and the telephony portion is DC-powered, DC-powered PCs would be much more reliable for this application.
In the SS7 telephone-signalling network, it was common for Signal Control Point (SCP) computer systems to incorporate a quorum design philosophy, in which two units of any three had to be operational. To avoid single points of failure, this meant providing three UPS systems at SCP sites. This solution was very expensive from both a first-cost and a maintenance perspective. Reliability also suffered. For the past several years, the SCP/ISCP systems have been migrating to 48-volt DC-powered systems. DC power reduces the cost and provides enhanced reliability. Part of that reliability comes from diode-isolated dual "A & B" feeders to each cabinet or subsystem. Even the loss of a secondary distribution fuse will not take down the computers.
The service-affecting incidents described above would be far less likely to occur if the equipment involved were powered from the telecom DC power plant.
4. AC MAINS AVAILABILITY
4.1. North America, Europe and Japan
Studies of disturbances on AC mains by IBM[] and AT&T[] in the 1970’s provided a view of AC power in computer centres during that decade. A more recent and more comprehensive study by the National Power Laboratory[] (NPL) covered 112 locations over a five-year period. North America has a vast grid of interconnected generation and distribution systems. This grid, coupled with adequate generating capacity, ensures excellent frequency stability and infrequent outages caused by lack of capacity. The NPL data for total outages (zero volts) has a median value of 1.0 events/month and a mean value of 1.3 events/month. The NPL data for rms voltages £75% of nominal suggests 2.66 events/year with duration >5 minutes and 1.52 events/year with duration >30 minutes. For North America, therefore, it would be reasonable to engineer for one outage/month, with one outage/year that lasts for one hour.
Outage data for AC mains in the UK are shown in Figure 4-1[].
Figure 4-1. Reliability of the Mains 240 VAC Supply in Urban and Rural Areas of the UK
The above data for mains availability in the UK shows that one-hour outages can be expected to occur 0.12 times/year in urban areas and 1.11 times/year in rural areas. Values similar to the UK or North America can be expected for most of Western Europe.
The data for commercial AC power outages in Japan are shown in Table 4-1[]. These data are for 200- and 100-volt AC mains. It can be seen that the values are not significantly different from those of North America and the UK.
Table 4-1. Commercial AC Power Outages in Japan
Fiscal Year Number of Outages per User* Duration of Outages for a User**
(minutes)Mean Duration of Outages
(minutes/event)Planned Outages Unplanned Outages Planned Outages Unplanned Outages Planned Outages Unplanned Outages 1990 0.09 0.24 11 19 122 79 1991 0.07 0.43 8 158 114 367 1992 0.06 0.13 8 9 133 69 1993 0.06 0.17 6 32 100 188 1994 0.04 0.21 4 38 100 181 * The total numbers of outages divided by the total number of users.
** The total duration of outages divided by the total number of users.
The AC mains in these three parts of the world, and undoubtedly some others, can be characterised as "good" and power systems can be engineered as suggested for North America.
4.2. Other Parts of the World
There are many areas of the world where a rapid build-up of cellular systems is underway. Much data on mains power quality has been collected to help determine the causes of telecom power systems’ problems. Many countries have insufficient generating capacity. The AC mains experience long outages, frequency instability and large transient disturbances. As a starting point, one can engineer for 30 outages/year with three outages having duration greater than five hours.
4.3. Trends
It is probably safe to predict that things will get worse, not better, in those parts of the world where AC mains availability is now "good". Increasing demand, concerns over emissions of pollutants and greenhouse gases, finite supplies of fossil fuels and separation of generation, transmission and distribution of electric power pose threats to the AC power systems in countries where high availability is taken for granted. Telecom operators will increasingly need to consider co-generation and to insist on lower power consumption in new equipment. The latter has the double advantage of reducing not only the power consumed in the telecom equipment but also the power required for cooling.
5. RELIABILITY ANALYSES
5.1. System Configurations
The objective of this section is to compare the reliability of DC and AC systems, which are considered as the principal options for powering datacom equipment. References [] and [] provide additional information on reliability and system architectures.
DC power supply systems and an AC power supply system are shown in Figures 5-1 through 5-3. In DC System 1, the batteries and the boost converter back up the rectifier. The battery string typically comprises 23 cells. The reserve time is 3 hours. The rectifier and the boost converter are composed of (n+1) units, in which the extra unit provides redundancy.
For small plants, a DC system without an engine-generator but with 8 hours battery reserve is used. This system is called DC System 2 in this discussion.
The AC system shown in Figure 5-3 is typically used for powering datacom equipment. The AC system omits the boost converters and adds an inverter and an AC bypass circuit. Electrical Switch SWe connects the output to the inverter or to the bypass circuit without interruption. The control circuit CONT controls the operation of the switch SWe. The number of cells in the battery string depends on the output power and supplier’s design; it ranges from 54 cells (10 kVA output) to 206 cells (300 kVA output) in NTT, with a maximum reserve time of 3 hours. In this section, cases for reserve times of 3 hours and of 10 minutes are calculated. The concepts of unavailability analysis used in this section are applicable to other AC power supply systems, such as those that have redundant sub-systems. The results will, of course, depend on the system configurations.
Figure 5-1. System Configuration for DC Power Supply
Figure 5-2. System Configuration for DC Power Supply without Engine-Generator Set
Figure 5-3. System Configuration for AC Power Supply
5.2. Models for Unavailability Analysis
Models for unavailability analysis are shown in Figures 5-4 through 5-6.
The model for DC System 1 comprises two blocks, Block 1 and Block 2, which are connected in parallel. The DC System 2 model is composed of only three components. The model for the AC system comprises two series-connected blocks, Block 3 and Block 4. Block 4, which contains the power path of the switch and the controller for the switch, is the common component in the AC system reliability model, because failures in these components cause system failures.
Figure 5-4. Reliability Model for DC System 1
Figure 5-5. Reliability Model for DC Power Supply without Engine-Generator Set
Figure 5-6. Reliability Model for AC Power Supply
5.3. Unavailability Calculations
5.3.1. DC System 1
Unavailability for the DC System 1 (WDC1) can be calculated by adding the following two components of unavailability, W1 and W2:
W1, in which the battery and the boost converter BC are operating but the failure of Block 1 persists longer than the battery reserve time
W2, in which the battery and the boost converter BC fail during failure of Block 1.
WDC1 = W1 + W2
WDC1 = {WCS/EG _ exp(-µCS/EG _ T) + WSWm _ exp(-µSWm _ T) + WRF _ exp(-µRF _ T)}
+ {(WCS/EG + WSWm + WRF) _ (WBat + WBC)}WDC1 = 3.45×10-7 + 6.37×10-12
WDC1 = 3.45×10-7 (for T = 3 hours)
Reliability values for this calculation are shown in Table 3-1. From this result, we can understand why battery failures in DC systems have little effect on availability of power because the probability of simultaneous battery and rectifier failures is very small.
Table 5-1. Reliability and Repair Data for each
Component
5.3.2. DC System 2
The unavailability for DC System 2 is calculated as:
WDC2 = WCS _ exp(-µCS _ T) + WRF _ exp(-µRF _ T) + (WCS + WRF) _ (WBat)
Using values from Table 5-1:
WDC2 = 9_10-10 (for T = 8 hours)
5.3.3. AC System
The unavailability for AC System is calculated as:
WAC = (WCS/EG + WSWm) _ (WCS/EG _ exp(-µCS/EG _ T) + WSWm _ exp(-µSWm _ T)
+ WAC-DC _ exp(-µAC-DC _ T) + WDC-AC) + (WSWe + WCONT)(1) In the case of T = 3 hours (WAC1):
WAC1 = (1.57 _10-6) _ (3.89 _10-5 ) + (7.4 _10-6) = 7.4 _10-6
(2) In the case of T = 10 minutes (WAC2):
WAC2 = (1.57 _10-6) _ (6.18 _ 10-5) + (7.4 _10-6) = 7.4 _10-6
The unavailability of Block 3 is much lower than the unavailability of Block 4 since the AC bypass backs up the inverter sub-system. The electrical switch SWe and its controller CONT can be said to be the "bottleneck" of the AC system reliability. To increase the reliability, the unavailability of the common part (SWe and CONT) of the AC system should be decreased.
5.4. Conclusions
The calculated availability of the DC system is more than twenty times that of the AC system. In the AC system, the failure of a single component, such as the control circuit for the electrical (bypass) switch, can cause loss of power to the loads. Loss of power results in loss of service to customers, which is intolerable for telecom systems. If, as expected, AC mains availability worsens, DC systems become even more advantageous for powering all equipment units.
6. COST ANALYSES
One general perception of UPS powering is that it is more economical than 48-volt DC powering. In this section, we compare the cost of deploying UPSs versus 48-volt DC powering into two typical telecommunications scenarios.
For sake of simplicity, the costs for upgrading the standby engine-generator set, the distribution and cabling from the AC panel to the UPS or 48-volt DC power plant, potentially reinforcing the floor for the weight of additional batteries, equipment transportation and travel and living expenses during installation are considered equivalent in both scenarios, and therefore ignored in the analysis. Only the true differential costs, which are the costs of equipment, distribution cabling and installation labour, are compared.
6.1. Central Office
Scenario 1 below (see Table 6-1) considers the addition of a 10 kW load in a central office. In this environment, three hours of battery backup time is usually required and a standby engine-generator set is typically used to provide power during AC mains outages. For an AC load, the cost shown includes the UPS and two large external battery cabinets. A second UPS alternative with only 30 minutes of reserve is also presented, but this option is not recommended since a three-hour battery reserve time has already been established for the telecom equipment. For a 48-volt DC load, the cost shown is that of adding additional 48-volt DC rectifiers, batteries and distribution to the existing power plant in the central office.
Table 6-1. Addition of 10 kW Load in a Central Office
Load Type AC AC 48-volt DC Powering
UPS with 3-hour battery
UPS with 30-min. battery
DC plant with 3-hour battery
Equipment included
- 10 kW UPS
- 2 external battery cabinets
- Load cabling
- 10 kW UPS
- 1 external battery cabinet
- Load cabling
- 200 A rectifier
- Battery for 230 A / 3 hrs
- Distribution and load cabling
Cost for equipment and installation
US$21K
US$15K
US$17K
The comparison shows that 48-volt DC powering is the lower cost alternative for central office powering, when equivalent battery reserve times are deployed. It should also be noted that a long duration UPS often requires an external battery charger unit to maintain a reasonable battery recharge time. This unit was not included in the cost comparison. The 30-minute UPS solution has only minimal savings compared to the DC plant, but as noted above is not recommended.
The distributed powering architecture introduced in Section 6.2 below can also be considered for central office applications.
6.2. Remote Office
In Scenario 2 (Table 6-2), we consider powering options for a 5 kW load in a remote location, where eight hours of battery reserve time is generally the norm and no standby engine-generator set is available. Again, we look at similar scenarios: a UPS with a long duration reserve time, a UPS with only 30 minutes of reserve time and a DC plant with long duration reserve time. We also consider an additional option: 48-volts DC with a distributed power architecture (DPA). This option reflects the use of a 48-volt distributed powering architecture that is increasingly being used for medium and higher power datacom equipment. In this option, for reasons of modularity and the difficulty in distributing logic-level voltages at high currents, an intermediate 48-volt bus is established and DC-DC converters or board-mounted power modules are then used to derive the lower voltages. This option also allows relatively easy addition of battery backup, since only the 48-volt intermediate DC bus has to be backed up.
Table 6-2. Powering a 5 kW Load in a Remote
Office
Load Type AC AC 48-volt DC 48-volt DC with DPA Powering
UPS with 4-hour battery
UPS with 30-min. battery
DC plant with 4/8-hour battery
DC plant using distributed powering architecture with 4/8-hour battery
Equipment included
- 5 kW UPS
- 3 external battery cabinets
- Load cabling
- 5 kW UPS
- 1 external battery cabinet
- Load cabling
- 150 A rectifiers (N+1)
- Battery for 118 A
(4 hrs. & 8 hrs.)- Distribution and load cabling
- Battery for 118 A
(4 hrs. & 8 hrs.)- Battery charging, control and shelf
- Battery distribution & cabling
Cost for equipment and installation
US$16K (4 hours)
US$9K
With new plant:
US$17K (4 hours)
US$23K (8 hours)With existing plant:
US$10.5K (4 hours)
US$16.5K (8 hours)US$6K (4 hours)
US$11K (8 hours)
It should be noted that eight hours of battery reserve time is rarely available with most UPSs. In addition, the cost shown in Table 6-2 above does not include the additional battery charger unit needed to maintain a reasonable recharge time for the four-hour UPS.
For the 48-volt DC plant, the top figures shown in the table include the cost of installing a 600-ampere (ultimate capacity) complete power plant with N+1 rectifiers, batteries, distribution and cabling. There is, however, a future benefit from the DC plant alternative. The cost for powering loads beyond the initial requirement will be less, since no additional plant infrastructure or redundant rectification will be needed.
If this scenario considered expanding an existing DC power plant rather than installing a new plant, the costs for the DC powering option would decrease to US$10.5K (4 hours) and US$16.5K (8 hours).
The 48-volt DC with DPA option also offers the opportunity to increase power conversion efficiency when on battery operation and thereby reduce the battery sizing. The additional costs when using this option are then solely due to the battery and charging/control equipment needed. This option results in a better than two-to-one cost advantage when compared to the 4-hour AC UPS option, and is even less expensive than the AC-UPS option with 30-minute reserve.
In summary, one can conclude that the cost of providing power with four hours of battery reserve time in remote locations is generally no more and may be less with a 48-volt DC plant than with a UPS. One must also remember that powering with eight hours of reserve time is often not feasible with a UPS.
The UPS with 30-minute battery reserve was included in both scenarios, but not recommended because a longer battery reserve time had already been established for the site. It should be noted that batteries sized for 30 minutes reserve are discharged at a higher rate than batteries sized for three hours or more. The batteries for the 30-minute UPS, therefore, typically have thinner plates than the telecom power plant batteries, and have a shorter installed life. The costs of battery replacement were not included in Tables 6-1 and 6-2 above, but would of course increase the cost of the 30-minute UPS alternative.
7. MARKET ESTIMATES
By the turn of the century, the global market for data communications equipment is expected to exceed US$71B per annum. This market consists of several categories of products intended for LANs, WANs, remote access and network support and services.
of sales have historically been for private network applications, a growing portion is now deployed into public networks. This deployment comes about because the Public Network Operators (mainly the Local Exchange Carriers, the Competitive Access Providers or Newly Licensed Operators, the Long Distance Carriers and the Internet Service Providers) must modernise and expand their network capacity to handle the fast-growing demand for data communications traffic. Consequently, by the turn of the century, sales to these operators are expected to represent from 25% to 35% of the total demand for data communications equipment. The major applications for Public Network Operators will be for WAN switching and access (ATM, frame relay and router) and for network servers and data security solutions.
Figure 7-1. Global Data Communications Equipment Segments
Figure 7-1 illustrates the relative size of each equipment category. While the majority To preserve the Quality of Service (QoS) and level of reliability that their customers expect, the Public Network Operators will demand "telecom-grade" solutions from the data communications equipment vendors. These levels differ markedly from those historically found in private networks, where price has often been the key driving factor. One can also conceive that in many critical applications where the traffic originates and terminates in a private network (LAN or other CPE terminal), the same higher level of performance and reliability will be required. One such requirement is to have the reliable source of DC power deployed in voice telephony networks today.
Most vendors who are already involved in the WAN switching business, or who have a strong base in voice telephony, already offer telecom-grade data communications products. Other vendors, who have historically been more involved in the LAN, CPE and server segments, still tend to design for AC powering. They will need to make the transition to DC powering if they intend to sell to Public Network Operators.
8. PROGRESS TOWARD GOALS
Much of the switching equipment associated with Internet service is provided in the telecom equipment areas, and is available with a DC power input as standard. Third party operators providing value-added services in their own premises have traditionally installed servers and routers in computer room environments and made use of AC-fed equipment.
Until recently, these two markets have been seen as distinctively separate. The growth of the Internet network, however, requires that more equipment be installed in central offices. This factor, coupled with awareness amongst the major equipment suppliers that telecom operators are becoming increasingly interested in providing Internet services, has increased the demand for optional DC-fed equipment.
This customer demand has prompted major industry suppliers to provide routers and servers that can be fed with either a DC or an AC source, and there is clear evidence that new products by other suppliers are in development. The "pressure" to supply DC-powered equipment varies with the position of the equipment in the communications chain, and, to a lesser extent, cost and assumed overall reliability of the product. The nearer to the central office, the more likely that a DC-powered option will be considered. On the other hand, large customers with in-house routers currently see no need for DC-fed equipment, arguing that their in-house PC’s are AC-powered and would not operate during mains outages unless supported by UPSs. Similarly, when used in customer premises, terminal adapters are available only with an AC power option. Some network terminating equipment with both ISDN and conventional telephone inputs will however, still support the telephone service in case of a mains outage.
Where routers, servers or switching equipment are available with different power options, they come in various formats. Examples are:
- AC as standard with DC as an option
- DC as standard with AC as an option
- Dual AC or DC supplies, i.e. either AC- or DC-fed with a redundant input capability
- Twin AC or DC supplies, with either a third or a fourth redundant AC or DC supply option.
Routers that normally use AC power supplies typically cost slightly more when they are equipped with an optional DC power supply. Some servers, on the other hand, were initially DC-powered, but now come with AC power as an option. Typical voltage ranges for DC inputs are -48 V to -72 V, but more commonly -60 V. For AC inputs, typical ranges are 100 V to 240 V rms and 50 Hz to 60 Hz.
To show the diversity of equipment, typical examples are shown in Table 8-1.
Table 8-1. Examples of Power Supplies for Switches,
Routers and Servers
|
|
|
|
Switches |
AC only AC and redundant AC option AC or DC with dual redundancy option |
|
Routers |
AC or DC, dual redundancy optional |
|
Servers |
AC or DC, dual redundancy DC is standard, but with custom rectifier |
To further increase reliability, equipment is currently being designed that uses AC or DC with dual redundancy and dual internal power supplies. This new equipment can be fed from two separate power sources and has a redundant internal power supply. With this approach, it will not be possible to mix AC and DC power supplies. However, equipment is being developed that has dual DC inputs and a separate AC input-DC output power pack, making it possible to feed the equipment from both an AC and DC source.
The introduction of a distributed powering architecture that uses a 48-volt DC bus to distribute power to DC-DC converters and/or board-mounted power modules should be welcomed by operators who must install datacom equipment in telecom facilities. The increased efficiency of power distribution within the datacom equipment is utilised at all times, and battery reserve operation is achieved by powering the DC bus from the telecom DC power plant. Before connecting this bus to the DC power plant, the operator should verify that the bus converters and power modules can operate over the full range of the power plant output voltage.
As in the PC industry, there is likely to be an increased trend of end users being able to specify and purchase equipment designed to meet their specific site requirements.
9. CONCLUSIONS AND RECOMMENDATIONS
We have examined the powering arrangements that have typically been chosen for Internet equipment installed in telecommunications facilities. We have reviewed the problems inherent in AC power systems, the difficulties encountered when both AC and DC power systems are used within a facility, and the advantages of using DC power for all equipment that is added at a facility. The entry of telecom operators to the business of Internet Service Provider (ISP) and the anticipated growth of IP telephony will lead customers to expect the same level of service on the Internet as they receive on the circuit-switched telephone networks today. This expectation mandates that all telecom and datacom equipment that provides communications service must have four to eight hours of battery reserve, as expressed in the first of our four vision statements.
Considerations of availability, equipment and maintenance costs, distribution, safety and the like reveal that operating all equipment from the telecom DC power plant is clearly the optimum choice as noted in our second vision statement. We did not discuss the use of inverters powered by -48 volts rather than UPSs; the four stages of power processing (one in the inverter, three in the SMR) are double those of a DC-DC converter. This difference and many of the problems associated with AC power are the bases of our third vision statement.
There is good news for telecom operators regarding the final vision statement: many vendors already offer optional DC-DC converter power supplies in their switches, routers and servers. Although the converters often have a higher price than the SMRs, the size of the market for datacom equipment for telecom operators is so large that the demand for converters should be large enough to generate high-volume production prices. The introduction by datacom equipment vendors of a distributed power architecture based on a 48-volt bus, discussed in Section 6.2, offers operators the opportunity to save money by omitting the bulk rectifier when purchasing datacom equipment for telecom facilities. The primary responsibility for achieving our vision therefore lies with the operators. Power engineers must identify the organisations in their companies that specify datacom equipment and convince them to insist on DC power supplies for all their applications in telecom facilities and also wherever telecom grade-of-service is needed. It is our hope that this document will be helpful, not only within operators’ companies but also in procurement discussions with vendors of Internet equipment.