In part one of this series, we addressed the basics of microgrid technology and its potential to reshape our energy landscape. In part two, we embark on a deeper journey, exploring the practical applications and use cases of microgrids. We’ll examine how the real-life implications and opportunities presented by microgrids are poised to revolutionize the way we power our communities and businesses.

Peak Demand and Net Metering
From a utility perspective, “peak demand” refers to the period of highest electricity consumption within a given time frame, typically occurring during times of high energy usage, such as hot summer afternoons or cold winter evenings. Peak demand often places significant stress on the grid infrastructure and can lead to higher operational costs for utilities, due to the need for additional generation capacity. To manage peak demand, utilities may implement demand response programs or charge higher rates during peak hours to incentivize consumers to reduce their electricity usage. 

For example, the utility company Rocky Mountain Power, considers peak demand to be 1pm-8pm from June to September, while in the winter months, peak demand is 8am-10am and 3pm-8pm. It’s during these times that residential customers can see an increase between 1.3 and 4.5 cents/kWh, while commercial customers can expect to pay 1.2-2x their normal rate.

Net metering, on the other hand, is a billing mechanism used in solar energy applications that allows consumers who generate their own electricity, typically through solar panels, to receive credit for the excess electricity they feed back into the grid. When they generate more electricity on-site than is consumed, the excess energy is exported to the grid, and the consumer receives credits on their utility bill for the kilowatt-hours (kWh) exported. These credits can then be used to offset the cost of electricity consumed from the grid when solar production is insufficient, effectively allowing consumers to "net out" their electricity usage over a billing period. 

Programs like these encourage the use of solar energy systems so they can generate their own renewable electricity while remaining interconnected to the grid for reliability. Net metering also applies to power fed to the grid through BESS (Battery Energy Storage System), fuel cell, or any numbers of DERs (distributed energy resources) connected to the grid.

Microgrid Structure
Let’s start with a simple example of a microgrid application. We will utilize a 250 kW solar array, diesel genset, and a 250 kW BESS.

In normal operation, the utility is charging the BESS during the day or non-peak demand hours. The load is being reduced with solar production, and excess solar is also going to the battery to charge it. If the BESS battery is full, the solar is back feeding the grid, via net meter, and the owner is gaining credit from the utility. The diesel genset is in standby mode until needed.

Use case/Application

Scenario 1:
Let’s assume the above paragraph is true. Say our 250 kW battery is sized for 4 hours of runtime. The facility we are powering draws 125 kW of load between 1pm - 8pm. Utility is operational, battery is fully charged, 1pm - 8pm is peak demand hours, resulting in a 1.5x multiplier/kWh used. From 1pm – 8pm, the BESS will discharge to 20%, depending on load, shaving the kWh used by 80%. This means, that from 1pm - 8pm, instead of paying 1.5x the normal cost /kWh, you are now “netting” a credit of 1.5x, because at that time, it is the going rate/kwh. From an ROI perspective, gaining this credit daily helps significantly reduce the initial upfront investment required. Once peak is over, the BESS begins to charge on utility through the night and into the morning, at the cheapest rate available and the whole process starts again.

Scenario 2:
The utility feed sees a blip in voltage greater than 10%. Without a BESS, this will kickstart your genset to turn on and ramp up to load. With a BESS, the load simply switches from utility to BESS, the genset stays off until a critical point is reached in BESS charge levels, and then the genset will kick on. Utah has a very stable grid, but markets in California, Texas, and New York can see blips daily. Reducing genset starts directly reduces carbon emissions, and saves the components on the generator from mechanical wear. 

Most outages we see are small scale that do not require a full generator start. Today, most controls and logic have a very low tolerance to voltage sags and dips, and will auto start the generator within 3 millisecond of seeing a voltage problem. Adding a BESS into this system will allow a much larger time delay on gen starts and ultimately save maintenance and operational costs over the genset’s lifetime. We are not at the point where a BESS can completely replace a generator, but it allows for down-sizing and decrease operational costs. Another point to consider is the controls behind a diesel generator and BESS system. You can favor the BESS or the generator all through controls schema, it just depends on the priorities of the end user.

Scenario 3:
Utility goes down. The BESS is kicked into islanding mode, which is achieved through controls and circuit breakers, and the critical load gets transferred to the BESS. Depending on the runtime specified in the BESS, this will eliminate a generator start for the duration specified on the BESS. If utility is not restored, then a gen start will be required. The added redundancy in this scenario cannot be understated. You can treat the BESS as an uninterruptible power supply (UPS), you can backup your UPS via BESS. Many configuration options exist in this instance and allow end users to be flexible when deciding how to manage outages. This scenario is where the importance of controls and isolation needs to be highlighted.

Electronic isolation and controls ensure the safe and efficient operation of microgrid components and prevent system failures and electrical hazards. Through advanced control algorithms and monitoring systems, operators can oversee the performance of microgrid assets in real-time, and make adjustments to maximize efficiency and reliability. Additionally, with the addition of an isolation switch, it is necessary for mission critical environments to include a maintenance bypass into the isolation loop. This prevents single points of failure and allows for on-line maintenance and equipment changes. Ask your local DVL sales representative about our Power Engineering and Controls team to learn more.

Scenario 4: 
Let’s get creative with the layout of a microgrid. So far, we’ve been discussing small amounts of renewable energy distributed on the site. We are now going to add some Hydrogen (H2) to the equation. Say we have 250 kW of a PV array, there is also a 250 kW BESS. Cue the addition of a hydrogen electrolyzer, which is a device that utilizes electricity to split water molecules (H2O) into hydrogen (H2) and oxygen (O2) through a process called electrolysis. This technology enables the production of hydrogen gas, which can be used as a clean and sustainable energy carrier for various applications, including fuel cell vehicles, energy storage, and industrial processes. Hydrogen electrolyzers play a key role in the transition to a low-carbon economy by providing a renewable source of hydrogen fuel without producing greenhouse gas emissions. 

We are now feeding the electrolyzer a steady supply of water and voltage (through a joint effort of utility and photovoltaic [PV] array) and we’re now producing hydrogen on-site. We then need to compress it to store it in a tank, which, again, can be achieved through utility or PV. We will then add a 1.3 MW fuel cell into the equation. H2 (hydrogen) is approximately 3x more energy dense/liter than diesel fuel. Which means if a 10,000 gallon tank of diesel could provide 36 hours (3 days) of runtime, a 10,000 gallon tank of hydrogen could provide you with 108 hours (9 days) of runtime. The capacity difference is enough, in my opinion, to get the end user to switch, but on top of that, here is where you start seeing the building blocks of a self-sustainable BYOP (bring your own power) solution, where you’re powering yourself and capable of creating more fuel on-site. 

This list of scenarios is by no means complete, but it helps to illustrate why a microgrid structure will be critical to the data centers of tomorrow. One more note on these systems is, while they may be expensive, end users are not the only ones who stand to reap benefits. Utility providers are very motivated to help support the build out of these energy programs. The more DERs that utilities have a feed into, the less they need to buy power from other supporting grids. Rocky Mountain Power has put approximately $5,000,000 into a Wattsmart battery incentive program that is looking to subsidize grid size BESS solutions. 

You can get an upfront incentive when you enroll, and you get monthly credits for supporting the grid. While it may seem like a lot of money to offer end users, the end goal of the utility is increasing their capacity without building a new power plant. This allows for a truly rare “win-win” situation for utility provider and end user.

Product Offering
Vertiv Dynaflex+ BESS. Available in 250 and 500 kW or 1, 1.5, and 2 mW. These batteries can be paralleled in groups up to 4 of the same kind, which would allow for up to 8 mW of storage per chain. Included in the BESS is a bi-directional inverter, for charge and discharge. These are containerized solutions capable of being dropped into any campus. They offer Li-Ion battery chemistry with a full suite of monitoring and controls. They also include a liquid cooling loop and can be placed in almost any environment. The Dynaflex is compliant with all UL fire listings, as well as the 9540 for Li-Ion batteries. And, as previously mentioned, peak shaving is a great use-case for these systems, as well as emergency power, solar energy storage and more.

Generac also has BESS offerings in the 500 and 1000 kW range. They include the full suite of features listed above.

One thing to note, is the Dynamic Online Mode available for addition on Vertiv/Liebert EXL S1 UPS. It will treat the UPS as a BESS and create a schedule that allows for the discharge of the UPS batteries to a setpoint throughout the day to create a “revenue” generating asset that is deployable at any time. Essentially a “Micro” BESS.

The way to prepare our industry for its best possible future is to not just follow status quo, but to question, learn, and educate ourselves, so we can understand the complexities of sustainable energy solutions. The scenarios outlined here illustrate the tangible benefits and opportunities offered by microgrids and battery energy storage systems. From managing peak demand to integrating renewable energy sources and more, we see the potential for a brighter and more sustainable future. 

Please stay posted for the third and final part to this blog series, out in May,
which will explore electric vehicle (EV) charging infrastructure.

Have a question or comment about this blog?

Reach out to blog author Alexander "D'Angelo" D'Angelo, Power Systems Sales Engineer,  (based out of our Salt Lake City office) at ADangelo@DVLnet.com.

Are you ready to revolutionize the way we power our communities and data centers? Picture a future where electricity isn't just distributed from centralized grids but generated and managed locally. Welcome to the world of microgrids, battery energy storage systems, and electronic isolation and controls. 

While it is fun to use these buzzwords and speak about the possibilities the future holds, why does this matter? Simply put, resources. Whether it is capital, space, power, water, or talent, we live in a resource constrained world. As our technology becomes more advanced, its demands for power and cooling will increase. This puts a large strain on our already fully loaded power grids, with the states ¹most at-risk being Texas, Michigan, Ohio, New York, and California. Texas is not interconnected to the national grid, which puts it at risk for downtime due to a lack of redundancy. New York and California, on the other hand, are strained due to their large populations and the decommissioning of traditional power plants. Additionally, with an increase in legislation supporting EV vehicles, the strain on the grid can be too large especially in inclement weather (i.e. hot and cold) increasing risk of downtime. 

Like it or not, soon we will have to supplement power and storage solutions that are smart and reliable enough to be treated as de-centralized grid assets. Let us dive deeper into the realm of Microgrids. 

What is a microgrid? 

Microgrids represent a paradigm shift in how we think about energy distribution. These localized grids can operate independently or in conjunction with the main grid, offering resilience and flexibility in the face of outages and disruptions.  

So, what are some of the basic components that we’d expect to see in a microgrid? Renewable energy, most commonly solar (PV), wind, or, in some cases, hydropower. Next, we would expect to see an inverter to convert the energy from the renewables to a usable form for the loads that are connected. After that, a BESS (Battery Energy Storage System), isolation with controls, a fuel cell, and/or hydrogen electrolyzer.  

While these individual components, alone, could not support an outage, when deployed together, the sky is the limit for “islanding” yourself from utility. These assets could be on a commercial site, outside of a housing community, a data center, and beyond. These are the building blocks for these locally deployed decentralized grids. 

Imagine a community powered by its own microgrid, seamlessly integrating renewable energy sources, like solar panels and battery storage systems, into its infrastructure. These technologies not only reduce reliance on fossil fuels but also pave the way for a more sustainable future.  

Outside of the communities, integrating renewables into their energy portfolio, there are mission critical operators who look to add redundancy to their utility connection and further control their uptime parameters. Mission critical operations are businesses that cannot suffer an outage even for a second. These customers are mostly data centers, healthcare providers, departments of transportation, utilities, etc.  

Furthering the point of living in a resource-constrained environment, these providers are seeing that the addition of high compute applications are driving their energy consumption up higher every year. To combat the risk associated with simply relying on utility, they deploy uninterruptible power supplies, generators, and, now, renewables and BESS systems to allow them even more flexibility during utility loss. 

Market Overview 

As AI and other high performance compute practices start becoming the norm in the market, the utilities won’t be able to adapt quick enough. Standard per rack power density in hyperscale and co-location data centers ranges from 10 - 20 kW of consumption. And, in the next 3-5 years, market analysis predicts for this to shoot to 50 - 300 kW/rack of consumption. While this can increase revenue per sq/ft tremendously in colocation data halls, it is also introducing challenges in cooling and power requirements. Liquid cooling, active rear door heat exchangers, and cold plates, are poised to address these challenges on the heat rejection side. However, the power requirements are an entirely different beast to deal with.  

Enter, the need to BYOP (Bring Your Own Power). This is a facility level strategy that is creating and managing your own distribution, generation, and energy asset deployment. This can be accomplished through a variety of solutions. Utilizing DERs (Distributed Energy Resources), which is a fancy terminology to describe the energy generating and storage assets that comprise a microgrid, facilities can manage peak demand, add layers of redundancy to their systems, and ultimately, completely island themselves from the grid.  

While a completely renewable and stand-alone data center is not happening in the next 1-2 years, it is just over the horizon, and it is critical to start having important conversations as these systems require large intellectual investment, planning, and capital to get them off the drawing boards and into the real world.  

While the matters mentioned above mainly concern data center providers, an energy intensive activity that more and more consumers are participating in, every day, is… Electric Vehicle (EV) charging. Subsequently, never have we seen before, parking garages and multifamily home developments requiring the addition of new transformers to support 1000 amp and above services. Super chargers and 220V standard EV chargers require a large amount of power to charge vehicles quickly. Understandably, this strains the utility provider, especially considering that most charging is occurring simultaneously. What this looks like is a large group of EV users who commute to work and charge during the day, and another other group of users who charge exclusively at home during the night. As adoption increases, these routinely popular charging times become more and more problematic for utility providers.  

So, as the US continues to push automakers to electrify their fleets, the demand on the grid and surrounding infrastructures cannot keep up. Critical equipment necessary to install these new services have lead times measured in years, while the cost to retrofit existing parking structures to support charging can add up quickly, pricing many providers out of the market.  

The need for more readily available power is here, and we are just barely knocking on the door of what is possible, as we will need to, as an adapted society, further expand upon the utilization of already existing technologies. And, as mentioned, a BESS and PV Farm separately will not achieve much, but the value lies in linking them together into a smart controllable system. As we continue to be creative with implementing these already existing solutions together, then we can iterate and create more efficient systems, which allows for more of a mainstream adoption across the industry. 

Looking Ahead 

Plain and simple, for most operations these solutions are currently cost prohibitive. However, let’s keep in mind a key learning from the ramp up of the solar industry; Utilities and governments are willing to subsidize and incentivize companies that choose to implement these solutions ahead of the curve. Currently, in Utah, Rocky Mountain Power (RMP) is rolling out an incentive program that is either per kWh or a one-time upfront incentive for the installation of a BESS. These are not small sums either, with some programs covering up to 75% of the cost of the BESS.  

One may ask, what is the angle for RMP? In short, the more DERs that are connected to the grid, the more redundancy is built into the utility framework. In the case of a contingency, these assets can all be controlled as one, spinning reserve for RMP. During normal operation, owners can enjoy peak shaving benefits, as well as outage protection. A truly rare “win-win” scenario. As peak demand charges continue to increase, ROI numbers start to make sense on 12- and 24-month timelines.  

Additionally, RMP is utilizing “Make-Ready” incentives to support the adoption and installation of EV charging. These incentives could cover up to 100% of the cost associated with powering EV chargers in commercial and residential applications. 

To further this discussion of the future, we can start to think of abstract solutions such as on-site hydrogen generation using natural gas. We can replace diesel gensets with hydrogen fuel cells, as hydrogen is three-times more energy dense/liter than diesel. We are even close to the deployment of small, self-contained, 300 – 500 MW nuclear reactors that can be deployed in remote environments and do not require service for 60 years.  

So, when it comes to reliability and cost savings, all signs point to BYOP. 

While the adoption of microgrid solutions may currently pose financial challenges, the tide is turning as incentives and awareness grow. Just as the solar industry witnessed exponential growth fueled by supportive policies, the trajectory of microgrids and BESS suggests a similar transformation in the energy landscape. As we stand on the cusp of this paradigm shift, it is necessary to initiate conversations and investments today for a more sustainable and resilient tomorrow. The journey towards decentralized, renewable energy is not merely an option; it's a strategic imperative for businesses and communities alike. 

If you enjoyed this high-level overview of the current market of microgrids, please join us for part two of this blog series, which will be released the last week of March. We'll do a deep dive on use/case and applications, and we’ll expand upon DVL’s current product offerings that support this infrastructure and qualify for utility incentives. Additionally, we will provide real-life applications to this equipment.

Have a question or comment about this blog?

Reach out to blog author Alexander "D'Angelo" D'Angelo, Power Systems Sales Engineer,  (based out of our Salt Lake City office) at ADangelo@DVLnet.com.

¹ https://www.generac.com/be-prepared/power-outages/top-5-states-where-power-outage-occur

As artificial intelligence (AI) continues to reshape industries and drive innovation, the demand for high-performance computing in data centers has reached unprecedented levels. Managing the cooling and power requirements of a 50kW rack density AI data center presents a unique set of challenges. In this blog post, we will explore effective strategies and cutting-edge solutions to ensure optimal performance and efficiency in such a demanding environment. 

Precision Cooling Systems

The heart of any high-density data center is its cooling system. For a 50kW rack density AI data center, precision cooling is non-negotiable. Invest in advanced cooling solutions such as in-row or overhead cooling units that can precisely target and remove heat generated by high-density servers. These systems offer greater control and efficiency compared to traditional perimeter cooling methods.

Liquid Cooling Technologies

Liquid cooling has emerged as a game-changer for high-density computing environments. Immersive liquid cooling systems or direct-to-chip solutions can effectively dissipate heat generated by AI processors, allowing for higher power densities without compromising on reliability. Explore liquid cooling options to optimize temperature control in your data center.

High-Efficiency Power Distribution

To meet the power demands of a 50kW rack density, efficient power distribution is paramount. Implementing high-voltage power distribution systems and exploring alternative power architectures, such as busway systems, can enhance energy efficiency and reduce power losses. This not only ensures reliability but also contributes to sustainability efforts.

Redundancy and Resilience

A high-density AI data center demands a robust power and cooling infrastructure with built-in redundancy. Incorporate N+1 or 2N redundancy models for both cooling and power systems to mitigate the impact of potential failures. Redundancy not only enhances reliability but also allows for maintenance without disrupting critical operations.

Dynamic Thermal Management

Utilize intelligent thermal management systems that adapt to the dynamic workload of AI applications. These systems can adjust cooling resources in real-time, ensuring that the infrastructure is optimized for varying loads. Dynamic thermal management contributes to energy efficiency by only using the necessary resources when and where they are needed.

Energy-Efficient Hardware

Opt for energy-efficient server hardware designed for high-density environments. AI-optimized processors often come with advanced power management features that can significantly reduce energy consumption. Choosing hardware that aligns with your data center's efficiency goals is a key factor in managing power and cooling requirements effectively.

Monitoring and Analytics

Implement comprehensive monitoring and analytics tools to gain insights into the performance of your AI data center. Real-time data on temperature, power consumption, and system health can help identify potential issues before they escalate. Proactive monitoring allows for predictive maintenance and ensures optimal conditions for your high-density racks.

Successfully cooling and powering a 50kW rack density AI data center requires a holistic and forward-thinking approach. By investing in precision cooling, liquid cooling technologies, high-efficiency power distribution, redundancy, dynamic thermal management, energy-efficient hardware, and robust monitoring tools, you can create a resilient and high-performing infrastructure. Embrace the technological advancements available in the market to not only meet the challenges posed by high-density AI computing, but to excel in this dynamic and transformative era of data center management.

Author's Note:

Not a bad blog post, right? I was tasked with writing a blog post on how to power and cool high density racks for AI applications. So, I had Chat GPT write my blog post in 15 seconds, saving me a ton of time and allowing me to enjoy watching my kid’s athletic events this weekend. As end users embrace AI technology, it is imperative that we understand how to support the hardware and software that enables us to achieve these time saving technologies. Over the past 6 months, about 20% of my time has been spent discussing how to support customer 35kW to 75kW rack densities.

Additionally, another key to understand, is the balance of AI and the end-user’s ability to recognize limitations and areas for improvement. AI taps into the database of information that is the Internet. Powerful, but it does so (at least currently) in a fashion that makes it appear to be two years behind. For example, this blog post was written to reflect a 35kW rack density, and subsequently, ChatGPT noted 35kW. However, today, I’m regularly working with racks supporting AI that average 50kW, and have seen go up to 75kW… and know that applications can hit upwards of 300kW per rack. So, please note, anywhere in the blog where it says 50kW, human intervention made these necessary edits to AI's outdated "35kW".

Also, just for reference, a 75kW application requires 21 tons of cooling for one IT rack! So, these new high-density technologies require the equivalent of one traditional perimeter CRAC to cool one AI IT Rack. DVL is here to help provide engineering and manufacturing support to design your Cooling, PLC Switchgear, Busway Distribution, Rack Power Distribution, Cloud Monitoring, and other critical infrastructure to support your efficient AI Technology.

Recently, liquid cooling has garnered significant attention in the data center landscape, yet its roots trace back to the 1980s in various forms. The recent surge of interest in this cooling method is largely due to the growth of artificial intelligence (AI) technologies, therefore introducing it to even MORE people, as they wonder what this “cool new thing” is. And, since the quest to increase your systems’ efficiency is never complete, it’s no wonder why liquid cooling has found its way into nearly all discussions about the future of data center operations. 

As mentioned, the emergence of AI-powered programs, like ChatGPT, have intensified the world's focus on AI across various sectors, from data science to university programs. According to a recent study by Accenture, 98% of company leaders are contemplating the potential impact of AI on their businesses. One inevitable consequence AI will bring to businesses, is the escalation of their data centers’ temperatures. Without adequate cooling mechanisms, these temperature increases can lead to equipment failure due to heat tolerance thresholds constantly being surpassed. This is where liquid cooling emerges as a vital solution. 

Liquid cooling offers superior thermophysical properties, making it more efficient for managing the extreme heat densities that are generated by IT racks supporting high-performance computing (HPC) applications. As data centers integrate liquid cooling solutions into their networks, several considerations come into play. 

One critical decision point is selecting the appropriate liquid cooling technology. Fortunately, cooling manufacturers have developed various methodologies, with immersion and direct-to-chip being the most popular. However, there is no universal guideline for determining which technology suits a specific setup.  

Each methodology presents unique advantages: direct-to-chip cooling offers better thermal resistance and easier maintenance, while immersion cooling tends to be simpler to manage and space-efficient, with greater flexibility across different hardware types. Both methodologies offer some opportunities for heat reuse and can be integrated with other infrastructure components to create hybrid solutions. For example, pairing with rear door heat exchangers for enhanced heat dissipation efficiency. 

If you decide to go the liquid cooling route, you’ll need to determine the most applicable liquid cooling methodology for your critical infrastructure’s needs. An important part of this process will be assessing the different methods’ impact on power consumption, ensuring that when implemented, you will have structural support for the additional weight (think: raised floors), and establishing a maintenance program to ensure smooth operations and a significantly prolonged lifespan. 

So, if and when you decide liquid cooling is the best path for your business’s needs, be sure to give careful consideration of these various factors. Most importantly, you’ll want to ensure optimal performance and longevity of infrastructure.  

To learn more on this topic, we encourage you to watch our recent webinar “The 2024 State of Liquid Cooling” in which guests Dr. Richard Bonner and Liz Cruz of Accelsius explore a more in-depth look at the world of liquid cooling. 

“Experience is simply the name we give our mistakes.” — Oscar Wilde

When you’ve worked in the deep trenches of critical infrastructure long enough, like quite a few of our longest tenured employee owners, you know that the most valuable lessons don’t come from a textbook or a policy manual. Rather, we learn the most, right in the field, from those real world mistakes and mishaps—or “experiences” as Oscar Wilde would say.

So, we asked our Associates about their most valuable gems of wisdom that they could pass on to colleagues in the industry. Here are some of the top answers we received. We're sharing them here, in hopes that we'll be able to save others from learning some unfortunate lessons the hard way.

• Spare parts are an on-site technician’s best friend.
• A single-point-of-failure is NOT an end user’s best friend.
• A hungry mouse can be disastrous to critical wires.
• The smallest of unchecked details can be the source of a project’s biggest (and most expensive) problem.
• Arc-flash is dangerous and not anything you ever want to see.
• Weekly exercises are the surest way to know your GenSet will work when needed.
• The UPS battery system is like a loaf of bread…you only get so many slices (or discharges)…and the bigger, and more often, the discharge, the quicker you’ll need a new loaf.

Let us know what you think of these nuggets. Have you had the unfortunate opportunity to learn any of these lessons on your own? Or do you have an important one to add to the list? Email us at Marketing@DVLnet.com. If we get enough responses, we’ll post a Part 2.