Energy and Earth Science Vol. 3, No. 2, 2020 www.scholink.org/ojs/index.php/ees ISSN 2578-1359 (Print) ISSN 2578-1367 (Online) 113 Original Paper Characterizing Opportunity Power in the California Independent System Operator (CAISO) in Years 2015-2017 Andrew A. Chien 1 1 Department of Computer Science, University of Chicago and Argonne National Laboratory, Chicago, USA Received: September 16, 2020 Accepted: October 2, 2020 Online Published: November 10, 2020 doi:10.22158/ees.v3n2p113 URL: http://dx.doi.org/10.22158/ees.v3n2p113 Abstract Studying the California Independent System Operator (CAISO) day-ahead and real-time markets for the period January 2015 to December 2017, we characterize the growth of curtailment and negative-priced power from renewable generators. Results show that renewable curtailment is growing rapidly, tripling to over 400 GWh from 2015-2018. Negative-priced renewable power is larger and also growing rapidly, reaching 1.5 TWh in 2017 for 40% CAGR. Resource-hours for negative pricing grew nearly 3-fold from 80,006 to 217,728 hours, with the highest single generator reaching 955 hours in 2017 or 33% of the daylight solar hours. Spatially, the quantity of negative-priced power is concentrated at a few dozen renewable generators, reaching peaks of 170GWh at the largest generator. We also consider an averaged-price model (NetPrice) that smooths over fluctuations to estimate the usable quantities of low-priced power. Results for NetPrice show a much larger quantity of low-priced power available than with either negative-pricing or curtailment alone. Overall, these results suggest both that opportunity power is a substantial and growing resource and a number of opportunities to exploit it. Keywords Renewable generation, power markets, curtailment, negative prices, power grid 1. Introduction Aggressive Renewable Portfolio Standards (RPS), and resulting rapid growth of variable renewable generation in power grids around the world aim to reduce the carbon-emissions and other negative environmental impacts of fossil-fuels. For example, California recently adopted even more aggressive goals of 60% renewable in 2030 and 100% carbon-free power by 2045 (Shoot, 2018). These ambitious goals are transforming the power generation and use landscape and pose serious power grid challenges www.scholink.org/ojs/index.php/ees Energy and Earth Science Vol. 3, No. 2, 2020 114 Published by SCHOLINK INC. including ability to achieve “merit order”, efficiency, stability, and resiliency. And of course, social welfare is an important consideration. Evidence of challenges include growing curtailment, negative-priced generation, and RPS stagnation. Curtailment occurs where the power grid is unable to accept renewable generation due to congestion, or excess generation, causes power to be discarded at the generation site Europe and the United States (Lew, 2013; Bird, 2013) and China (GWEC, 2016). And, despite programs to increase transmission capacity and employ economic dispatch, curtailed power in Europe, United States, and China exceeds 50 TWH per year (GWEC, 2016). Economic dispatch has successfully reduced curtailment, providing economic incentives (payments) and disincentives (negative payments) for generation. Renewables are increasing their participation in markets by bidding their opportunity costs, increasing the ability to manage and optimize supply and demand recognizing transmission constraints. However, one result is negative-priced power generation, power purchased by the power grid at a negative price (Gross, 2015; Reed, 2017). In this article we use the term Opportunity Power (OP) to describe both curtailed power and negative-priced generation as they together represent excess renewable power generation (Note 1). We use the term opportunity power to capture the notion that there is an opportunity if the negative pricing can be leveraged to enable productive use of the power. Recent studies examine not only increased curtailment, but also the lesser utility (capacity factor) of renewables as their fraction is increased (Wiser, 2017). Our prior studies characterized this opportunity power in the Midcontinent Independent System Operator (MISO) (Chien, 2018). This study analyzes opportunity power in the California Independent System Operator (CAISO) for the period January 2015 to December 2017 with the objective of characterizing the growth of curtailment and negative-priced generation to create insights that can inform strategies to both reduce their occurrence and to exploit the opportunity that this power provides for some productive use (Note 2). In the period of study, CAISO’s total renewable generation grew from 45 to 55 TWh (CAISO, 2018). Our prior studies characterized stranded power in the Midcontinent Independent System Operator (MISO) (Chien, 2018). We studied CAISO day-ahead and real-time market bids, market prices, dispatch prices, and production for over 375 renewable generation resources. Power is settled at 1 hour intervals in the day-ahead market, and dispatched at 15 and 5 minute intervals in the day-ahead and real-time markets respectively, so this data collection includes over 125 million records. We characterize negative pricing in the day-head and real-time markets and also include CAISO’s reported curtailment for renewable generation (initiated in 2016). We use this information to estimate negative-priced generation. Together these form an assessment of current opportunity power today and its trends of change. Findings include:  Negative-priced renewable power is significant and growing. Day-ahead Market (DAM) combined with estimated Real-Time Dispatch (RTD) show this at 1.5 TWh in 2017. Estimates based on DAM alone approach 1 TWh and RTD alone are 5 TWh in 2017. These quantities www.scholink.org/ojs/index.php/ees Energy and Earth Science Vol. 3, No. 2, 2020 115 Published by SCHOLINK INC. and growing at nearly 40% CAGR, and are similar to MISO which is 2.5x larger.  While curtailment is a smaller quantity, it also continues to increase rapidly, tripling to over 400 GWh from 2015 to 2018. And despite a pause in recent 2018 due to lower hydro generation, it is expected to resume its rapid growth in 2019 and beyond.  Resource-hours in CAISO for negative pricing are growing rapidly from 80,006 to over 217,728 resource-hours, and already cover all of the renewable generators. All resources are experiencing steady increase in number of hours with averages growing, a peak of 955 hours/year in 2017, a ~33% duty factor during solar hours, increasing to higher duty factors as RPS levels increase. Nearly all experienced >600 hours/year.  Negative-priced power is several times larger than curtailed power for renewable generation; 4x based on conservative estimation (DAM-RTD-scaled), more than 10x by our aggressive measure (RTD).  Spatially, the quantity negative-priced power is concentrated at a few dozen renewable generators. And, using RTD-LMP measures, reach significant quantity—170 GWh at a single generator, comparable to 19MW continuous or 47MW in sunlight hours.  The duty factor negative-pricing is more dispersed across resources, with a median close to the average grown to 576 hours/year in 2017. The peak of 955 hours represents a duty factor of 11% (24 hours) or 33% (for 10 hours).  We also consider average power price models (called NetPrice) that have been shown to smooth over market fluctuations, increasing realistic estimates of duration and quantity of negative-priced power (Chien, 2018). These price models have similar impact in CAISO, and show greater quantity of negative-priced power available. In summary, opportunity power, both negative-priced and curtailment is growing rapidly. Quantities are increasing at more than 40% per year, and duty factors rising across all renewable generators. These trends represent significant challenges for the power grid and perhaps opportunities for alternate usage. Further study in this area is a worthwhile endeavor. The remainder of the paper is organized as follows. We define opportunity power and methods for computing it in Section 2. Curtailment in CAISO is discussed in Section 3. In Section 4, we estimate negative-priced power, using three different models. In Section 5, we combine negative-priced power and curtailment to provide an overall opportunity power assessment. In Sections 6 and 7, we refine this view, using the real-time market data to examine spatial and temporal distribution of negative-priced power. In Section 8, we consider usage-oriented models that look at average power pricing, exploring how they would impact the usability of the resource. In Section 9, we discuss our results and compare to related work. Section 10 summarizes the results of the study and suggests directions for future work. Finally, in the Appendix (Section 11), we present several scatter plots illustrating the statistical temporal properties of opportunity power. www.scholink.org/ojs/index.php/ees Energy and Earth Science Vol. 3, No. 2, 2020 116 Published by SCHOLINK INC. 2. Definition of Opportunity Power We define opportunity power as two components: 1) curtailment, the power a generator would like to generate and transmit into the power grid, but ultimately does not, and 2) negative-priced power, that which is generated and transmitted into the grid, but at a market price that is negative. That is, in effect, the generator is paying the load to take the power. 2.1 Curtailed Renewable Generation The methodology of (CAISO, 2018) computes renewable generation as the difference between forecast and actual generation for each dispatch interval. This data is publicly reported at (CAISO, 2018), and is available only at the level of the entire CAISO region. We plot and analyze curtailed renewable generation from several perspectives in Section 3. 2.2 Negative-Priced Power CAISO employs an intricate set of markets and dispatch mechanisms to ensure reliable power to a diverse set of loads, dispatching generation and scheduling transmission. We use several methods to estimate negative-priced power. 1) DAM-LMP: Using the Day-Ahead Market (DAM) hourly settlements, we characterize the quantity of negative-priced power. Because the DAM accounts for only a portion of the power transacted by CAISO, and DAM negative pricing is less frequent than RTD, this is a conservative estimate. 2) RTD-LMP, RTD-NP: Using the real-time market (RTM or RTD) 5-minute settlements, at the resource level, for RTD-LMP, we sum the total power produced when there is a negative price (RTD price) for the interval. For RTD-NP, again at the resource level, we sum the total power produced as long as the average price for that interval remains negative. We consider NP0 and NP5 for average price thresholds of $0/MWh and $5/MWh respectively. This is an aggressive model and likely an overestimate (Note 3), but matches the methodology in (Chien, 2018), allowing direct comparison to MISO. 3) DAM-RTD-Scaled: Combine the DAM-LMP with an estimate of the additional negative-priced power dispatched in the real-time market, using the fraction of the power covered by RTD but not DAM. This model uses the assumption that the fraction of negative-priced power in the RTD is the same as that in the DAM, perhaps a conservative projection. In the above three models, we use two underlying definitions of negative-priced. The first is instantaneous (single interval) negative pricing. We call this model LMPx or locational marginal pricing with threshold of $x. Analyzing detailed market and dispatch data, we can compute both CAISO aggregate negative-priced power at the www.scholink.org/ojs/index.php/ees Energy and Earth Science Vol. 3, No. 2, 2020 117 Published by SCHOLINK INC. resolution of dispatch intervals, and also per anonymized market participant. The second is an average price model for negative-priced power. We call this model NetPriced, because it computes intervals for which the average price of power is less than d. We studied three years of CAISO renewable bids, dispatch prices, and production for over 375 renewable generation resources for a three-year period. We consider negative pricing in both the day-ahead and real-time markets, calculating the transacted power for each generator. As discussed, the definition of negative-priced power varies with our LMP and NP models. In CAISO, power is dispatched at 1-hour intervals in the day-ahead and 5 minute intervals in the real-time market, to this data collection includes over 125 million records. 3. Curtailed Renewable Power 3.1 System-wide Seasonal Characterization Renewable generation bid into the market is sometimes not dispatched; it is curtailed. This is another element opportunity power beyond the negative-priced power. As shown in Table 1, the quantity of renewable power curtailed has grown significantly from 2015 to 2017. This power represents a small portion of renewable generation, and an even smaller portion of the overall CAISO generation. However, at 402 Gigawatt-hours it represents over $12M of power (at $30/MWh), so its loss is a significant cost. We report aggregate renewable numbers, but it is worth noting that renewable curtailment in CAISO is dominated by solar, at a 4.5:1 ratio to wind generation. Table 1. CAISO System-wide Renewable Curtailment, 2015-2017 Year Curtailed Renewable Power (Wind + Solar) Increase YoY 2015 187,770 MWh 2016 308,423 MWh 64% YoY 2017 401,972 MWh 30% YoY 2018 (thru July) 317,213 MWh N.A. www.scholink.org/ojs/index.php/ees Energy and Earth Science Vol. 3, No. 2, 2020 118 Published by SCHOLINK INC. Looking at seasonal variation in curtailment (see Table 2), the largest quantity occurs in the winter (January to March) and the spring (April to June), and that quantity is has grown at an annual rate exceeding an average of 50% CAGR from 2015 to 2018 (tripling). Reasons for this might include greater mandatory hydropower in the winter and spring. Curtailment in summer is much smaller, and while larger in the fall, it is fluctuating. In 2018, this growth has slowed due to decreased hydro generation, but it expected to resume its rapid growth in 2019 and beyond. Table 2. CAISO System-wide Renewable Curtailment, 2015-2017 Year Winter (MWh) Increase YoY Spring (MWh) Increase YoY Summer (MWh) Increase YoY Fall (MWh) Increase YoY 2015 47,024 n.a. 84,453 n.a. 21,496 n.a. 34,797 2016 74,885 59% YoY 92,597 2% YoY 38,899 81% YoY 102,043 194% YoY 2017 185,116 147% YoY 142,855 54% YoY 31,329 -20% YoY 42,672 -59% YoY 2018 145,191 -22% YoY 163,309 14% YoY n.a. n.a. 3.2 CAISO Daily and Monthly Reporting CAISO provides a nice graphical summary of monthly renewable curtailment on is “Managing Oversupply” web page. We reproduce the June 2018 version of this graph in Figure 1. Of course, its captures similar trends to those we have summarized above. Daily reporting of curtailed renewables is also available on this site (CAISO, 2018). Figure 1. CAISO Renewable Curtailment by Month, January 2016 thru July 2018 www.scholink.org/ojs/index.php/ees Energy and Earth Science Vol. 3, No. 2, 2020 119 Published by SCHOLINK INC. 4. Negative-Priced Power We first consider the aggregate occurrence of negative-priced power in the day-ahead and real-time markets for renewable generation, system-wide. Subsequently, using the real-time market data, we characterize seasonal variations. 4.1 System-wide Negative-Priced Power As shown in Figure 2, the quantity of negative-priced power in both the day-ahead and real-time markets has grown rapidly in the period. A significant quantity of power is transacted at negative prices in the both the day-ahead and the real-time markets, indicating a growing quantity of excess renewable power. The DAM negative priced power is nearly 1TWh for 2017. The DAM-RTD-Scaled estimates of negative-priced power are significantly larger, as they reflect both the growing DAM negative priced power, and a scaled portion of the RTD negative priced power. Finally, the RTD-LMP negative-priced power is the largest quantity and growing rapidly. This quantity of 2.5 to over 5 terawatt-hours overestimates the power transacted at negative prices (the DAM prices the majority of the power), but does reflect the net balance at real-time dispatch, and can be directly compared to studies in other ISO’s. Figure 2. Estimates for Negative-priced Power from Day-ahead, Real-time, and a Scaled Estimate, CAISO, 2015-017 Looking at Table 3, we see that all three estimates of negative-priced power have large CAGR’s: >300% for DAM-LMP, 49% and 102% for DAM-RTD-scaled, and 35% and 54% for RTD-LMP. All of these measures are growing fast, and at terawatt-hours, these are large quantities of power. The RTD-LMP negative-priced power represents a significant fraction of total renewable generation in the CAISO region and is now a magnitude that is comparable to the negative-priced power documented in the MISO region (6 TWh in 2013 increasing to over 7 TWh in 2016 (Chien, 2018)). The growth www.scholink.org/ojs/index.php/ees Energy and Earth Science Vol. 3, No. 2, 2020 120 Published by SCHOLINK INC. observed from 2015-17, if extrapolated, suggests a doubling period of 2 years that if realized would produce in excess of 10 TWh of negative-priced power in the real-time market. This quantity would be far larger than that occurring in MISO, and larger as a fraction of total power delivered by more than 2.5x. Table 3. CAISO System-wide Negative-priced Renewable Generation, 2015-2017 Year DAM-L MP (MWh) DAM-RT D-Scaled (MWh) YoY RTD-LMP (MWh) YoY Total Renewables DAM (MWh) Total Renewables RTD (MWh) Fraction Total Renewables 2015 48,655 539,271 n.a. 2,512,571 n.a. 20,377,783 25,322,336 9.9% 2016 213,952 801,911 49% 3,879,929 54% 26,770,145 31,551,400 12.3% 2017 945,791 1,626,848 102% 5,263,853 35% 32,686,789 37,544,424 14.0% 4.2 Real-time Dispatch (RTD-LMP) Seasonal Characterization Study of negative-priced generation using RTD-LMP across seasons is shown Table 4 and Figure 3. The largest quantities occur in the winter (January thru March) and spring (April thru June) seasons, with those seasons accounting for 44% and 37% of the annual quantities respectively. Increase in these seasons has been rapid and continued, but fluctuating, perhaps due to changes in mandatory generation due to runoff requirements in hydroelectric generation. However, while those fluctuations may mask the trend, they present no mechanistic argument for changing it. The winter 2017 negative-priced generation corresponds to 25.5 GWh/day, a 24-hour rate of 1.06 GW, but if over the solar generation hours, closer to 2.6 GW. In Summer and Fall, negative-priced generation is much smaller, due to higher loads, but still significant, 5.5 GWh/day, with corresponding 24-hour rates of 0.23 GW, and solar generation hours of 0.55 GW. Table 4. CAISO System-wide Negative-priced Renewable Generation (RTD-LMP), by Season, 2015-2017 Year Winter (MWh) Increase YoY Spring (MWh) Increase YoY Summer (MWh) Increase YoY Fall (MWh) Increase YoY 2015 656,484 n.a. 1,294,340 n.a. 224,489 n.a. 336,677 2016 962,276 47% YoY 1,273,677 (-2%) YoY 468,794 109% YoY 975,180 190% YoY 2017 2,316,267 140% YoY 1,964,827 54% YoY 535,722 14% YoY 447,037 -55% YoY www.scholink.org/ojs/index.php/ees Energy and Earth Science Vol. 3, No. 2, 2020 121 Published by SCHOLINK INC. Figure 3. Seasonal Growth of Negative-priced Power (RTD-LMP), CAISO 2015-2017 4.3 Daily Variation of CAISO System-wide Negative-priced Renewable Generation Aggregate statistics cannot capture the variation in generation and load that create negative-priced power in power markets. Such variation is a principal challenge in efforts to reduce negative-priced generation through better planning and management, as well as to efforts that seek to exploit it as a resource (Kim, 2017),(Yang, 2016), (Yang, 2017). We present temporal graphs of the daily quantities of negative-priced generation in the MISO region and basic statistics that highlight its variability. Figures 4, 5, and 6 plot magnitudes of Negative-priced power occurrences, on an irregular x-axis scale of 5-minute intervals (bears no fixed relationship to time as the many 0-value data points have been omitted). And while the overall quantity of negative-priced power has grown YoY, there is a striking drop in the magnitude of events from 2015 to 2016. Presumably, this reflects some improvements in the markets (forecasting, dispatch, etc.). All three of the figures show that negative-priced power varies widely in quantity and seasonally. In all three years, the January-June period accounts for significantly more than half of the x-axis, and the smaller magnitude of events in the summer is striking—though a significant number of events are still occurring. Likewise the fall seems to have smaller magnitude and fewer occurrences of negative-priced generation. www.scholink.org/ojs/index.php/ees Energy and Earth Science Vol. 3, No. 2, 2020 122 Published by SCHOLINK INC. Figure 4. Daily Negative-priced Power (MWh) in the CAISO Real-time Market, 2015 Figure 5. Daily Negative-priced Power (MWh) in the CAISO Real-time Market, 2016 Figure 6. Daily Negative-priced Power (MWh) in the CAISO Real-time Market, 2017 www.scholink.org/ojs/index.php/ees Energy and Earth Science Vol. 3, No. 2, 2020 123 Published by SCHOLINK INC. 5. Opportunity Power (Curtailed and Negative-Priced) Opportunity power comprises two components—curtailed and negative-priced generation. We have examined both, and here we combine them to gain perspective for relative impact. As shown in Figures 7 and 8 as well as Table 5, both curtailment and negative priced power are growing rapidly within CAISO. In Figure 7 shows that RTD-LMP is far larger than curtailment (13x), but is perhaps an overestimate of negative priced power. However, DAM-RTD-Scaled is a more conservative estimate of negative priced power, and is still 3-4x larger than curtailment. This is also consistent with studies of the MISO real-time markets (Chien, 2018). Figure 7. Opportunity Power=Curtailment+Negative-priced for RTD-LMP, CAISO, 2015-2017 Figure 8. Opportunity Power=Curtailment+Negative-priced for the DAM-RTD-Scaled Model, CAISO, 2015-2017 www.scholink.org/ojs/index.php/ees Energy and Earth Science Vol. 3, No. 2, 2020 124 Published by SCHOLINK INC. In Table 5, we summarize several estimates of opportunity power, and compare them to curtailment data. Note that all are growing rapidly, and the ratios for DAM-RTD-Scaled/curtailed have recently increased significantly. The RTD-LMP/Curtailed ratio is remarkably stable. With the rapid growth all are large quantities of power, with 2017 DAM-RTD-Scaled reaching more than 1.6 Terawatt-hours. Table 5. Opportunity Power by Type, CAISO, 2015-2017 Year DAM-RTD -Scaled (MWh) Increase YoY DAM-RTD- Scaled/ Curtailed RTD-LM P (MWh) Increase YoY Curtailment (MWh) Increase YoY RTD-LMP /Curtailed 2015 539,271 n.a. 2.9x 2,512,571 n.a. 187,770 13.4x 2016 801,911 49% 2.6x 3,879,929 54% YoY 308,423 64% 12.6x 2017 1,626,848 102% 4.0x 5,263,853 35% YoY 402,972 30% 13.1x 6. Opportunity Power Spatial Dispersion (across Resources) 6.1 Per Resource Negative-priced Generation We further analyze the RTD-LMP data to characterize the experience of individual generators, computing basic summary statistics as shown in Table 6. They shows that the rapid growth in CAISO system negative-priced generation has already spread across all of the renewable generators, and doubling in quantity from 2015 to 2017. Further, the variation across resources is large—a large multiple of the average. This is reflected in quantity of negative-priced generation at the highest power generator increasing steadily from 130 GWh to 168 GWh. Note that this amount of power corresponds to 19MW (24-hour basis) or 46MW (10-hour basis) of generation. Table 6. Per-resource Annual Negative-priced Renewable Generation (RTD-LMP), 2015-2017 Year RTD-LMP (GWh) Resources Average Negative-priced (GWh) Standard Deviation (GWh) Maximum Negative-priced @ 1 Resource (GWh) 2015 2,513 GWh 367 7 17.7 131 2016 3,880 GWh 313 12 24.5 170 2017 5,264 GWh 378 14 26.1 168 6.2 Distribution across Resources of Negative-priced generation We examine the detailed distribution of negative-priced generation across resources to explore the magnitude of the greatest quantities of opportunity power available at a single physical location, but also to explore differences in the allocation of negative prices by the market. www.scholink.org/ojs/index.php/ees Energy and Earth Science Vol. 3, No. 2, 2020 125 Published by SCHOLINK INC. Figure 9. Annual Quantity of Negative-priced Power per Resource, CAISO 2015 (Sorted, Largest First) We consider the year 2015 data, presenting the per-site total negative-priced power at renewable resources in Figure 9. Note that the maximum negative-priced power at one site was 131 GWh ($3.9M power @ $30/MWh), and all of the resources experienced negative pricing (except one!). If we compare to the average negative-priced power for 2015, 6.9 GWh/resource, we see that 68 resources exceed this average out of a total of 367. Even with a large standard deviation of 17.7GWh, we see that the maximum is 7 standard deviations above the mean. And a collection of resources are well above the mean, with the Top 10 accounting for 930 GWh ($27.9M). The Top 20 account for 1,427 GWh ($42.8M). The Top 30 account for 1,704 GWh ($51.1M). The Top 50 account for 2,064 GWh ($61.9M), or over 80% of the opportunity power (2,512 GWh overall). Figure 10. Annual Quantity of Negative-priced Power per Resource, CAISO 2016 (Sorted, Largest First) www.scholink.org/ojs/index.php/ees Energy and Earth Science Vol. 3, No. 2, 2020 126 Published by SCHOLINK INC. We consider the year 2016 data, presenting the per-site total negative-priced power at renewable resources in Figure 10. Note that the maximum negative-priced power at one site was 171 GWh ($5.1M power @ $30/MWh) and all of the resources experienced negative pricing. If we compare to the average negative-priced power for 2016, 12 GWh/resource, we see that approximately 68 resources exceed this average out of a total of 313. Even with a large standard deviation of 24.4 GWh, we see that the maximum is 6.5 standard deviations above the mean. And a collection of resources are well above the mean, with the Top 10 accounting for 1,170 GWh ($35.1M). The Top 20 account for 1,892 GWh ($56.7M). The Top 30 account for 2,294 GWh ($68.8M). The Top 50 account for 2,840 GWh ($85.2M), or over 73% of the opportunity power (3,880 GWh overall). Figure 11. Annual Quantity of Negative-priced Power per Resource, CAISO 2017 (Sorted, Largest First) We consider the year 2017 data, presenting the per-site total negative-priced power at renewable resources in Figure 11. The picture is similar to 2015 and 2016, but slightly different in degree. Note that the maximum negative-priced power at one site was 168 GWh ($5.1M power @ $30/MWh), about the same as 2016. And, that again, all of the resources experienced negative pricing. If we compare to the average negative-priced power for 2017, 14 GWh/resource, we see that approximately 85 resources exceed this average out of a total of 378. Even with a large standard deviation of 26 GWh, we see that the maximum is 5.9 standard deviations above the mean. And a collection of resources are well above the mean, with the Top 10 accounting for 1,307 GWh ($39.2M). The Top 20 accounts for 2,181 GWh ($65.4M). The Top 30 accounts for 2,723 GWh ($81.7M). The Top 50 accounts for 3,389 GWh ($101.7M), or over 64% of the opportunity power (5,264 GWh overall). www.scholink.org/ojs/index.php/ees Energy and Earth Science Vol. 3, No. 2, 2020 127 Published by SCHOLINK INC. 6.3 Year to Year Changes (per-resource) Looking at trends from 2015 to 2017, we see a dramatic increase not only in the total quantity of negative-priced power but also in its breadth of impact (resources participating). Further we see that as the overall quantify of negative-priced power increases from year to year, the disparities become large for a growing number of resources (see Table 7). If we look at the top-ranked resources by quantity of negative-priced power, we see significant growth in all of the top groups. We compute the average daily negative-priced generation per resource by simple dividing the total negative-priced power in the group by 365 days and by the number of resources in the group. All of these show steady growth and significant quantities of power. Note that if these renewables are solar, their generation will be concentrated in the daylight hours, so the effective daylight negative-priced generation will be 2 or even 3 times larger (e.g., >150MW rate per site for Top 10 in 2017). Table 7. Negative-priced Power at the Top 10, 30, and 50 Resources in CAISO Year Top 10 Total (MWh) Daily Average @ Resource (MWh) Top 30 Total (MWh) Top 30 Daily Average (MWh) Top 50 Total (MWh) Top 50 Daily Average (MWh) 2015 930,000 255 1,704,000 155 2,064,000 113 2016 1,170,000 321 2,294,000 209 2,840,000 156 2017 5,264,000 1,442 2,723,000 249 3,389,000 186 7. Opportunity Power Temporal Dispersion The quantity of negative-priced power is important, but its temporal distribution (all in one hour vs. spread evenly) is critical for its utility for loads. We further analyze the RTD-LMP data to characterize the experience of individual generators, “duty factor” of negative-priced power at individual generators. That is, the number of hours that each site is experiencing negative-priced dispatch, a useful measure to understand if a physically localized load could exploit the negative-priced power. We then normalize to a 24-hour window (to compare against reliable power), and also to a 10-hour window (to compare against solar generation). Table 8. Temporal Distribution of Negative-priced Power Occurrence at Generators Year Total Negative Priced Generator-Hours Resources Average Negative-priced Hours / Resource Std Deviation of Negative-priced Hours / Resource Maximum Negative-priced Hours / resource 2015 80,006 hours 367 218 hours 211 hours 604 hours 2016 148,675 hours 313 475 hours 183 hours 883 hours 2017 217,728 hours 378 576 hours 157 hours 955 hours www.scholink.org/ojs/index.php/ees Energy and Earth Science Vol. 3, No. 2, 2020 128 Published by SCHOLINK INC. The data shows that the number of negative priced hours is growing rapidly, over 2.5-fold from 2015 to 2017. There is a large variation across resources, as shown by the large standard deviation, and the maximum hours is only 2.5x standard deviations above the mean. This is a much lower variation than we saw with quantities of negative-priced power; this difference is presumably due to different peak generation capacities. For the sites with maximum hours, the numbers are large enough to correspond to a significant duty factor, particularly for a variable renewable generator. For example, the 2015 number, 604 hours, corresponds to duty factor of 7% (vs. 24 hours), and 21% (vs solar hours). The maximum for 2016, 883 hours, corresponds to duty factor of 10% (vs. 24 hours), and 30% (vs solar hours). And finally, the maximum for 2017, 955 hours, corresponds to a duty factor of 11% (vs. 24 hours) and 33% (vs. solar hours). Obviously, a generator that is receiving negative prices for 33% of its operating hours, even if it only applies to a fraction of its power, is not likely to be happy about it. On the positive side, if an opportunistic load had the goal of exploitation of the negative-priced power, 33% x 10 hours yields an availability of about 3.3 hours per day—conceivably the duty factor of an unreliable, emerging region grid. In Figures 12, 13, and 14, we examine the distributions of negative-priced power hours across renewable generation resources. We first consider 2015 data, presenting the per-site total negative-priced power hours at renewable resources in Figure 12. Note that the maximum negative-priced hours at one site was 604 hours. About two-thirds of the renewable resources experienced negative pricing. If we compare to the average negative-priced power hours for 2015, 218 hours, we see that approximately 155 resources exceed this average out of a total of 367 resources; not far from one-half the resources, so the negative-priced hours are reasonably spread out. Figure 12. Negative-priced Power Hours by Resource, CAISO 2015 www.scholink.org/ojs/index.php/ees Energy and Earth Science Vol. 3, No. 2, 2020 129 Published by SCHOLINK INC. We next consider the year 2016 data, presenting the per-site total negative-priced power hours at renewable resources in Figure 13. Note that the maximum negative-priced hours at one site increased to 883 hours. 6 resources experienced >800 hours of negative pricing. The number of resources experiencing negative pricing increased, with nearly all of the renewable generators included. If we compare to the average negative-priced power hours for 2016, 475 hours, we see that approximately 198 resources exceed this average out of a total of 313 resources; a large fraction (63%). So compared to 2015, the negative-priced hours are spread over a much larger group of resources, and falling much harder (more hours). However, there is a notable spike in the maximum number of negative-priced hours with six resource suffering nearly 50% more than peers. It would be interesting to understand the market dynamic that leads to this disparity. Figure 13. Negative-priced Power Hours by Resource, CAISO 2016 We next consider 2017 data, presenting the per-site total negative-priced power hours at renewable resources in Figure 14. Note that the maximum negative-priced hours at one site increased to 955 hours. At the high-end, now 6 resources experienced >900 hours of negative pricing, and the number experiencing >800 hours has grown from 6 in 2016, to 17 in 2017. Amongst this group, there is some consistency, with many of the same sites ranking high from year to year. The number of resources experiencing negative pricing increased, with nearly all of the renewable generators included. If we compare to the average negative-priced power hours for 2016, 576 hours, we see that around 250 resources exceed this average out of a total of 378 resources; a large fraction (66%). So, the negative-priced hours are being spread over a much larger group of resources, and much more hours. However, there is a notable spike in the maximum number of negative-priced hours with six resource suffering nearly 50% more than peers. It would be interesting to understand the market dynamic that leads to this disparity. Compared to 2016, the notable spike at the highest levels of negative-priced www.scholink.org/ojs/index.php/ees Energy and Earth Science Vol. 3, No. 2, 2020 130 Published by SCHOLINK INC. hours is less pronounced, but a small group of ~15 generators are suffering significantly higher numbers of negative-priced hours. Figure 14. Negative-priced Power Hours by Resource, CAISO 2017 Looking across the period from 2015 to 2017, we see a steady increase in the number of negative-priced hours, as the phenomenon grows. However, in contrast to the quantity of negative-priced power, the hours of negative priced power are spread more evenly across resources. This suggests some degree of shared fate—that perhaps the increases are defined by greater durations where there is widespread negative-pricing, rather than spatially localized negative pricing. Confirming this with further study should be possible. 8. Usage-oriented Characterization (Opportunity Power Models) 8.1 Quantity Versus Opportunity Power Model Market pricing mechanisms can be prone to short-term price fluctuations as documented in (Chien, 2018) with little connection to the longer term economic performance of a generator resource or long-term price of power to a load. To explore the magnitude of this effect, we apply a net price model of power cost to the RTD data, smoothing local price fluctuations by computing weighted average price of power as defined in Section 2. Here we compare RTD-NP0 and RTD-NP5 “average price” studies to our RTD-LMP studies based on instantaneous negative prices (LMP0 model) used in prior sections. www.scholink.org/ojs/index.php/ees Energy and Earth Science Vol. 3, No. 2, 2020 131 Published by SCHOLINK INC. Table 9. Model Sensitivity Study, Negative-priced Power Model, 2015-2017 (RTD with Various) Year Opportunity power Model Total Opportunity power (MWh) Intervals Longest Interval (hours) Average Length (hours) St Dev (hours) Total Hours Interval Average Power (MWh) Interval Stdev Power (MWh) 2015 LMP0 2,512,571 2015 NP0 3,816,788 395,917 311 0.31 2.57 124,180 9.64 107 2015 NP5 4,663,958 426,667 622 0.37 3.57 157,921 10.93 142 2016 LMP0 3,879,929 2016 NP0 6,018,084 362,168 165 0.64 2.4 231,384 16.6 115 2016 NP5 8,178,759 473,059 169 0.68 3.2 321,486 17.3 129 2017 LMP0 5,263,853 2017 NP0 7,796,664 501,693 557 0.64 2.8 320,600 15.54 99.06 2017 NP5 7,941,630 706,170 557 0.64 2.6 319,698 15.83 106.49 8.2 Interval Statistics vs Opportunity power Model The quantity and duration of opportunity power in CAISO exhibits sensitivity to the pricing/acquisition model used. For example, the increase in opportunity power from LMP0 to NP0 varies from 50% to over 100% (see Table 8). However, the increase for NP0 to NP5 is varied, producing a smaller increase from ~0% to 35%. The maximum interval durations for opportunity power are surprisingly long! The maximum duration for a continuous interval exceeds the daylight hours for a single day, and presumably arises from wind generation sites. Changing the pricing model causes the average length of intervals to increase, but are still short as there are many short intervals. However, the growing standard deviation indicates that there are also an increasing number of longer intervals. At the far right of Table 8, we present average power per interval and the standard deviation of the power. These show significant quantity and increasing standard deviation, but not much increase in average power. This suggests that only a small number of the intervals are being substantially augmented by the more aggressive pricing models. The conclusions drawn here for opportunity power in CAISO in 2017 are www.scholink.org/ojs/index.php/ees Energy and Earth Science Vol. 3, No. 2, 2020 132 Published by SCHOLINK INC. qualitatively similar to those in MISO, where NP0 yielded 30% more opportunity power over LMP0, and NP5 yielded 15-20% more power than NP0 over a multi-year period (Chien, 2018). 9. Discussion and Related Work The period of analysis, 2015-2017 encompasses extraordinary changes in the CAISO balancing area. In the three-year period from 2015 to 2017, total renewable generation grew from 45 to 55 TWh (CAISO, 2018). While that is a significant increase of 22% over two years, it is clear that quantity of curtailed and negative-priced power, by all of our measures conservative to aggressive (DAM-LMP, RTD-LMP, and DAM-RTD-Scaled) is growing much faster—in excess of 50% per year. Amongst these, the moderate estimate, DAM-RTD-scaled is over 1.6 TWh in 2017, and projected to exceed 2 TWh in 2018. The RTD-LMP measure is already proportionately larger than that seen in MISO, and exceeds 5 TWh in 2017. These number represent 10’s of millions to 100’s of millions of dollars of power at average CAISO prices. As the quantity increases, greater duty factors are observed with a number of sites seeing large numbers of “full day” intervals of 15.5 hours of negative pricing. Section 6 shows that the distribution of the quantity of opportunity power is extremely uneven across the opportunity power generators—concentrated on a small number of sites. This information should be analyzed with respect to generator capacities to see how much of this extreme imbalance can be accounted for by this, or should be attributed to other factors such as transmission congestion or bidding behavior of renewable generators. Regardless, the concentration of opportunity power increases its potential for productive used if a consuming load can be geographically localized to where the opportunity power is occurring. Alternatively, the concentration may suggest that an increase in transmission resources in the area is appropriate. Section 7 shows a steady shift of the entire distribution of renewable generators to higher and higher duty factors of negative pricing. In 2017, the majority of renewable generators saw >600 hours of negative pricing—the equivalent of nearly two full months of 12-hour negative pricing days. This suggests that opportunity power could be useful resource for intermittent, but power-intensive activities such as fertilizer manufacture, water desalination, or perhaps bitcoin mining. A more challenging, but promising area that we are pursuing is to exploit opportunity power for networks of hyperscale cloud data centers—a rapidly growing user of electric power (Kim, 2017; Yang, 2016; Yang, 2017; Chien, 2018). Opportunity power has been documented as a large, growing phenomenon in studies of curtailment in the Mid-continent Independent System Operator (MISO) where 2.2 terawatt-hours (TWh), and 5.5 TWh negative priced power for a total of 7.7 TWh of opportunity power from wind resources (Bird, 2013; Lew, 2013; Yang, 2016). And in China, curtailed wind power has grown to 34 TWh in 2015 (GWEC, 2016). Around the world, as renewable generation fraction increases due to rising RPS standards, opportunity power is projected to increase significantly in both wind-heavy (Kim, 2017, www.scholink.org/ojs/index.php/ees Energy and Earth Science Vol. 3, No. 2, 2020 133 Published by SCHOLINK INC. MISO, 2018) and solar-heavy (Denholm, 2016a, 2016b, E3) renewable power grids. Notably, a recent Department of Energy summary reports documents this phenomena as a reduced capacity factor for renewables as RPS above 15% to 25% (Wiser, 2017). All of these studies suggest that opportunity power is of increasing quantity, making its study of increasing importance. Studies that characterize opportunity power (curtailed and negative priced power) availability at a fine-grained temporal level are rare. Chien and colleagues studied two and a half years of opportunity power in the Midcontinent Independent System Operator (Chien, 2018), documenting quantities, spatial and temporal distributions, and the potential impact and limitations of adding storage to increase utility of opportunity power. The MISO grid has significant renewables that are predominantly wind turbines, and much of it dispersed widely in unpopulated areas, and thus often transmission constrained. Further, MISO generally operates at a renewable portfolio fraction below 15%, and this fraction has changed only slowly over the past few years. This situation differs significantly from CAISO which has predominately solar renewables, an RPS exceeding 33%, and is less transmission limited. While both the MISO study and this report document that opportunity power is available in >5TWh quantities on an annual basis, the relative magnitude of opportunity power in CAISO is much greater—given its smaller size. Both ISO’s exhibit stable geographic patterns of concentration of opportunity power, both conferring a disproportionate economic impact on certain generators, but also creating a geographically localized opportunity for exploitation. The occurrence of opportunity power in the two ISO’s differs in its temporal structure as well. Because of the large quantities of solar power in CAISO (generation largely synchronized by sunlight hours), the occurrence of opportunity power is heavily concentrated in those daylight hours, as well as in certain seasons. In contrast, MISO’s wind generation is seasonal, but the occurrence of opportunity power tends to occur at night, when load is lower. Progress in energy storage is promising, but the low price of power makes its large-scale deployment for time-shifting challenging and some studies suggest on energy return-on-investment (EROI) criteria, it may never make sense to store large quantities of wind power, though solar’s higher energy quotient makes it more attractive. Grid-scale storage for time-shifting, rather than small quantities for peak shaving and regulation, faces significant economic challenges. CAISO is a leader in this area, with a plan to deploy 1.3 GW of storage by 2024, but even this is a small fraction of the storage capacity needed to time-shift power. In fact, recent studies show that much larger quantities of storage had little net impact on opportunity power duty (Chien, 2018). 10. Summary and Future Work Detailed analysis of CAISO’s day-ahead and real-time markets focused on renewables shows the rapidly changing dynamics as the fraction of renewables grows to 30% and beyond. Highlights include terawatt-hours of opportunity power, dominated by negative-priced power, not curtailment. The number of hours that negative pricing occurs—at all resources—is growing rapidly; however, the www.scholink.org/ojs/index.php/ees Energy and Earth Science Vol. 3, No. 2, 2020 134 Published by SCHOLINK INC. quantity of opportunity power is concentrated at a few dozen generators. In 2017, nearly all of the renewable generators are experiencing close to 600 hours or more of negative pricing per year. Finally, flexible power acquisition models such as “NetPrice” significantly increase the duty factor of opportunity power, but not as much in CAISO’s solar-dominated renewables grid, when compared to MISO’s wind-dominated renewables grid. The rise of negative-priced power in both markets, and the additional growth of renewable curtailment in every dimension—quantity, duty factor, fraction of generators impacted, etc. reflect significant challenges for renewable integration at high RPS levels. The result is a rising tide of opportunity power. This opportunity power is a new and growing opportunity. It creates opportunities for new uses of power that can act as “compliant loads”, adaptable in time and consumption level to availability of this low cost and environmentally friendly power resource. Integrating such loads into current power markets requires new innovation. While we have studied three years of CAISO data, spanning a recent period of rapid change, this change is expected to continue. So study of future data, 2018 and beyond, to explore new trends, and to assess if the trends we have identified persist. To date, there are detailed published studies on opportunity power in MISO and CAISO (this one), and we would hope to see such studies for further ISO’s, particularly those with large fractions of renewables. One area we were not able to explore is the impact of new market products for storage and distributed energy resources. Those products are being designed to affect metrics used in this study, and others. Study of these impacts as renewable generation continues to grow would doubtless yield valuable insights. The growing quantity of Opportunity Power (OP) poses the natural question—how can it be exploited? We have been exploring the possibility of using it to power cloud data centers, the fastest growing source of electric power consumption in the developed world (Yang, 2016, 2017). By adapting the magnitude of the computational load and its geographic distribution, the power needs of a network of cloud data centers can be used directly to consume negative priced power. If matched well, the result would include both financial savings for the cloud computing provider, but also better integration of even higher levels of renewable generation that overall reduce the carbon impact of both cloud computing and broadly the power absorbed by the power grid. www.scholink.org/ojs/index.php/ees Energy and Earth Science Vol. 3, No. 2, 2020 135 Published by SCHOLINK INC. Acknowledgements Thanks to the team at CAISO, notably Mark Rothleder and Shucheng Liu for hosting my visit, and to Zhu Liang for invaluable help with the data. Also thanks to Angela Glover, Hong Zhou, and other members of the Market Quality and Renewable Integration team for putting up with endless questions about data and how CAISO markets work! Thanks also to the University of Chicago whose support of Andrew A. Chien with a sabbatical leave for time at CAISO, essential to learning about myriad data systems and products, and the current and emerging operations of the CAISO markets and Energy Imbalance Markets (EIM). This work was supported by in part by the National Science Foundation under Award CMMI-1832230 References Bird, L., Milligan, M., & Lew, D. (2013). “Integrating Variable Renewable Energy: Challenges and Solutions. In NREL, Technical Report. https://doi.org/10.2172/1097911 CAISO (California Independent System Operator). (2018). Managing Oversupply. In Data summaries and daily reports. Retrieved from http://www.caiso.com/informed/Pages/ManagingOversupply.aspx Chien, A. A., Yang, F., & Zhang, C. (2018). Characterizing curtailed and uneconomic renewable power in the mid-continent independent system operator. AIMS Energy, 6(2), 376-401. https://doi.org/10.3934/energy.2018.2.376 Denholm, P., & Margolis, R. (2016). Energy storage requirements for achieving 50% solar photovoltaic energy penetration in california. Tech. rep., US Department of Energy, National Renewable Energy Laboratory. https://doi.org/10.2172/1298934 Denholm, P., & O’Connell, M. (2016). On the path to sunshot: Emerging issues and challenges in integrating high levels of solar into the electrical generation and transmission system. Tech. rep., US Department of Energy, National Renewable Energy Laboratory. https://doi.org/10.2172/1253978 E3, Investigating a higher renewables portfolio standard in california: Executive summary. (2014). Tech. rep., Report from Energy and Economics, Inc. Gross, D. (2015). The Night They Drove the Price of Electricity Down. In Slate Magazine., Retrieved September 18, 2015, from slate.com GWEC. (2016). Global wind report: Annual market update. Global Wind Energy Council, Tech. Rep., 2016, documents curtailment around the world. Kim, K., Yang, F., Zavala, V. M., & Chien, A. A. (2017). Data centers as dispatchable loads to harness stranded power. IEEE Transactions on Sustainable Energy, 8(1), 208-218. https://doi.org/10.1109/TSTE.2016.2593607 Lew, D., Bird, L., Milligan, M., Speer, B., Wang, X., Carlini, E.M., … Yoh, Y. (2013). Wind and Solar Curtailment. In Workshop on Large-Scale Integration of Wind Power into Power Systems, 10. https://doi.org/10.2172/1097911 https://doi.org/10.3934/energy.2018.2.376 https://doi.org/10.2172/1298934 https://doi.org/10.2172/1253978 https://doi.org/10.1109/TSTE.2016.2593607 www.scholink.org/ojs/index.php/ees Energy and Earth Science Vol. 3, No. 2, 2020 136 Published by SCHOLINK INC. MISO. (2018). The Mid-continent Independent System Operator (MISO). Retrieved from https://www.misoenergy.org/ Reed, S. (2017). Power Prices Go Negative in Germany, a Positive for Energy Users. In New York Times, December 25, 2017. Shehabi, A., Smith, S., Sartor, D., Brown, R., Herrlin, M., Koomey, J., … Lintner, W. (2016). United States Data Center Energy Usage Report. 2016. LBNL-1005775. https://doi.org/10.2172/1372902 Shoot, B. (2018). The World’s Fifth-Largest Economy, California, Just Committed to 100% Carbon-Free Power by 2045. In Fortune Magazine. Retrieved from http://fortune.com/2018/09/10/california-governor-carbon-free-power-energy/ Wiser, R. H., Mills, A., Seel, J., Levin, T., & Botterud, A. (2017). Impacts of variable renewable energy on bulk power system assets, pricing, and costs. In Technical Report LBNL-2001082, 11/2017 2017. https://doi.org/10.2172/1411668 Yang, F., & Chien, A. A. (2016). ZCCloud: Exploring wasted green power for high-performance computing. In Proceedings of the International Parallel and Distributed Processing Symposium (IPDPS 2016). https://doi.org/10.1109/IPDPS.2016.96 Yang, F., & Chien, A. A. (2017). Extreme scaling of supercomputing with stranded power: Costs and capabilities. IEEE Transactions on Parallel and Distributed Systems, 29(5), 1103-1116. https://doi.org/10.1109/TPDS.2017.2782677 Notes Note 1. In prior research, we have referred to this Opportunity Power (OP) as Stranded Power (SP) (Chien, 2018; Yang, 2017; Yang, 2016). Note 2. We are currently exploring how to couple rapidly growing networks of hyperscale cloud data centers (Shehabi, 2016) to the power grid to both enhance renewable integration and reduce cloud computing’s growing carbon footprint, as well as increase the resilience of both critical infrastructures (Kim, 2017; Yang, 2017). Note 3. This simple methodology overestimates negative-priced power by including power that may have been settled at a positive price in the Day-Ahead Market (DAM). https://doi.org/10.2172/1372902 https://doi.org/10.2172/1411668 https://doi.org/10.1109/IPDPS.2016.96 https://doi.org/10.1109/TPDS.2017.2782677 www.scholink.org/ojs/index.php/ees Energy and Earth Science Vol. 3, No. 2, 2020 137 Published by SCHOLINK INC. Appendixes Sample scatter plots for resources with high quantities and incidences of negative-priced power. As shown in Figure 15, res_170 has maximum power of just below 10 MW, and the scatter plot shows maximum duration of opportunity power interval of nearly 14 hours. It was the highest duty factor resource in our study for 2017. Figure 15. Power/Duration Scatter Plot for res_170 As shown in Figure 16, res_102 has maximum power of just below 130 MW, and the scatter plot shows maximum duration of opportunity power interval of nearly 13 hours. It was the 2 nd highest duty factor resource in our study for 2017. www.scholink.org/ojs/index.php/ees Energy and Earth Science Vol. 3, No. 2, 2020 138 Published by SCHOLINK INC. Figure 16. Power/Duration Scatter Plot for res_102