Based in Fiction: A Closer Look at a Heavy-Lift Market Report

Problem: What do you do if an organization that informs public/defense policy seems to be making suggestions based on inaccurate/irrelevant information? Especially if that information is repeated through several different media outlets without question?

That’s what I’m faced with when I look at RAND’s “Assessing the Impact of U.S. Air Force National Security Space Launch Acquisition Decisions: An Independent Analysis of the Global Heavy Lift Launch Market.” When I first saw it was released, I had hoped for clarity. But reading it led me to many questions.

In the space industry research field, it's not unusual to see variations in totals of activities such as launches or satellite deployments. The variation is often no more than a few percentage points difference. More often than not, I'll see a misinterpretation or transposition of a mass number. Some of these differences can be attributed to a research organization's definitions and categorizations--the methodologies it's using--but there is usually some kind of explanation about those differences.

The problem is, the way RAND's researchers did the categorizing and defining seems...inconsistent. This may be because the report's customer, the United States Air Force (USAF), has wacky ways of categorization and definition. If this were another industry report, such as that produced by Bryce or NSR, it wouldn't be so concerning. But the USAF's/SF's focus in supporting new launch systems is to ultimately gain the capability to defend the nation while giving U.S. troops an edge on the battlefield. How can it do that nominally if the basis--the foundation--for gaining that capability (the RAND report) is seemingly inaccurate?

Questions and Findings

I’m not the smartest banana in the bunch by far, but this document was confusing to me. The 144-page analysis was intended to answer two questions:

• How many U.S. launch service providers can the global heavy-lift launch market support?
• What are the effects of near-term acquisition decisions on the USAF's ability to:
  • meet NSS launch demand using U.S. launch service providers
  • sustain two or more U.S. launch service providers over the next ten years

Key findings from RAND’s analysts were:

The global addressable market (the share for which commercial firms compete) will likely grow only moderately over the next decade

• The annual number of launches worldwide grew to 71 in 2018 from 47 in 1998, but the commercial portion over which launch firms compete—addressable share—stayed steady at around 20 launches; today, the addressable share represents only 35 percent of the total market.
• SpaceX's Falcon 9 handles more than half of addressable launches today, with communications companies being the dominant customers.
• This market is likely to see only moderate growth over the next ten years, and the U.S. share of this market is likely to fall. The net result is that the addressable share of the launch market is unlikely to support more than one U.S. supplier of launch services focused on commercial heavy lift.

The National Security Space market is likely to face a near term supply shortage

• In many conceivable futures, the USAF's current acquisition plan is unlikely to provide sufficient supply of NSS launch vehicles certified to carry U.S. NSS payloads.
• The USAF can lower the risk of insufficient supply of NSS launch vehicles by gaining nonpriority access to legacy launch systems and/or by supporting three launch services providers through the 2022–2025 time frame.

Already, the questions begin forming--such as--what is an addressable market? Where did that 2018 annual launch number come from? What do they mean by “commercial heavy lift?” There are more questions...oh, so many more. But my time (and yours) is limited.

What is “Heavy-Lift?”

Let’s knock down what should be the easiest question to answer: where did that 2018 annual launch number come from? It’s easy because I did the research, data-gathering, and write-up of 2018 launch attempts for the Space Foundation’s “The Space Report.” The number of launch attempts for 2018 was much higher than RAND’s 71--it was 114 (which included two failures). Hell, we made flyers touting that fact. I know how and from where the Space Foundation collects data--every day, from primary and secondary sources. It’s not perfect, but it’s better than most.

The discrepancy is not a difference of single-digit percentages--it's around 36%. Because of this discrepancy, you know how it is--especially if you’ve been an evaluator--you start looking much more closely at what’s being presented. You start asking questions because you want to provide a benefit of the doubt or at least figure out if you’re missing something.

Those questions dovetail into the other questions asked already: what do the analysts mean when they talk of an addressable market and commercial heavy-lift? Do the parameters outlined from those caveats explain the discrepancy?

The following is what RAND’s analysts used to define “heavy-lift” (page 15):

...globally, only a handful of launch vehicles can launch the mass-to-orbit required for U.S. NSS payloads. For acquisition purposes, the USAF defines the NSS market in terms of mass-to-orbit requirements shown in Table 2.1…

There’s one problem there, then. Even though RAND is looking at heavy-lift, the USAF’s NSS requirements seem to encompass all lift capability. The table below is the result of RAND staying with this definition even though the table title itself includes the words “heavy lift” (page 18):

Looking at the sources and notes, there are a few worrying things. First, RAND is relying on the FAA’s “The Annual Compendium of Commercial Space Transportation,” which isn’t too terrible. But then, it lists two blogs for its online sources and for verification? Not knocking RussianSpaceWeb or SpaceFlightNow (I peruse them myself), but those are not official sources for data.

It would have been better for RAND’s analysts to go to the source--a space agency, a launch service provider’s or manufacturer’s launch history (they list them). What about Space-Track?

Even worse, the table’s note “a” then goes on to say RAND’s analysts were “...unwilling to take supplier marketing information as a valid source of data.” Does this mean that it doesn't take as gospel the mass capabilities of launche systems on a manufacturer's or launch service provider's site? Does that unwillingness extend to user guides--because those are easy to get and generally accurate. Where do they think the FAA’s space research contractor gets its data from? For that matter, where do the blogs the analysts are using for verification getting their information from? Seeing notes like those explains a few things--but in a worrisome way.

As someone who has written about this for a few years now, I also have some issues with a few of the rockets on the table’s list. In “China’s CZ-5B, Hand-wringing, Capabilities, and Credibility,” I listed the launch vehicles I considered that are currently in the heavy-lift category. They are the Ariane 5, Delta IV Heavy, Falcon 9, Falcon Heavy, Long March 5, and Proton. They are the launch vehicles whose manufacturers note can, at a minimum, launch 20 tons to low Earth orbit (LEO). And the chart below was included in that analysis. The numbers used to create it came from “supplier marketing information.”

Now, let’s put the other rockets from RAND’s table on this list, using supplier sources again. If there were families of a particular rocket, I used the highest mass capability listed.

The chart shows just how widely mass capabilities run using RAND’s methodology based on the USAF’s NSS requirements (not just heavy-lift). It almost borders on useless. I have to wonder why the company’s analysts used such the USAF’s definition that resulted in such a wide variation for “heavy-lift.” Yes, the USAF requested the report, but why not include some better data resolution so the customer can make a better decision? What is the relevancy?

Also, the inclusion of two launch vehicles historically used ONLY for International Space Station (ISS) missions--Soyuz-FG and Antares--is questionable. They are not used for commercial lift, either. Not only are they out-massed, but in the case of the Soyuz there are also commercial variants such as 2.1a/b, and ST-A/B that are used for GTO missions. For what it's worth, those variants are listed near the end of the report, but not used for the comparison.

The answer to what does commercial heavy-lift mean to RAND’s analysts is not clear then, but it appears to encompass more than just heavy-lift. The analysts do acknowledge this halfway through the document (page 72):

We define heavy lift launches on a sliding scale where a launch vehicle capable of “heavy lift” in 1998, such as the Molniya-M, might only be considered a medium lift vehicle had it entered service in 2018.

How is that useful? It doesn't explain the inclusion of the Antares or GSLV Mk III.

Market non-Definition

What about the “addressable market?”

The report defines the addressable market by what’s not included(?) starting on page 11. The basic definition doesn’t appear before or after this section of the report, but here’s what’s not included in the unincluded definition:

• Launches that are subject to international agreements
• National security launches
• National affinity launches
• Self-provisioning

Furthermore, the report notes that, distressingly, its definition for “addressable” is different from the FAA’s definition.

Our definition of addressable commercial launches is different than that used by the FAA in their annual reports. The FAA definition includes all launches with a commercial payload or that are competed in the commercial market. Therefore, it includes many launches that are subject to international agreement such as travel from/to the ISS. Furthermore, the FAA definition does not recognize self-provisioning launches as nonaddressable.

Great...just, great…with all these different definitions already floating around the industry, we now have yet another one.

So, maybe, and this is just a shot in the dark based on RAND’s “uninclusions,” the 71 launches were for the addressable market?

It turns out the answer to that is “no,” based on this chart on page 13:

Only 35% of the 71 is for the addressable market. And throughout the report, it’s not clear where the other 36% of the 114 launches in 2018 disappeared to. It’s an inaccuracy with no explanation. How can the first two questions the analysts are asking be answered with any accuracy?

Keep in mind this post is discussing just two areas of the report that seem to be inaccurate/irrelevant.

That’s troubling for a report that’s making recommendations to the USAF/SF about how to work with commercial launch providers in the U.S. How can recommendations be taken seriously if the supporting data appears to be skewed or missing? These recommendations are supposed to be for systems used ultimately for defending the nation and keeping U.S. soldiers from dying in the next battle. But if the data is incomplete...well, it seems wrong to sell a report like this. Even more troubling is the DoD believes the report is in sync with the its internal studies, indicating the DoD itself isn't using reliable data.

And while I haven’t looked at RAND’s other reports, I now have to wonder a little each time it publishes one. Is it accurate? Is it complete?