Go to Home Page - DOT&E Director, Operational Test & Evaluation  
Home   Search  
 About  Publications  Organization  Related Sites 
FY00 Annual Report
Download Printable PDF Version
Comment on this Report

JOINT STRIKE FIGHTER (JSF)


Joint ACAT ID Program Prime Contractor
Total Number of Systems:3,128Lockheed Martin or Boeing
Total Program Cost (TY$):$200B 
Average Unit Cost (TY$):$35M 
Full-rate production:1QFY09 


SYSTEM DESCRIPTION & CONTRIBUTION TO JOINT VISION 2020

The Joint Strike Fighter (JSF) Program will develop and deploy a family of strike aircraft by capitalizing on commonality and modularity to maximize affordability while addressing the needs of the Air Force, Navy, Marine Corps, and United Kingdom Royal Navy and Royal Air Force. This family of strike aircraft will consist of three variants: (1) Conventional Takeoff and Landing (CTOL); (2) Aircraft Carrier Suitable (CV); and (3) Short Takeoff and Vertical Landing (STOVL). The focus of the program is affordability: reducing the development, production, and ownership costs of the JSF family of aircraft. The family of JSF variants will provide the Navy with a first-day-of-the-war, survivable aircraft to complement the F/A-18E/F; the Air Force with a replacement for the F-16 and A-10 and complement the F-22; the Marines with a single STOVL platform to replace the AV-8B and F/A-18C/D; and the Royal Navy and Royal Air Force with a supersonic STOVL fighter/attack aircraft to replace the Sea Harrier and GR-7, respectively.

Foreign interest in the program is high, and a number of allies have entered into cooperative agreements to participate in the program. All variants will contribute to the Joint Vision 2020 concepts of precision engagement and full-dimensional protection. The JSF will be a single-seat, single-engine aircraft capable of performing and surviving lethal strike warfare missions using an affordable blend of key technologies. A high degree of commonality (70-80 percent) exists amongst the three variants. Aircraft sub-systems that will be identical in all three designs include the wing and fuselage structures, engine core, avionics and onboard computers, cockpits, electrical wiring, electronics, flight controls, control surfaces, and fuel tanks. The JSF system consists of the JSF air vehicles and all support training equipment, related facilities, materiel, software, services, and personnel to ensure that the system can accomplish its intended operational role.


BACKGROUND INFORMATION

The purpose of the JSF Program is to affordably develop the next-generation strike fighter weapons system to meet an advanced threat (2010 and beyond), while improving lethality, survivability, and supportability. The JSF Program originated from the Joint Advanced Strike Technology (JAST) program.

A multi-year $2.2 billion JSF Concept Demonstration and Risk Reduction (CDRR) effort commenced in November 1996, with competitive contract awards to Boeing and Lockheed Martin for the CDRR Program. These competing contractors have each built two concept demonstrator flight test aircraft to conduct concept-unique ground demonstrations and continue refinement of their ultimate delivered weapon system concepts. Pratt & Whitney is providing propulsion hardware and engineering support for both Boeing's and Lockheed Martin's ongoing JSF CDRR efforts. The JSF Alternate Engine Program with General Electric continues to develop an alternate engine for production in order to reap the financial and performance benefits of competition.

The EMD program, which will follow CDRR, will be structured to develop the JSF weapon system in a series of block upgrades with successively increasing capabilities. The first three blocks will be intended to achieve the full performance capabilities currently set forth in the Joint Operational Requirements Document. As EMD progresses, the users are expected to require new capabilities for future block upgrades as the strengths and weaknesses of various emerging technologies become apparent.

While survivability has two primary components, threat avoidance and damage tolerance, the JSF focused early on defining its susceptibility requirements (threat avoidance) in terms of radar and infrared signature levels. The vulnerability requirements were raised to the JSF System Program Office over three years ago, but only recently has the program identified engagement probabilities of kill given a hit (damage tolerance) to establish vulnerability and reparability requirements.

The JSF is required to be less vulnerable than the F-16. The aircraft is expected to feature a "not-to-exceed" engagement Probability of Kill (Pk ) for aircraft loss or pilot casualties as the result of an impact of a 23mm API/HEI, 30mm HEI projectile, MANPADS threat, or proximity fuzed missiles. JSF susceptibility and vulnerability to directed-energy, chemical, and biological weapons will also be addressed. The design guidelines to reduce the vulnerability of the JSF include redundancy and separation of critical sub-systems, components, lines, and structure. These guidelines also encourage the placement of fuel cells to minimize the ingestion of leaking fuel into the engine inlets. The standard evaluation process for JSF has evolved from the specification of a given vulnerable area against a 30mm projectile to an engagement Pk-based methodology. The goal for aircraft battle damage repair is to be able to repair damage from 23mm hits with organizational maintenance within 24 hours.


TEST & EVALUATION ACTIVITY

DOT&E has continuously participated in JSF OT&E and monitored LFT&E planning activities since June 1995 when it was known as the JAST program. Integrated Product Team meetings are being held to address OT&E and LFT&E. The Combined Test Working Group (CTWG) (a systems test IPT) is responsible for all T&E efforts in executing the JSF CDRR program and planning for the EMD program. The CTWG provides a single point of contact for the member services, OSD, and the weapon systems contractors for all T&E related matters. During the JSF CDRR Phase, competing contractor teams led by Boeing and Lockheed Martin have each built, qualified, and are flying two Concept Demonstrator Aircraft designated the X-32 and X-35, respectively. Rather than being prototypes with full-up systems, these demonstrators will incorporate the engine and outer mold lines of the contractor's JSF design and largely use off-the-shelf systems and avionics. These demonstrators are intended to demonstrate the viability of each contractor's airframe design concept, including the ability to accomplish short take-off, hover and transition to wingborne flight, up-and-away performance, and low-speed handling consistent with landing aboard a carrier. During this phase, each contractor is responsible for planning and executing the ground and flight tests and demonstrations. During the current CDRR phase, government personnel will actively participate in test planning and execution at the discretion of the respective competing contractors. The OTAs for JSF, AFOTEC, and COMOPTEVFOR are conducting an EOA to support the Milestone II decision. During EMD, an integrated test team will perform all developmental testing and OTAs will conduct operational assessments and dedicated OT&E.

During EMD, ground test and flight test aircraft will be built representing all three variants, and will be augmented by full-mission simulators and flying avionics testbed aircraft. The OTAs and DOT&E will continue as active participants in the Combined Test Working Group throughout EMD; and the OTAs as members of the JSF Integrated Test Force, will independently plan, conduct, and report a series of OAs. OTA activity will culminate with the conduct of dedicated OT&E in the FY10 timeframe in support of a full-rate production decision.

LFT&E activities have centered on the development of an LFT&E strategy, transferring applicable background information on LFT&E to the contractors, and ensuring that the appropriate lessons learned from completed LFT&E and Joint Live Fire programs were transferred to the JPO and contractors. A separate Survivability Integrated Product Team (IPT), which includes a DOT&E representative, has been formally established to work on JSF survivability matters as well as the JSF LFT&E Program. Each contractor team separately briefed the JSF Survivability IPT on their proposed designs, survivability, and LFT&E programs.

DOT&E and the Joint Program Office (JPO) have agreed on an LFT&E strategy to evaluate the survivability of the JSF variants. In testimony to the Senate's Tactical Air Combat Subcommittee on March 22, 2000, DOT&E made clear the requirement to conduct full-up, system-level Live Fire testing of at least one variant of the JSF program due to the uniqueness of the aircraft design, new technologies being used in the design, and the inadequacy of modeling and simulation to adequately predict aircraft vulnerability or identify needed design changes. This testing would be coupled with component and sub-system level Live Fire testing of all of the JSF configurations, assuring that those areas which are particularly unique to each aircraft configuration were included in the LFT&E plan. This agreement is based on the current high degree of commonality (70-80 percent) shared amongst the three variants.

This strategy will require waiver certifications and approved alternative LFT&E plans for any variant that will not undergo full-up, system-level Live Fire Testing. The JPO will prepare the waiver request(s) and alternative LFT&E plan(s) for the variant(s) that will not undergo full-up, system-level Live Fire testing and have the necessary certification(s) and approval(s) prior to Milestone II. The TEMP will contain one integrated, overall LFT&E Plan for all three aircraft variants, consisting of the LFT&E program plan for the variant(s) that will undergo full-up, system-level Live Fire testing, as well as the approved alternative LFT&E plan for the variant(s) receiving waivers from full-up, system-level Live Fire testing.

The JSF LFT&E strategy also includes realistic, end-to-end lethality tests of the new gun system. The gun lethality LFT&E Program consists of ammunition lethality characterization tests, followed by actual gun firings from an aircraft engaging targets expected to be attacked in combat. Since realistic end-to-end testing is planned, a waiver from full-up, system-level Live Fire Testing will not be pursued for the JSF gun lethality LFT&E program.

The LFT&E issues in the current TEMP address, in a generic fashion, how the program intends to complete realistic survivability testing of the aircraft and realistic, end-to-end lethality testing of the gun. The TEMP does not provide details that will prejudice any one contractor or divulge proprietary information. This will permit the contractor, using the TEMP as one of its tools, to develop and propose a survivability program plan that will address all of the issues pertaining to their unique designs. The DOT&E staff briefed each contractor team on the LFT&E requirements in June 2000 at each contractor's facility. The briefings were identical and included background information on the evolution of the statute that requires LFT&E, how DOT&E prefers to address LFT&E through a building block approach, and examples of lessons learned through various LFT&E programs. Both contractor teams have prepared preliminary plans for LFT&E and briefed them to DOT&E and the Joint Program Office. Both contractors' proposals are adequate at this point in the program but will be updated immediately following contract award.

Since the JSF is a single-engine aircraft, the vulnerability of the engine is particularly critical for survival. In light of this, DOT&E has recommended for the last two years that the F119 engine, which will be the core for the JSF engine, undergo live fire testing now. Lessons learned from the F-18E/F LFT&E tests on the F414 engine show that engine vulnerabilities uncovered early can be significantly reduced by re-design of either the engine components, the engine bay, and/or the surrounding aircraft structure and components. This year, the JPO submitted a plan to DOT&E to conduct ballistic tests of two critical F119 engine components in 2001: the first stage Integrally Bladed Rotors (IBR) and the second stage IBR. Additional engine components are expected to undergo ballistic testing in subsequent years, and the survivability IPT is working to develop an engine vulnerability program as part of the overall JSF LFT&E program.


TEST & EVALUATION ASSESSMENT

At this stage of the JSF program, the integration of program planning and T&E planning appears to be on a solid foundation. However, in view of the complexity of the program objectives, numerous T&E opportunities and challenges are being, and will likely continue to be encountered.

In support of its commitment for an affordable, highly common family of next-generation multi-role strike fighter aircraft; the JSF program has adopted an iterative approach toward facilitating the Services' development of fully validated, affordable operational requirements. This approach emphasizes the early and extensive use of cost-performance trades. To assess military utility in support of these trades, the JSF program is continuing development of its Strike Warfare Collaborative Environment (SWCE), a baseline-common modeling and simulation environment to ensure consistent models and data bases. The open process for requirements development and the availability of the SWCE provide needed avenues to improve the linkage between test and requirements processes. In addition, the models used in conjunction with the SWCE may prove useful in the T&E process, although experience has shown that the best available models are not always sufficiently credible for T&E needs.

The JSF will have some technological issues that are important to OT&E. The JSF is expected to have significantly improved interoperability and C4ISR capabilities, as well as a very highly evolved set of sensors, all of which will be highly integrated with the avionics systems. In particular, these systems will provide the JSF with some of its most distinctive and important operational capabilities. Adequately testing these advanced capabilities at an operational mission level will be another challenge to the test program. The design of appropriate metrics and the development of appropriate scenarios and environments will be a long and involved process. Integration of M&S with flight testing holds potential for addressing this issue.

The JSF will employ some new technologies, and these must be identified early in the program so that they can be monitored during the test program. As one example, the method of providing vertical thrust to the STOVL variant will be a significant advance over the current operational systems and thus carries a corresponding risk, and extra attention should be given to this sub-system. Another area that should be given extra attention is the performance and maintenance requirements of the Low Observables (LOs) and other classified capabilities on the JSF, particularly in the shipboard environment. Current LO systems have experienced difficulty after being fielded, and the JSF test program should endeavor to identify these during OT&E so that any required corrections can be completed prior to fielding the system.

Due to the expected high degree of commonality of the three variants, a single integrated test program is planned, which should save both time and money during the test program. As part of their proposals, the contractor test teams have identified test points that will apply to more than one variant so that unnecessary testing is not performed.

Although economy of testing is expected, the JSF is expected to perform many different missions, and the way each service carries out these missions sometimes have subtle but important differences that must be taken into account in OT&E planning. Adequately exploring this range of missions will be another challenge for the JSF test team.

Modeling and simulation is expected to play a large role in the operational evaluation of the JSF. For example, the JSF plans on supplying the Strike Warfare Collaborative Environment (SWCE) as Government Furnished Equipment, while the contractor will provide digital product descriptions (DPDs) of the JSF that will operate within the SWCE. These efforts will provide new opportunities for integrating M&S with flight testing, but they will also raise new issues. For example, the program intends to validate SWCE and to leave the validation of the DPDs up to the contractor. It is not clear at this time whether this will be adequate for DOT&E, or if an independent validation of these components would be justified. Complete government validation of the DPDs would add costs that are not now programmed, and would place demands on the flight test program that are not currently in place.

The ongoing CDRR Phase will allow early test insights into the viability of basic aircraft designs of the competing contractors to meet the requirements of commonality/modularity for an affordable family of multi-Service aircraft. In addition, these aircraft will demonstrate specific short take-off and vertical landing, hover, transition, and low-speed approach characteristics. More challenging to assess during the CDRR Phase will be the contractors' progress in developing the integrated avionics suite that will be essential to the final JSF design, as well as validating needed improvements in operational supportability and the cost of ownership. Improved insights into the risks of integrated avionics may be available prior to the JSF Milestone II decision from the ongoing F-22 program, which is leading the way in facing such challenges. Since both of the competing JSF contractors are key members of the F-22 team, the lessons learned from that program should reduce the risks in similar areas of the JSF. The planning for EMD provides ample opportunities for the conduct of OAs leading up to dedicated OT&E. As the program matures, it will be essential to define specific accomplishments/characteristics that each of the operational test periods can confirm consistent with the event-driven acquisition strategy required by DoD Regulation 5000.2-R and adopted by JSF. The current planning for dedicated OT&E includes 12 LRIP test articles. While this quantity of aircraft is adequate for the conduct of a thorough operational test, it is not too many since three different aircraft configurations must be tested in the accomplishment of a variety of missions.

The LFT&E portion of the JSF TEMP will be written from a high-level perspective; i.e., it will not contain an approach specific to any one design since contract award will not be made until Milestone II (after the TEMP will be submitted). LFT&E issues in the TEMP will have to address, in a generic fashion, how the program intends to complete realistic survivability testing of the JSF, while at the same time not defining it to a level of detail that will prejudice any one contractor. This will permit the contractor, using the TEMP as one of its tools, to propose a survivability program plan/proposal that will address all of the issues pertaining to their unique designs.

There is a risk, however, that if the TEMP is too generic, the competitors might misinterpret the testing requirements and develop a proposal that will not adequately address how real survivability testing will be completed prior to full-rate production. Since down-select is shortly before Milestone II (waiver/alternate LFT&E strategy deadline), a least-cost strategy may not adequately address real testing. This strategy might not gain approval from DOT&E after a contractor (and associated program plan) has been selected, leading to heavily contested testing issues and associated program costs. DOT&E is working closely, as a member of the CTWG, to ensure the TEMP is written to the detail required to prevent this from happening.

The most significant LFT&E issue is whether the program will conduct full-up, system-level testing. The program has not yet committed and is still investigating the waiver to full-up, system-level LFT&E. DOT&E has proposed that the JSF program conduct a full-up, system level test on one of the designs and request a waiver from full-up, system-level testing for the remaining two designs. Additionally, this office has recommended that they pursue a strategy that also includes testing one full-scale test article (possibly the Navy's drop test article) in addition to the required component and sub-system tests.

The LFT&E portion of the JSF TEMP was written from a high-level perspective; i.e., it does not contain an approach specific to any one design since contract award will not be made until Milestone II (after the TEMP will be submitted). LFT&E issues in the TEMP address, in a generic fashion, how the program intends to complete realistic survivability testing of the JSF, while at the same time not defining it to a level of detail that will prejudice any one contractor. This will permit the contractor, using the TEMP as one of its tools, to propose a survivability program plan/proposal that will address all of the issues pertaining to their unique designs.

Both contractor teams have prepared plans for LFT&E and briefed them to DOT&E. Both contractors' plans appear adequate at this point in the program.

DOT&E has on several occasions recommended that the F119 engine should undergo live fire testing now. Lessons learned from the F-18E/F LFT&E tests (F414 engine) show that engine vulnerabilities uncovered early may be reduced by re-design of either the engine components, the engine bay, and/or the surrounding aircraft. Plans were made to conduct some ballistic tests on F119 components during FY01.

There is a risk that a competitor might scale back his current LFT&E plans to gain a cost advantage in the proposal. Down-select is shortly before Milestone II (waiver/alternate LFT&E strategy deadline). A least-cost strategy might not adequately address real testing and the strategy might not gain approval from DOT&E after a contractor (and associated program plan) has been selected. This could lead to heavily contested testing issues and associated program costs. Since there is very little time from receipt of proposals, source selection, and down-select to Milestone II, the issues that such an unsatisfactory plan will raise might not be resolved before the milestone, which is the waiver deadline. DOT&E is working closely, as a member of the CTWG, to ensure the TEMP is written to the detail required to prevent this from happening and the contractors understand DOT&E's interpretation of the TEMP.

An important LFT&E concern is whether the program will conduct full-up, system-level testing. The JSF Program plans to conduct full-up, system-level testing for one variant and to request waivers for the other two variants. Those features that are unique to the two variants will be tested in nearly complete, full-scale test aircraft and components. One of the less-than-full-up specimens will be a drop-test article. Prior to Milestone II, the program will apply for waivers for the two aircraft that will be tested in less-than-full-up condition. The TEMP will contain one Live Fire Evaluation Plan for all three aircraft variants, which will also constitute the Alternative Plan for the two waived variants.

Although both contractors' LFT&E plans appear adequate at this point in the program, there is a risk that a competitor might scale back their LFT&E program plan to gain a cost advantage in the proposal process. A least-cost strategy might not adequately address real testing and the strategy might not gain approval from DOT&E after a contractor (and associated program plan) has been selected. This could lead to heavily contested testing issues and associated program costs. Since there is very little time from receipt of proposals, source selection, and down-select to Milestone II, the issues might not be resolved before the milestone, which is the waiver deadline. DOT&E is working closely, as a member of the CTWG, to ensure the TEMP is written to the detail required to prevent this from happening, and the contractors seem to understand DOT&E's interpretation of the TEMP.


CONCLUSIONS, RECOMMENDATIONS AND LESSONS LEARNED

Based on the complexity of the F-22 program, we anticipate a great challenge in testing and evaluating the effectiveness of sensor fusion in the JSF.

The durability and maintainability of low-observable materials and processes will have to be included in early operational assessments. In particular, in a shipboard environment, maintenance of classified aspects of the system may prove particularly challenging.

The unprecedented high level of interoperability with other systems expected of the JSF will present unique challenges for testing and evaluating.


Return to FY 00 Table of Contents
Return to DOT&E Home Page
Comment on this Report