SETFST is recursive (step-by-step and repeatable), flexible, and scalable.
The SETFST process provides a consistent framework for Subject
Matter Experts (SMEs) and managers to
do the following:
- Capture, discuss, negotiate, and evolve alternatives toward consensus;
- Realize the highest probability of system success;
- Explore the range of acceptable values expressed in desirement
definitions and achieve outcomes superior to those that result
from settling for minimally acceptable levels of performance with
respect to requirements;
- Reveals sensitivities and quantifies risk;
- Is easily updated as desirements and knowledge evolve;
Step 1: Form Integrated Process Team
SETFST is a collaborative effort. The process begins by creating a
clear understanding of the problem. Often, that requires looking at
the project from several standpoints. The first step is to form an
Integrated Product Team (IPT). The knowledge and expertise of team
members will bring a number of perspectives to the problem, creating a
fuller, more well-rounded understanding. Project teams work best when
they contain a relatively small number of Subject Matter Experts (
SMEs)e.g., engineers, scientists, regulators, logisticians,
financial analysts, customers, and users. The team creates
systematic and traceable inputs that encourage what if scenarios as
they work through the definition of desirements and perform Value Analysis.
Step 2: Negotiate Desirements
SETFST is a collaborative effort. The process begins once the full IPT is in place.
The IPT begins capturing desirements—definition and documentation of what the final
product, technology, or system must do rather than how it can be done. Customers
are identified, initial desirements are defined, Key Performance Parameters (KPPs)
and Evaluation Criteria (EC) are established.
The customers provide criteria related to performance, reliability,
portability, schedule to completion, life-cycle cost, maintainability, etc.
Each criterion is documented with a description, a unit of measure, customers
or applications to which it pertains, and assumptions made. Details specified
for each criterion for each customer include priority, objective, desirability
limit, desirability function (d-curve), and a weighting factor. Each criterion
is assigned a numerical weighting factor reflecting its importance relative to
other desirements of the same type.
Types and Constructed Customer
Types are weighted for each customer's view of that type's relative importance.
The d-curve, with its objective and desirability limit(s) defines one measure
of "goodness" for a particular customer. Individual desirements are documented
for every application or customer. A composite set of desirements is identified
comprising desirements for all customers, i.e., for the "Constructed Customer".
Exit criteria encompass the subset of the constructed set of desirements that
define successful completion; they include Key Performance Parameters (KPPs) and
all other criteria that must be met. An objective function is generated that
incorporates all desirements. The evaluation and analysis activities enable prediction
of where, within the multidimensional solution space, desirements are met, and provide
pointers to the best solutions. This approach may be used to guide investment
of program resources to the candidate technologies most likely to
provide an affordable solution—meeting customer's needs at a price those
customers are willing to pay.
Step 3: Generate Alternatives
The team identifies possible solutions (called alternatives) to satisfy
the desirements. These may be new technologies; combinations of existing
materials, technologies, or components; novel approaches; or other systems with
potential to satisfy the set of desirements. Identifying requirements for what
the final product, technology, or system must do, is crucial.
The customer provides information such as: performance, reliability,
portability, schedule, life-cycle cost, and maintainability.
Criteria are documented with description, unit of measure, customer,
priority, objective, threshold, desirability, and type. Requirements
are documented for each customer, and exit or solution criteria are determined.
Step 4: Evaluate Alternatives
SETFST Value Analysis
Value Analysis evaluates the alternatives against the desirements. The measures
of merit are desirability and risk. Desirability is a measure of "goodness".
Risk is the probability of failure to exceed the desirability limit.
Theoretical models, simulations, expert opinion, statistically designed
experiments, or other quantitative assessment methods may be used to
predict how each alternative will fare with respect to a desirement.
A worksheet for each alternative evaluates it against every criterion of a
given type. Tabulated in the worksheet for each desirement are the predicted
mean response value of the desirement's unit of measure and some measure of
spread of the prediction interval for that alternative. The predicted mean
value is translated into a desirability value, di, using the desirability
function for desirement i.
The di values are aggregated into a composite desirability (Dtype)
for the alternative with respect to all desirements of the given type.
A measure of risk is calculated for each alternative for each desirement
and aggregated to produce a risk measure for the alternative against all
desirements of a given type. Similarly, top-level desirability is calculated
using Dtype values and weighting factors, while the overall risk measure is
generated from the lower-level probabilities of failure to meet desirability limits.
From these worksheets, a Value Scorecard compiled for each customer—one scorecard
for each type of desirement and one Affordability Scorecard showing top-level results.
Scorecard metrics include expected value, the desirability value to which
it maps, and risk. The Value Scorecard presents desirability and risk for
multiple alternatives for all desirements for a single customer. Desirability
and risk are displayed at three levels: individual desirement, desirement type,
and overall. The scorecard assists in identifying risk drivers and technical
challenges in meeting the desirements. The Value Scorecard for a single customer
is used to communicate with that customer when reporting progress or raising issues.
The scorecard for the Constructed Customer supports program decisions such as
selecting a single technology alternative for further development.
Once the team has focused on a particular technology alternative,
the process moves to refining it to best meet the program objectives
and exit criteria. Levels of design variables are evaluated in different
combinations to produce response-value predictions using statistically
designed experiments and response-surface methodology. Regression
analysis or other quantitative methods are performed to generate
predictive functional relationships. The Multiattribute Desirability
Optimization Methodology applies an optimization formula to establish
values of design variables that relate to the best-value, most robust solution.
Step 5: Document Results
SETFST Document Results
The SETFST process documents the history of the work done to determine the
project direction. Applying the SETFST process early in a program permits
the team to focus on the few alternatives most likely to lead to successful
product development. The process can be efficiently and quickly revisited
periodically to refine the assessment or reevaluate alternatives in light
of criteria that may have changed due to customer involvement and insight
gained into the realm of the possible, a changing world, or as a result of
knowledge derived through further research.
The SETFST process is an intense approach to a project.
It involves dedicated commitment on the part of the project leadership and
the team members, but the results are worth the time and effort
spent. Well-researched desirements and carefully analyzed alternatives
define the direction toward the best solution, so that the project
may move ahead without the delays inherent in less well-defined
research projects. Repeatedly, teams report that they have
developed better products in a shorter time using this process.
Systems Engineering Tailored for Science and Technology (SETFST) is a structured
approach to identifying optimal solutions to complex problems. Applied to research and
development, SETFST contributes to sound systems engineering over the life cycle of
those products that emerge from the effort. SynGenics developed the SETFST process
to help ensure that customer requirements/desirements are understood
and to support good decision-making in the absence of complete information at
each stage—from conceptual design to critical design review. The strengths of the
SETFST process include its conceptual simplicity and its analytical rigor.
At its simplest, desirements are identified and alternatives that might
satisfy those desirements are evaluated, rated, and compared. It is a robust process with multiple benefits. SETFST
- Is recursive (step-by-step), flexible, and scalable.
- Provides a consistent framework for subject-matter experts (SMEs) and managers to
- Capture, discuss, negotiate, and evolve alternatives toward consensus
- Realize the highest probability of "system" success
- Explore the "grayness" in desirements and go beyond the traditional approach to satisfying multiple, competing criteria, which typically involves barely meeting the minimums.
- Reveals sensitivities and quantifies risk.
- Is easily updated as desirements and knowledge evolve.
Through statistical analysis, the process predicts the optimal locus within the solution space at which desirements (evaluation criteria) are met, thus giving researchers and customers an objective function that accommodates all the criteria. This approach helps to assure that research funds and efforts are channeled to the candidate technologies most likely to provide the overall best possible value—meeting the customer’s needs at a price that customer is willing to pay.