|
BACK TO 125
EXAMPLES in Engineering, Operations Management and Computer Information
SystemsLE |
|
Do it once, do it right, and do it now. |
|
Back to Lawson Computing Homepage |
Apply as needed,
when needed. |
APPENDIX
C
DCAA
REVIEW OF PARAMETRIC
NOTE: This is an excerpt from the Defense Contract Audit Agency Contract
Audit Manual. The last page of this appendix includes an order form to obtain
the entire manual.
APPENDIX
C
9-1000 SECTION 10 - REVIEW OF
PARAMETRIC
9-1001 INTRODUCTION
This
section contains an overview and general guidance on reviewing cost-to-noncost
estimating relationships, primarily in the context of contractor price
proposals. This section also contains guidance on the use of estimating
standards in price proposals. This supplementary guidance contains criteria
contractors should meet before submitting proposals based on parametric cost
estimates.
9-1002 PARAMETRIC
9-1002.1 Definition of Parametric Cost
Estimating
a. Parametric
cost estimating ("parametrics") has been defined as a technique
employing one or more cost estimating relationships (CERs) to estimate costs
associated with the development manufacture, or modification of an end item. A
CER expresses a quantifiable correlation between certain system costs and other
system variables either of a cost or technical nature. CERs are said to
represent the use of one or more independent variables to predict or estimate a
dependent variable (cost).
b. Parametrics
encompasses even the simplest traditional arithmetic relationships among
historical data such as simple factors or ratios used in estimating scrap
costs. However, for audit purposes our guidance will limit special
consideration of parametrics to more advanced or complex applications. These
may involve extensive use of cost-to-noncost CERs, multiple independent
variables related to a single cost effect, or independent variables defined in
terms of weapon system performance or design characteristics rather than more
discrete material requirements or production processes. EDP data bases and/or
computer modeling may be used in these types of parametric cost estimating
systems.
c. Parametric
estimating techniques may be used in conjunction with any of the following
estimating methods:
(1) Detailed—also
known as the bottoms-up approach. This method divides proposals into their
smallest component tasks and are normally supported by detailed bills of
material.
(2) Comparative—develops
proposed costs using like items produced in the past as a baseline. Allowances
are made for product dissimilarities and changes in such things as complexity,
scale, design, and materials.
(3) Judgmental—subjective
method of estimating costs using estimates of prior experience, judgment,
memory, informal notes, and other data. It is typically used during the
research and development phase when drawings have not yet been developed.
9-1002.2 Distinction Between Cost and
Noncost Independent Variables
a. Although
the basic criteria for cost-to-cost and cost-to-noncost CERs are generally
comparable, the supplementary criteria in this section pertain to
cost-to-noncost CERs. Audits of traditional cost-to-cost estimating rates and
factors are covered in other sections of this chapter and in referenced
appendixes.
b. Cost-to-noncost
CERs are CERs which use something other than cost or labor hours as the
independent variable. Examples of noncost independent variables include
end-item weight, performance requirements, density of electronic packaging,
number or complexity of engineering drawings, production rates or constraints,
and number of tools produced or retooled. CERs involving such variables, when
significant, require that the accuracy and currency of the noncost variable
data be audited. Special audit considerations are described in 9-1003 and
9-1004.
9-1002.3 Uses of Parametric Cost Estimates
a. Parametric
cost estimating is used by both contractors and government in planning,
budgeting, and executing the acquisition process. Parametric cost models are
generally made up of several CERs and can be used to estimate the costs for
part of a proposal or the entire proposal. The cost models are often
computerized and may be made up of both cost-to-cost and cost-to-noncost
interrelated CERs. The guidance contained in this chapter is intended to assist
in the review of parametric estimates, CERs, and/or cost models used in
developing price proposals for negotiation of government contracts.
b. Parametric
cost estimates are often used to cross-check the reasonableness of estimates
developed using other estimating methods. Generally, it would not be prudent to
rely on parametric techniques based on a broad range of data points to estimate
costs when directly applicable program or contract specific historical cost
data is available, as in the case of follow-on production for the same hardware
in the same plant. Nor would parametric techniques be appropriate for contract
pricing of specific elements such as labor and indirect cost rates which
require separate forecasting considerations such as time and place of contract
performance. The use of a parametric estimating method is considered
appropriate, for example, when the program is at the engineering concept stage
and the program definition is unclear, or when no bill of materials exists. In
such cases, the audit evaluation should determine that:
(1) the
parametric cost model was based on historical cost data and/or was calibrated
to that data, and
(2) the
contractor has demonstrated that the CER or cost model actually reflects or
replicates that data to a reasonable degree of accuracy.
9-1003 PARAMETRIC ESTIMATING CRITERIA FOR
PRICE PROPOSALS
When
a contractor uses parametric cost estimating techniques in a price proposal,
the auditor will apply all pertinent criteria applicable to any proposal along
with the supplemental criteria provided in 9-1004.
9-1003.1 Disclosure of Parametric
Estimating Data
a. The
purpose of the Truth in Negotiations Act, 10 U.S.C. 2306(a), is to provide the
government with all facts available to the contractor at the time of
certification and that the cost or pricing data was current, complete, and
accurate. Parametric estimates must meet the same basic disclosure requirements
under the act as detailed estimates.
b. Although
the principles are no different, proposals supported in whole or in part with
parametric estimating will present new fact situations concerning cost or
pricing data which is required to be submitted. A fundamental part of the
definition of cost or pricing data is "all facts ... which prudent buyers
and sellers would reasonably expect to have a significant effect on price
negotiations" (FAR 15.801). Reasonable parallels may be drawn between the
data examples provided in FAR for discrete estimating approaches and the type
of data pertinent to parametric estimating approaches. For example, if a
contractor uses a cost-to-noncost CER in developing an estimate, the data for
the CER should be current, accurate, and complete.
c. Many
contractors use parametric cost estimating for supplementary support or for
cross-checking estimates developed using other methods. Judgment is necessary
in selecting the data to be used in developing the total cost estimate relied
upon for the price proposal. In distinguishing between fact and judgment, FAR
states the certificate of cost or pricing data "does not make
representations as to the accuracy of the contractor's judgment on the
estimated portion of future costs or projections. It does, however, apply to
the data upon which the contractor's judgment is based" (FAR 15.804-4[b]).
Therefore, if a contractor develops a proposal using both parametric data and
discrete estimates, it would be prudent to disclose all pertinent facts to
avoid later questions about full disclosure.
d. The
auditor should address the following questions during the evaluation of
Parametric cost estimates:
* Do
the procedures clearly establish guidelines for when parametric techniques
would be appropriate?
* Are
there guidelines for the consistent application of estimating techniques?
* Is
there proper identification of sources of data and the estimating methods and
rationale used in developing cost estimates?
* Do
the procedures ensure that relevant personnel have sufficient training,
experience, and guidance to perform estimating tasks in accordance with the
contractor's established procedures?
* Is
there an internal review of and accountability for the adequacy of the estimating
system, including the comparison of projected results to actual results and an
analysis of any differences?
9-1004 SUPPLEMENTAL ESTIMATING CRITERIA
The
auditor should also consider the following supplemental criteria when
evaluating parametric cost estimates.
9-1004.1 Logical Relationships
The
contractor should demonstrate that the cost-to-noncost estimating relationships
used are the most logical. A contractor should consider all reasonably logical
estimating alternatives and not limit the analysis to the first apparent set of
variables. When a contractor's analysis discloses multiple alternatives that
appear logical, statistical testing (9-1004.3) of selected logical
relationships may be used to provide the basis for choosing the best alternative.
9-1004.2 Verifiable Data
The
contractor should demonstrate that data used for parametric cost estimating
relationships can be verified. In many instances the auditor will not have
previously evaluated the accuracy of noncost data used in parametric estimates.
For monitoring and documenting noncost variables, contractors may have to
modify existing information systems or develop new ones. Information that is
adequate for day-to-day management needs may not be reliable enough for
contract pricing. Data used in parametric estimates must be accurately and
consistently available over a period of time and easily traced to or reconciled
with source documentation.
9-1004.3 Statistical Validity
The
contractor should demonstrate that a significant statistical relationship
exists among the variables used in a parametric cost estimating relationship.
There are several statistical methods such as regression analysis that can be
used to validate a cost estimating relationship, however, no single uniform test
can be specified. Statistical testing may vary depending on an overall risk
assessment and the unique nature of a contractor's parametric data base and the
related estimating system. Proposal documentation should describe the
statistical analysis performed and include the contractor's explanation of the
CER's statistical validity.
9-1004.4 Cost Prediction Results
The
contractor should demonstrate that the parametric cost estimating relationships
used can predict costs with a reasonable degree of accuracy. As with the use of
any estimating relationship derived from prior history, it is essential in the
use of parametric CERs for the contractor to document that work being estimated
is comparable to the prior work from which the parametric data base was developed.
9-1004.5 System Monitoring
The
contractor should ensure that cost-to-noncost parametric rates are periodically
monitored in the same manner as cost-to-cost rates and factors. If a CER is
validated and will only be used in a one-time major new pricing application,
rate monitoring capability is not essential (see 9-1004.2). However, if it is
expected that the rates should be considered as an ongoing estimating
technique, CER monitoring is critical. The contractor should revalidate any CER
whenever system monitoring discloses that the relationship has changed.
9-1005 AREAS FOR SPECIAL CONSIDERATION IN
PARAMETRIC COST ESTIMATING
9-1005.1 Parametric Estimating for Change
Orders
Change
order pricing using parametric cost estimating relationships may need to be
considered in a different light than initial contract pricing actions. The
contractor may use cost estimating relationships which are unique to change
order proposals. In general, contractors do not segregate costs separately for
individual change orders Therefore, it is important that the contractor have a
system in place to validate, verify, and monitor CERs unique to change orders.
However, if the CER was applicable to the basic contract and change orders, the
CER could be validated without cost segregation.
9-1005.2 Forward Pricing Rate Agreements
a. Contractors
may submit proposals for forward pricing rate agreements (FPRAs) or formula
pricing agreements (FPAs) for parametric cost estimating relationships to
reduce proposal documentation efforts and enhance government understanding and
acceptance of the contractor's system. Government and contractor time can be
saved by including the contractor's most commonly used CERs in FPRAs or FPAs.
(See FAR 15.809 for basic criteria.) However, such an agreement is not a
substitute for contractor compliance at the time of submitting a specific price
proposal. FAR requires that the contractor describe any FPRAs in each specific
pricing proposal to which the rates apply and identify the latest cost or pricing
data already submitted in accordance with the agreement. All data submitted in
connection with the agreement is certified as being accurate, complete, and
current at the time of agreement on price on each pricing action the rates are
used on, not at the time of negotiation of the FPA or FPRA (FAR 15.809 [d]).
b. Key
considerations in auditing FPRA/FPA proposals for parametric CERs follow:
(1) FPRAs/FPAs
do not appear practicable for CERs that are intended for use on only one or few
proposals.
(2) Comparability
of the work being estimated to the parametric data base is critical. FPRA
proposals for CERs must include documentation clearly describing circumstances
when the rates should be used and the data used to estimate the rates must be
clearly related to the circumstances.
(3) Validation
of all the parametric criteria (9-1003) is especially important if a single CER
or family of CERs is to be used repetitively on a large number of proposals.
9-1005.3 Subcontract Pricing
Considerations
a. FAR
15.806-2(a) requires that when a contractor is required to submit certified
cost or pricing data, the contractor will also submit to the government
accurate, complete, and current cost or pricing data from prospective
subcontractors in support of each subcontract cost estimate that is:
(1) $1,000,000
or more,
(2) both
more than $100,000 and more than 10 percent of the prime contractor's proposed
price, or
(3) considered
to be necessary to adequately price the prime contract.
Use of parametric CERs does not relieve a
contractor of its responsibility to disclose planned subcontract procurements
and the related subcontractor cost or pricing data.
b. When
proposed material costs are based on parametric estimates the contractor must
demonstrate that the type of materials required for the proposal are the same
as included in the CER data base. The auditor should perform audit procedures
to determine if:
(1) materials
included in the CER data base are not estimated separately in the proposal, and
(2) adjustments
have been made to the CER data base for those items which were previously
manufactured in-house and now are being purchased. If the CER data base has not
been adjusted, the contractor should provide a detailed cost estimate for
purchased materials.
c. The
contractor should explain any major differences between parametric estimates of
subcontract costs and the subcontractor's quoted price and provide the
rationale for using the parametric estimate instead of the quote.
d. Consistency
in subcontract cost estimating must be maintained within the contractor's
estimating system. Any significant deviations from normal practices in the
proposal must be identified and justified by the contractor.
9-1005.4 Parametric Estimating Efficiency
a. A
primary justification for using parametrics is reduced estimating and
negotiation costs. Contractors should perform a cost-benefit analysis before
implementing an elaborate parametric estimating model. Their analysis should
show that implementation and monitoring costs do not outweigh the benefit of
reduced estimating costs. In many instances, new reporting systems may have to
be developed to provide reliable noncost independent variables. In addition,
the costs of CER validation and monitoring may be substantial.
b. When
the contractor's cost-benefit analysis indicates that the parametric system
implementation costs might outweigh the benefits of reduced estimating costs
and/or increased estimating accuracy, the matter should be pursued for
potential cost avoidance recommendations.
9-1005.5 Data Base Adjustment
Considerations
a. One
basic criterion (9-1004.4) is that the parametric data base be comparable to
work being estimated. However, a contractor may have to adapt a partially
comparable data base to its cost history using a "calibration" factor
An example would be an adjustment to the data base to estimate the savings as a
result of continuous improvement initiatives such as TQM. The utilization of
complexity factors and/or adjustments to modify contractor developed in-house
CERs is a valid technique. However, the use of such factors or adjustments
should be fully documented and disclosed. In addition, this approach increases
the contractor's burden to document compliance with the other criteria.
b. If
a contractor does not support the adjustment factors, the contracting officer
should be promptly notified (see 9-1005.7). In addition, the auditor should
determine if a qualified or adverse opinion is required. The audit report
should disclose the costs associated with the unsupported factors.
9-1005.6 Contract Administration Interface
a. Upon
receipt of a request to review a price proposal, the auditor will coordinate
with the Plant Representative/ACO to make arrangements for any needed technical
reviews of the proposal. Because of the special nature of cost-to-noncost
estimating relationships, and the possibility of limited cost history and added
audit testing, complete coordination is especially important when parametric
estimates are involved.
b. While
the auditor will address special areas of concern as requested by the PCO
and/or the Plant Representative/ACO, the audit scope will be established by the
auditor in accordance with the auditing standards, unless the PCO only requests
a review of part of a price proposal.
c. Auditors
should be available, on request, to explain applicable price proposal criteria
and identify any prospective audit concerns to both government and contractor
personnel. An example of such audit advice would be to identify operating
reports or records that have not been previously used to forecast costs and
would therefore require added contractor support and audit testing. Such
advance coordination will help avoid unnecessary contractor system development
costs.
9-1005.7 Reporting of Estimating Deficiencies
All
proposal and estimating deficiencies found during the review of the parametric
estimating technique should be immediately reported to the Plant
Representative/ACO. These may include incorrect, incomplete, or non-current
data and use of inappropriate estimating techniques. When a proposal evaluation
discloses estimating system deficiencies, a separate report entitled
"Estimating System Deficiency Disclosed during Evaluation of Proposal No.
XXX" will be issued immediately after the proposed audit report.
9-1006 ESTIMATING STANDARDS
9-1006.1 Distinction Between Estimating
Standards and Parametric Cost Estimating
a. In
terms of historical evolution and sophistication, the terminology of estimating
standards as covered in this paragraph might be viewed as falling between
traditional cost-to-cost estimating rates and factors and the more advanced
types of parametric estimating systems (see 9-1002). However, a contractor may
elect to use any combination of these evaluating methods, perhaps in the same
proposal.
b. Estimating
standards are normally developed through the use of motion time-measurement
studies performed by industrial engineers. Parametrics, on the other hand, are
developed by relating historical costs to one or more noncost drivers. While
estimating standards usually represent cost-to-noncost relationships they have
traditionally been limited to narrower or more discrete elements of estimated
cost than may be the case in more complex parametric CERs. Also, the logic of
the estimating relationship and the appropriateness of the mathematics in
estimating standards will usually be readily apparent.
c. Estimating
standards will not necessarily require validation under the criteria for
parametric cost estimating relationships contained in 9-1003 and 9-1004.
Especially when such standards (e.g., hours/pound, hours/drawing, hours/page)
have been in place and accepted by government personnel, the evaluation
guidance in this paragraph will likely be sufficient.
9-1006.2 Use of Estimating Standards
a. Estimating
standards may be established by relating engineering and/or production costs
(efforts time, and/or materials) to specific characteristics of a product such
as composition, weight, size or duration. This approach is designed to save
estimating effort and has been used frequently in estimating construction costs
and costs of recurring job orders such as printing. Many contractors use the
technique in shop-order budgeting and production control.
b. Estimating
standards may be used to estimate the cost of a single material item required
for the work, or the cost of a single labor operation, for example, welding
electrodes per ton of structural steel, press operations time per page, or
guard service costs per week. More complex, composite standards may be used to
estimate costs of groups of components or broader classes of labor operations.
c. Use
of estimating standards may be appropriate in contract cost estimating
situations when there is a close correlation between an amount of production cost
and the related product or process characteristic. The data sets being
correlated must have been measured in a uniform manner. The cost data used
should be verifiable by reasonable means. The units of measure used for base
characteristics should be uniform and readily identifiable; the quantity or
value of a characteristic should be readily determinable. Standards may be
derived from industry-wide statistics but should be relevant and verifiable to
the experience of the particular contractor using them.
9-1006.3 Applicability to Price Proposals
Traditionally,
estimating standards have been used to estimate costs in lump sums, often
including supervision, indirect costs, and occasionally general and
administrative expense. To comply with the SF 1411 and cost accounting
standards the contractor will normally have to factor the estimate to identify
the costs by cost element or function. Alternatively a proposed cost based on
an estimating standard might qualify for submission as an "other"
cost element if the cost can be tracked as such and is a relatively minor pan
of the total proposal.
9-1006.4 Audit Procedures
a. Depending
on materiality and risk of the costs estimated, the auditor should examine the
development and application of estimating standards to determine whether their
use is proper in the circumstances. Evaluate all cost and noncost data
applicable to each significant estimating standard and determine whether the
data has been properly used in the computations. Assure that the measurements and
correlation are adequate for the purpose. Determine whether the basis for the
standard (for example, the product mix, production rates, and production
methods) is sufficiently similar or comparable to that contemplated in the
estimate at hand.
b. When
changes are contemplated in the design or production of an end item or the rate
or method of production, the contractor's adjustments of the estimating
standards require special scrutiny. Review by government technical specialists
may be necessary in this situation.
c. During
audits of historical costs, sufficient information may be readily available
from which the auditor could develop estimating standards to use as one means
of appraising recurring contractor estimates. However, this will not substitute
for audit review of cost estimates as submitted by the contractor.
Superintendent
of Documents Order Form
The total cost of my order is $________.
Price includes regular shipping and handling and is subject to change. International
customers please add 25%.
_______________________________________________________
Company or personal
name (Please type or
print)
________________________________________________________________________
Additional
address/attention line
________________________________________________________________________
Street Address
________________________________________________________________________
City, State, Zip Code
________________________________________________________________________
Daytime phone
including area code
________________________________________________________________________
Purchase order number
(optional)
Check method of
payment:
__________________________________________________________________
Authorizing signature
Mail to: Superintendent
of Documents
P.O.
Box 371954, Pittsburgh, PA 15250-7954
|
BACK TO 100 examples in Business, Operations and
Engineering. |
|
Do it once, do it right, and do it now. |
|
Back to Lawson Computing Homepage |
Apply as needed,
when needed. |
