The Federal Transit Administration (FTA) has a vested interest in assisting grantees to maintain efficient and effective procurement systems as well as a legal responsibility to ensure that its grantees expend their funds in accordance with FTA regulations, the Uniform Administrative Requirements for Grants and Cooperative Agreements to State and Local Governments (49 CFR Part 18), and the contractual agreements between FTA and the local grantee. In order to carry out this responsibility, FTA has established an oversight framework that is comprised of grantee self - certifications, annual single audits conducted in accordance with OMB Circular A-133, and FTA oversight reviews, including the Procurement System Review (PSR).
The FTA regional offices annually perform a risk assessment of each grantee. The risk assessment is intended to identify the potential risk that the grantee poses to the overall integrity of FTA programs. FTA uses the results of the risk assessment to identify grantees that pose relatively greater risk and to focus its oversight resources on those grantees. The risk assessment process may result in an oversight review, including a PSR.
The PSR Guide is a reference document for FTA contractors and FTA staff who participate in PSRs. FTA contractors employ these guidelines for each PSR conducted. The Guide documents the process for conducting a PSR, describes the required procurement elements, provides the standard forms and checklists, and includes copies of key reference documents. The Guide has been developed to focus the PSRs and foster consistency in the process.
This section of the Guide provides a discussion of the objectives of the PSR and the approach to conducting a PSR.
The objectives of the PSR are to encourage and facilitate improved grantee procurement operations, foster the use of industry best practices, and assess the grantee’s compliance with Federal procurement requirements, specifically the requirements of FTA Circular 4220.1F and applicable statutory and administrative requirements. The PSR is designed to be a customer-oriented review that emphasizes mutual respect between FTA and the grantee.
FTA’s approach to conducting PSRs promotes the following FTA vision strategies:
The PSR consists of three phases:
The Assessment Phase is designed to establish an understanding of the grantee’s procurement system environment and to assess the grantee’s procurement system risk. The Contract Review Phase is designed to assess the grantee’s compliance with FTA Circular 4220.1F and other applicable regulations through review of a sample of procurement files. Based on the assessment of risk determined during the Assessment Phase, the PSR will determine the quantity of contract files to review. The Reporting Phase is designed to report the results of the review.
The PSR integrates best practices into a thorough compliance review. By reporting deficiencies, and by using best practices as samples of a satisfactory procurement system, the PSR approach will enhance acceptance of the findings, improve compliance with FTA Circular 4220.1F and improve procurement practices generally. This Guide presents the requirements of FTA Circular 4220.1F as concisely as possible, and provides procedures that should be adhered to as strictly as possible to achieve fair and uniform reviews of compliance. However, the Guide relies heavily on the professional expertise of the reviewer to make recommendations to the grantees regarding best practices.
The PSR begins with the Assessment Phase. This phase is designed to establish an understanding of the grantee’s procurement system environment, to assess the grantee’s procurement system risk, and to focus the scope of the next phase of the PSR, the Contract Review Phase, based on the assessment of risk. The results of the risk assessment will dictate the number and type of contracts to review in the Contract Review Phase. The systematic assessment of relative risk of grantees permits FTA and its contractors to allocate oversight resources where they are most needed.
The Assessment Phase looks at the organizational structure, staffing, management direction, and policies and procedures that define the grantee’s procurement system environment. An initial assessment of the grantee’s conformance with systemwide requirements of FTA Circular 4220.1F is made. In addition, trends in procurement types or outcomes that could indicate lack of attention to procurement requirements, such as sole source awards and protested procurement actions, are assessed.
The Assessment Phase involves review of documentation and interviews with grantee management and FTA Regional Office Staff. The grantee’s procurement system risk is assessed based on the following specific criteria:
The reviewer evaluates the grantee's procurement system risk and determines an overall risk rating of low, medium, or high for the grantee. The overall risk assessment rating is then used in conjunction with the size of the grantee to determine the number of contract files to review in the next phase (see Section I.3.a., Selection of Sample).
The review begins with a visit to the FTA Regional Office to review relevant documentation on file and to discuss the grantee with Regional Office personnel. The reviewer schedules the Regional Office Review with the Regional Office. Generally, this part of the review can be accomplished in one day.
The documentation that should be reviewed in the Regional Office includes:
Each Regional Office annually completes an FTA Grantee Risk Determination Form for each grantee. The FTA Grantee Risk Determination process assesses the grantee’s potential risk to the overall integrity of FTA’s programs and assists FTA to focus oversight resources on the grantees that pose relatively greater risk. The FTA Risk Determination process evaluates risk in several areas including procurement management. Grantees are ranked according to overall relative risk (low, medium, or high). FTA uses the results of the FTA Grantee Risk Determination process to identify grantees within the region for particular oversight reviews, including the PSR.
The reviewer obtains and reviews any oversight reviews or other audits or reviews that address procurement issues. If the grantee has already had a PSR, the files, report, and monitoring correspondence should be reviewed carefully to identify specific procurement requirements with which the reviewer found deficiencies and the actions taken by the grantee to correct the deficiencies. Likewise, the Triennial Review or Financial Management Oversight Review may have identified deficiencies with specific procurement requirements.
The Triennial Review, which is conducted every three years on each grantee regardless of risk, addresses procurement as one of approximately 21 review areas. The Triennial Review takes a broad look at the grantee’s procurement procedures for compliance with the requirements of FTA Circular 4220.1F. Financial Management Oversight Reviews, which are conducted based on the FTA Grantee Risk Determination process, look at the grantee’s internal controls with respect to the requirements of the Common Grant Rule and also addresses procurement. Findings of these reviews should be noted and addressed in the PSR to ensure that corrective actions have been implemented.
The reviewer meets with Regional Office staff including program oversight staff, legal counsel, the individual assigned responsibility for procurement issues (if such assignment has been made), and other Regional Office staff, as appropriate. Discussions should focus on the grantee organization, the results of the FTA Grantee Risk Determination process, and specific procurement-related issues and concerns.
Once the grantee has been advised by FTA that a PSR will take place, the reviewer should contact the grantee to obtain a copy of the grantee’s written procurement policies and procedures, organization charts, list of contracts, copies of job descriptions, copies of internal audit reports, and templates used. In some cases, copies of procurement policies and procedures and other documents may be on file in the Regional Office, if for example they were obtained as part of a recent Triennial Review. If the policies and procedures are on file in the Regional Office, the reviewer should verify with the grantee that they are current.
The review of grantee written policies and procedures provides a foundation for determining the grantee’s deficiencies with respect to Federal procurement laws and regulations. In this phase, the reviewer determines whether the written policies and procedures address FTA C 4220.1F and the Master Agreement, and whether they are consistent with them. The review of written policies and procedures may also identify areas where best practices advisory comments are appropriate.
Interviews with grantee management and staff (conducted during the Initial Site Visit) will further address the grantee’s policies and procedures, practices, measures, and initiatives with respect to specific requirements, including how the policies and procedures are conveyed to staff and implemented.
An Initial Site Visit is conducted as part of the Assessment Phase. The Regional Office may contact the grantee to schedule the Initial Site Visit or may request the reviewer to do so. A letter is sent to the grantee outlining the dates for the review and an agenda by either the FTA or the reviewer (see sample letter). Where time or circumstances do not permit, it may be more advantageous to combine the initial and second site visits.
The Initial Site Visit begins with an entrance conference with management. The reviewer presents the background and objectives of the PSR and discusses the review process and schedule. The reviewer should note that the review will assess the grantee’s compliance with the requirements set forth in FTA Circular 4220.1F and will report deficiencies found as well as the areas where no deficiencies were found. The reviewer may also advise the grantee of best practices and procedures currently used within the transit industry that could potentially improve the grantee’s procurement system. A sign-up sheet should be circulated to identify all attendees. A copy should be offered to all participants.
The reviewer will conduct interviews (either in person or by phone) with grantee personnel such as the following:
The appropriate personnel and questions to be asked of each will vary from small systems to large systems and will depend on each system’s organizational structure and assignment of responsibilities. The Interview Questionnaires (see Section III.1.) are only a guide and may be used as a checklist of topics to be discussed with personnel, as appropriate. Additional questions should be added where appropriate for clarification.
The interview questionnaires for general manager or CEO, head of procurement or highest ranking procurement official, and selected procurement staff should be applicable for any grantee undergoing a PSR. The questionnaires for the legal staff and the finance and/or grant staff may not be applicable depending on the size and particular organizational structure of the grantee. For example, if the grantee uses outside counsel rather than in-house counsel, all of the questions on the legal staff questionnaire may not be appropriate. The reviewer should determine how best to obtain the information and should schedule additional interviews, if necessary. Further, the reviewer may determine that additional interviews are necessary to obtain additional information as a result of issues discussed. If additional interviews are conducted, the questionnaires should be consulted for sample questions.
Several of the requirements in FTA Circular 4220.1F apply to the grantee’s procurement system as a whole, rather than to individual contracts. These elements include:
An initial assessment of the grantee’s system with respect to these elements is conducted at the Assessment Phase based on review of the grantee’s written policies and procedures and interviews with grantee management. The reviewer completes the Systemwide Elements Check List (see Section III.2.a.). During the Contract Review Phase, the grantee’s implementation of these requirements in specific procurements may be assessed. For example, if a contract reviewed during the Contract Review Phase involved a protest, the reviewer would examine the grantee’s resolution of the protest in accordance with its written protest procedures, and disclosure to FTA.
The Contract Review Phase is designed to assess the grantee’s compliance with FTA Circular 4220.1F and other applicable procurement regulations through review of a sample of procurement actions funded by FTA in the last two years. The Contract Review Phase includes selection of a sample of contracts for review, review of the selected contract files, development of preliminary findings, and the conduct of an exit conference with the grantee.
The selection of contracts to review is based on the procurement risk assessment rating determined during the Assessment Phase and the size of the grantee. The overall number of contracts selected for review increases with the relative risk and size of the grantee.
The reviewer requests that the grantee provide a contract listing for the review period (two years) by type of procurement and method of procurement. The contract listing should include all procurement actions and contracts awarded utilizing Federal funds during the time period, including contracts not yet completed. The reviewer should request the grantee to organize the contract listing into the following categories:
In addition, the grantee should identify architectural and engineering services and construction contracts. The grantee should also identify bus and rail vehicle
procurements made during the past five years. Since this is a procurement system review, if a sample of each of these types of contract actions is not available from actions using Federal funds, request a sample that includes non-Federal funds. Non-FTA funded contracts should be reviewed based on funding source and the grantee’s policies and procedures.
The Sample Selection Matrix (see Section I.5.) is used to identify the number of contract files to review for each method of procurement. The Matrix reflects two factors: risk level (low, medium and high) and size of grantee (small, mid-size, large mid-size, over 500 vehicles - bus only, and buses/rail). Using the Matrix, the reviewer determines the number of contract files to review for each method of procurement.
Once the reviewer determines the number of contracts for review, the reviewer selects the contracts from the grantee’s contract listing. The reviewer should request the complete contract file, which includes the purchase request, solicitation, evaluation, selection, cost or price analysis, negotiations, award, contract administration documentation, and any other documentation that provides a history of the procurement.
If the initial sample selected by method of procurement does not yield any architectural and engineering services, construction, or rolling stock contracts, the reviewers should select additional contracts for these types of procurements shown in the Matrix.
In addition to the contracts selected using the Matrix, the reviewer may supplement the sample with specific procurement files that have been identified by Regional Office staff, size of the purchase, or based upon recent newspaper articles or internet-based documents.
The Matrix is intended as a guide for reviewers in selecting the sample size. In some circumstances, the reviewers may want to increase the recommended sample size. For example if the reviewer selects a request for proposal contract file and identifies deficiencies in elements specific to the request for proposal check list, such as evaluation, or adequate solicitation, the reviewer may want to increase the sample size to obtain further documentation to support the findings.
The reviewer will review each contract file using the appropriate Review Findings Check List developed for the specific method of procurement (see Section III.2.). The Review Findings Check Lists include requirements specific to the method of procurement as well as requirements applicable to all types of procurement. Individual Review Findings Check Lists have been developed for:
Additionally, Supplemental Review Findings Check Lists have been developed for:
A separate Review Findings Check List is used for each contract file reviewed. When reviewing an architectural and engineering services or construction contract, except for micro-purchases, the reviewer will use the Review Findings Check Lists appropriate for the method of procurement and the applicable Supplementary Review Findings Check List. The Micro-Purchases Review Findings Check List includes construction contracts between $2,000 and $3,000.
When reviewing contracts for rolling stock, the reviewer will use the Review Findings Check List appropriate for the method of procurement and the appropriate Pre-Award and Post-Delivery Review Requirements and Check List form (see Section III.2.i. or III.2.j.). The reviewer will complete this form for all rail and bus procurements selected for review during the past five (5) years.
In addition, the reviewer may encounter facts or circumstances involving options or change orders that warrant their review using the noncompetitive negotiation (sole source) check list. Where this is the case, the reviewer should conduct these reviews in addition to the planned sample of contracts, and should add these reviews into the tabulations as if they were separate procurements.
Each statement or set of statements in the Review Findings Check List corresponds to an individual procurement element. Deficiencies will be determined for each individual procurement element (see Section II. for a listing of Required Procurement Elements). In other words, the reviewer must enter either “Not Deficient,” “Deficient,” or “NA” (NA may be used when the area is unshaded) for every numbered element in the check list. If the complete statement is true, the grantee is “Not Deficient” with the requirement of that element. If any part of the statement is not true, the grantee is “Deficient” with the requirement of that element. The reviewer must enter the method or basis of determination for each numbered element in the comment section of the check list.
Discussion with staff should supplement the document review to complete the Review Findings Check Lists. If the procurement file documentation appears to be incomplete,
the reviewer should discuss open issues with staff and try to resolve any unanswered questions.
After completing the contract file review, the reviewer compiles the preliminary results of each individual procurement element using the Contract File Summary Sheet (see Section I.6). The spreadsheet lists all of the individual procurement elements along the left side of the spreadsheet. Across the top of the spreadsheet the reviewer records the appropriate data for the purchase orders or contracts reviewed. Under each purchase order or contract, the reviewer enters a numerical value of “1” in the applicable “Not Deficient”, “Deficient”, or “Not Applicable” column for each procurement element of that procurement type (RFP, IFB, etc.). The software will add the numerical values for each column and provide totals at the bottom of the spreadsheet for the procurement action. For each individual procurement element, the far right columns of the spreadsheet provide totals of the number of procurement actions that were determined to be “Deficient,” “Not Deficient,” or “Not Applicable.” For these elements, the spreadsheet provides the percentage of procurement actions reviewed that are determined to be “Not Deficient.” The reviewer evaluates the spreadsheet results and determines deficiencies for individual procurement elements, not for individual contract files.
The reviewer determines deficiencies for each systemwide and individual procurement element. The reviewer determines deficiencies for systemwide elements based on the results of the Systemwide Elements Review Check List. Systemwide elements are requirements that apply to the procurement system as a whole. The reviewer determines deficiencies for each individual procurement element based upon all the contract files reviewed.
The reviewer completes the Report Summary Table (see Section III.3.b.) for each individual procurement element by totaling the number of not deficient, deficient and not applicable items from the Contract File Summary Sheet. The reviewer uses the level of findings guidelines described below to determine overall compliance for an individual procurement element. The reviewer also states the applicable requirement from FTA Circular 4220.1F and the required corrective action.
Two levels of findings are used:
The reviewer discusses the preliminary findings with Regional Office staff participating in the review or with responsibility for oversight within the Regional Office.
The site visits conclude with an exit conference with the reviewer, Regional Office staff, and grantee staff. The reviewer thanks the grantee for their cooperation, highlights the grantee’s best practices, presents the preliminary findings and presents the schedule for issuance of the draft and final reports. The exit conference is an opportunity to confirm that the reviewer has not overlooked any information in reaching determinations regarding deficiencies, and, to the extent practical, to suggest possible corrective actions. The grantee is also asked to develop an implementation schedule for the suggested improvements that will be proposed later when responding to the PSR report.
At the conclusion of the exit conference, the reviewer provides a copy of the Contract File Summary Sheet to the appropriate grantee officials. The spreadsheet should be marked “DRAFT” so that needed minor adjustments may be made after the exit conference as a result of further evaluation and analysis. An Excel spreadsheet is used to provide additional detail.
Reporting of the procurement system review includes:
Preparation of the draft report
Receipt of FTA comments
Preparation of draft final report
Receipt of Regional Office and grantee comments
Preparation of final report
Completion of work paper files
After the conclusion of the site visits, the reviewers prepare the draft report summarizing the findings for systemwide and individual procurement elements identified in Section II of this guide. The findings are based on the Assessment Phase and the Contract Review Phase. The primary source of information includes interviews with staff and review of contract files using the Review Findings Check Lists.
Systemwide procurement elements refer to the requirements that apply to the procurement system as a whole and the reviewer determines whether the grantee is deficient or not deficient with respect to each element. Individual procurement elements are applicable to one or more methods of procurement and assessed through the review of individual contracts. For each individual procurement element, the reviewer determines whether the grantee is deficient or not deficient with respect to the procurement requirements in FTA Circular 4220.1F (see Section II.2 for definitions of required procurement elements). For deficient elements, best practices and suggestions on how to corre3ct the deficiencies should be provided. The reviewer also provides suggestions on how to improve the procurement system. These are advisory comments and are not specifically related to a procurement element such as organization structure, filing systems, training, and synchronization of staff.
The reviewer prepares the initial draft report and submits it to the FTA for review within 30 days of the conclusion of the site visits. The reviewer sends copies of the initial draft report to the Regional Office, the Office of Oversight, and the Office of Procurement for review and comment. FTA comments are then transmitted to the reviewers to be incorporated into the final draft report before the Regional Office sends the final draft report to the grantee. The final draft report requests the grantee to submit a corrective action plan and schedule within 30 days of receipt of the final draft report. The corrective action plan and schedule may include the development of policies and procedures, additional training for personnel, or other corrective actions.
A sample report format is included in Section III.3.a. of this Guide.
Within 30 days of receipt of the final draft report, or as otherwise agreed upon by FTA and the grantee, the grantee is required to provide comments to the Regional Office and/or reviewer. In addition, the grantee must develop a corrective action plan and schedule for each deficient element and must submit the plan to the Regional Office. The corrective action plan should include sample actions that the guarantee will take, including submittal of revised policies and templates, evidence of training, and evidence of implementation.
The reviewer analyzes the grantee comments and proposed corrective action plan and schedule. If the corrective action plans and schedules are satisfactory, the reviewer incorporates and addresses (as appropriate) the grantee comments and corrective action plans and schedules in the final report. If the corrective actions and schedules are unsatisfactory, the Regional Office and the reviewer discuss more appropriate corrective actions or schedules before the final report is prepared and transmitted. Within 14 days of receipt of grantee comments, the reviewer sends copies of the final report to the Regional Office, the Office of Oversight, and the Office of Procurement. The Regional Office transmits the final report to the grantee. In addition, the reviewer will also work with the grantee to complete as many corrective actions as possible within 30 days of the date of the last site visit.
The reviewer organizes the work papers according to the Review Findings Check Lists (see Section III.2.). The work papers include Assessment Phase interview notes, individual check lists, and other supporting documentation necessary to substantiate the conclusions of the report. While the work papers contain all the detailed information necessary for future FTA reference, the final report transmitted to the grantee is a more concise, balanced document, free of the cumulative contract-by-contract detail. The work papers include documentation obtained during the review not already on file in the Regional Office. The reviewers submit the workpapers to the Regional Office.
The reviewer shall enter findings into OTRAK within 30 days of the submission of the final report to the Regional Office. This data consists of grantee name and location, dates of Regional Office desk review and site visit(s), and the findings for each review item. The reviewer shall coordinate this effort with the Regional FTA Procurement System Review Coordinator.
SAMPLE SELECTION MATRIX
Over 500 Vehicles
Over 500 Vehicles
|Sole Source||1||Sole Source||1||Sole Source||1||Sole Source||1||Sole Source||2|
|Sole Source||2||Sole Source||2||Sole Source||2||Sole Source||2||Sole Source||3|
|Sole Source||3||Sole Source||3||Sole Source||4||Sole Source||4||Sole Source||5|
|Additional Contracts, If Necessary*|
* If the initial sample selected by contract type does not yield any architectural and engineering services or construction contracts, the reviewers should select additional contracts using these guidelines. The sample sizes are minimums. The reviewer should use his/her discretion if a larger sample is needed.
DRAFT CONTRACT FILE SUMMARY SHEET DRAFT
| Contract No./Purchase Order No./Date
Name of Contractor/Vendor
|Element||ND||D||NA||ND||D||NA||ND||D||NA||Tot||% of Not Deficient|
|7)||Independent Cost Estimate|
|8)||A&E Geographic Preference|
|9)||Unreasonable Qualification Requirements|
|10)||Unnecessary Experience and Excessive Bonding|
|11)||Organizational Conflict of Interest|
|13)||Brand Name Restrictions|
|15)||Contract Term Limitations [Rolling Stock & Parts]|
|16)||Written Procurement Selection Procedures|
|17)||Solicitation Prequalification Criteria|
|18)||Award to Responsible Contractors|
|19)||Sound and Complete Agreement|
|20)||No Splitting [Micro-purchase]|
|21)||Fair and Reasonable Price Determination [Micro-purchase]|
|22)||Micro-Purchase Davis Bacon|
|23)||Price Quotation [Small-Purchase]|
Clear, Accurate, and Complete Specification
|25)||Adequate Competition - Two or More Competitors|
|26)||Firm Fixed Price [Sealed Bid]|
|27)||Selection on Price [Sealed Bid]|
|28)||Discussions Unnecessary [Sealed Bid]|
|30)||Adequate Number of Sources Solicited|
|31)||Sufficient Bid Time [Sealed Bid]|
|32)||Bid Opening [Sealed Bid]|
|33)||Responsiveness [Sealed Bid]|
|34)||Lowest Price [Sealed Bid]|
|35)||Rejecting Bids [Sealed Bid]|
|37)||Price and Other Factors [RFP]|
|38)||Sole Source if Other Award is Infeasible|
|39)||Cost Analysis Required [Sole Source]|
|40)||Evaluation of Options|
|41)||Cost or Price Analysis|
|42)||Written Record of Procurement History|
|43)||Exercise of Options|
|44)||Out of Scope Changes|
|47)||Time and Materials Contracts|
|48)||Cost Plus Percentage of Cost|
|49)||Liquidated Damages Provisions|
|51)||Qualifications Exclude Price [A&E]|
|52)||Serial Price Negotiations [A&E]|
|53)||Bid Security [Construction over $100,000]|
|54)||Performance Security [Construction over $100,000]|
|55)||Payment Security [Construction over $100,000]|
|Pre Award Certifications|
|Post Award Certifications|
ND - Not Deficient D - Deficient NA - Not applicable Tot - Total Contracts Reviewed