Performance Measurement

It is a widely held perception that paratransit trips are unavoidably expensive to provide. Historically, when individual agencies provide transportation directly to their clientele, this has been the case. The high cost per paratransit trip is a result of not being able to maximize the use of vehicles and personnel by spreading the cost of a given trip over a greater number of passengers. A coordinated system provides the ability to link trips and passengers between various agencies and expands the pool of potential riders. This should serve to enable the provider to group trips more efficiently and decrease the per trip cost. These benefits can become apparent when performance measures are put in place to measure system performance and cost, and to compare the costs of providing services before and after coordination.

Public transit operators have traditionally established performance measures to evaluate the efficiency and effectiveness of their transportation services. Transit operators typically use such measures as operating cost per revenue vehicle hour, operating cost per passenger boarding, fare box revenue per operating cost, passenger boardings per revenue vehicle mile, and passenger boardings per revenue vehicle hour. These measures allow transit operators to assess the effectiveness of individual routes by volume, time of day, and day of week for their planning purposes. Many health and human service agencies provide transportation as a tangential service to their primary mission. These agencies have limited, if any, reporting requirements for their transportation operations and therefore do not typically have the same type of data available to assess transportation performance. As agencies cooperate to develop coordinated paratransit services, the development and adoption of appropriate performance measures and the data to support them are key to their ability to evaluate services and the benefits of coordination.

Performance measures as applied to paratransit services will incorporate many of the traditional measures of revenue hours/miles per vehicle/passenger. However, some agencies are broadening the way performance is measured, particularly because of the different nature of paratransit versus fixed-route services. Ride statistics such as total number of rides, number of rides denied, average miles per passenger trip and average ride time are being applied to gauge the impact of paratransit services in terms of improving transportation access. Paratransit providers are also beginning to measure their performance in terms of vehicle capacity, instead of the number of vehicles in their fleet, to reflect the mixed fleet used to deliver paratransit services.

The establishment of performance measures can further contribute to a coordinated approach to transportation service delivery in a number of ways. Performance measures allow paratransit administrators to assess system performance based on their established criteria, and compare that to past measures of performance and target goals. They also enable providers to calculate the benefit of coordination in financial terms and passengers served and further base their resource allocation decisions on that information. Finally, performance measures also provide data to support further advocacy for coordination at a local, state and Federal level through the illustration of cost-savings and improved services.

The following two examples illustrate how appropriate performance measures were developed to gauge paratransit effectiveness, how those measures were developed, and how the results are used to manage and improve paratransit operations. For additional information about performance measurement, please see "Planning Guidelines for Coordinated State and Local Specialized Transportation Services," Checklist of Transportation Planning Steps, Step 10.¹⁰

A. Sweetwater County, Wyoming - Measuring System Costs and Performance

STAR serves the sparsely populated area of Sweetwater County in Southwest Wyoming, a service area of 10,400 square miles. STAR was created in 1989 and replaced a number of health and human service agency-based transportation services to form a coordinated public transportation system providing transportation to the general public and to agencies on a contract basis. The transit agency uses a fleet of 15 vehicles to provide door-to-door services with no fixed-route operations. Today, STAR provides approximately 6,500 trips per month on an annual budget of roughly $500,000. Under STAR’s coordinated system, the number of trips being provided has increased fourfold over the number of rides previously provided under agency-based transportation services. To meet this demand, STAR has kept a close eye on the cost and performance of their system. This attention to system cost and performance ensures that they maximize the use of the scarce resources available by providing cost-effective services at a price below what agencies were formerly spending for transportation on their own.

STAR was born out of the need to provide public transportation to agencies and individuals where little service existed. In 1981, a number of Sweetwater County human service agencies applied for and received grants under the then Urban Mass Transportation Administration's (UMTA) Nonurbanized Area Formula Grant program (rural and small urban community transportation assistance) to purchase transportation vehicles. However, operating funds to support transportation services were not readily available through the agencies. In 1985, the Child Development Center (CDC), two local senior centers, the Youth Home, and the South West Counseling Service decided to submit one grant application covering all agencies for transportation operating funds and began to discuss the feasibility of the formation of a coordinated system. While the original grant was funded, the County Commission decided not to form a single transit agency at the time, but instead to have individual agency transit providers to operate as a coalition.

As part of the evaluation of the grant process, each agency provided information on the number of riders being served and the expenditures each agency was putting forward. In early 1988, the lead staff of the coordination effort worked with the senior agencies and developed a fully allocated cost analysis for providing transportation, which had previously not been done by the agencies. The cost allocation system enabled agencies to capture costs for items that had not been accounted for previously such as fuel, maintenance, and insurance which had, in some cases like the senior centers, been supplied by the county. After going through this process with each of the various agencies and gathering data for a number of consecutive years, staff calculated that the agencies were spending a combined $400,000 annually on transportation. This information helped to increase the support for coordination, and by late 1988, a County Transportation Committee was established to consider the formation of a single transit agency. The cost allocation system was used as a way to establish initial agency financial support for the proposed transit agency, based on past agency expenditures.

In 1989, the County Commissioners passed a resolution forming the regional transit authority. The agencies turned their vehicles over to STAR and operations commenced in May 1990. From the very beginning, STAR staff maintained detailed records of every trip provided by trip purpose, miles traveled, vehicle hours, and trip duration. Using operational data obtained from the individual agencies preceding the formation of transit authority, STAR staff established benchmark operating costs under the old agency operated system for the purposes of comparing these with STAR’s operations. STAR established an internal system to track three specific measures of system performance and three measures of system cost.

On a cost basis, staff used the trip data to establish measures of cost per passenger trip, cost per vehicle mile, and cost per vehicle hour. On the performance side, staff used trip data to track ride statistics including average miles per passenger trip, passengers per vehicle hour, and the average trip time. The availability of these performance measures has been critical to the transit agency’s ability to track and contain costs. From 1990 through 1995, STAR’s service was provided free to the rider, supported by contracts with the agencies, in addition to 5309 and 5311 money, State Impact Assistance Funds, local grants, and Community Service Block Grants. When ridership increased significantly, and staff discovered that more rides were being requested than STAR could possibly afford to provide, they used their performance measures to calculate a general public rider fare of $2 per trip to help partially cover the full cost of the trip. STAR staff is also able to use these performance indicators to establish contractual rates with local agencies as well as the private sector. Using these measures, the agency is able to illustrate that eight years after the transit agency began operations, the costs to the agencies are still less than prior to 1990, while the service has increased.

Today, STAR is able to boast of a door-to-door trip cost of roughly $7, and average passengers-per-vehicle-hour of 4-6 passengers, each of which are well below and above the national averages respectively. In a service area as large as some states, the establishment of computerized dispatch was critical to making these operating efficiencies possible. By careful coordination and dispatch, the transit agency has been able to achieve the high passenger-per-vehicle-hour averages that have been critical to their ability to remain financially viable. The decision to commingle clients was also a key element of efficient use of vehicles and resources, particularly considering the size of the service area.

In 1997, U.S. DOT sponsored a study¹¹ that estimated $1.6 million in annual savings to state and local governments resulting from the coordinated transportation system in Sweetwater County. These savings were calculated based on the detailed trip information, including trip type such as work, nutrition, medical, and education, maintained by STAR. Economic benefits were calculated in the six areas shown below, along with a brief description of the calculations applied to each area:

• Access to Employment - used the number of workers who are transit dependent and who would otherwise be subject to the loss of their jobs without transit, and calculated their annual wages as a benefit. The savings from welfare reductions were also calculated based on the number of persons recorded to have moved off public assistance through the use of access to jobs provided by transit.

• Access to Medical Care and Other Social Services - calculated benefit using a conservative estimate of the higher per trip cost for taxi service incurred by people who take those medical trips that would still need to be taken if transit were not available.

• Providing Rides to the School Age Children of Working Parents - calculated based on the hours of wage earnings that would be lost by parents transporting their children to the CDC if transit were not available to provide that service.

• Access to Shopping, Recreation, and Other Personal Services - calculated as the greater cost to make these trips than would be incurred using the transit service, assuming at least one-third of the trips would still be made.

• Access to Educational and Counseling Services - calculated based on the assumption that these trips will enable travelers to increase their long-term chances of employment. Using similar calculations to the Access to Employment above, the number of affected persons using transit are calculated, and their potential wages, in addition to potential savings through welfare reductions, are credited as benefits due to the transit system.

• Enabling the Continuation of Independent Living - based on detailed records of its riders, calculated as the number of persons able to remain living independently and out of nursing homes because of transit services. A percentage of the cost for those people to live in a nursing home is then used as an avoidance cost attributable to maintenance of independent living.

B. Pittsburgh, Pennsylvania - Establishing Equitable Agency Service Contracts Based on Systems Performance and Cost Data

In the Pittsburgh, Pennsylvania metropolitan area, ACCESS Transportation Systems provides ADA paratransit services under a contract with the Port Authority of Allegheny County (PAT), the public transit operator for the area. In 1978, PAT applied for an FTA Service and Methods grant with the objective of demonstrating that it is possible to contract with existing service providers for paratransit services instead of developing in-house paratransit services. Under the grant, PAT contracted with ACCESS as the broker for paratransit services. ACCESS has continued as the contract broker for PAT since the original grant, rebidding the contract every five years. ACCESS is now one of the largest paratransit providers in the nation, providing close to 2.5 million trips in the past year.

ACCESS serves as the broker using annual service purchase contracts with seven for-profit companies and three non-profit agencies, operating from 13 distinct facilities. Local riders have a choice of two to three providers in each area. ACCESS develops one-year contracts with the local health and human service agencies for the provision of transportation services. From the start, local health and human service agencies agreed to purchase service through contracts with ACCESS as opposed to directly from the provider. One of the keys to the success of the brokered system and the strong relationship between ACCESS and the contracting agencies has been the establishment of annual service contracts based on detailed system performance and cost data.

ACCESS developed a series of performance measures for their providers to evaluate their services. ACCESS established measures including those for:

• on-time performance
• complaints per 100,000 trips
• vehicle condition
• responsiveness
• target revenue passengers per billable hour by provider

As the system broker, ACCESS collects and maintains the performance data from each service provider, which is supplemented by road audit data provided by each of the providers on a monthly basis and ACCESS surveys of paratransit riders. ACCESS develops this information into system performance and cost measures for each provider and uses the information as a deciding factor in allocating trips. ACCESS will shift market share to carriers who can provide less expensive services, particularly trips provided for more cost-conscious agencies.

Cost information on a per trip basis is calculated using a specially developed model. The model establishes 198 fare zones, each roughly equivalent in size, and calculates per trip costs based on the number of airline miles between central nodes in the origin and destination zone, with a minimum of $7.50 per trip. This detailed cost information enables ACCESS to recover the costs for service provided based on origins and destinations. The cost information provides a basis for developing average unit cost estimates for each agency contracting with ACCESS for services. ACCESS takes a large sample of the trip information including trip length, trip purpose, and the performance measures discussed earlier, and develops an average trip cost for the agency based on actual trips taken by the agency’s clients. The agencies audit this information on an annual basis as part of the contracting process.

The contracting procedure begins when the agency estimates the level of service desired for the year. ACCESS’ two largest customer agencies are the Area Agency on Aging (AAA) and the Medical Assistance Agency (under the County Department of Federal Programs) responsible for administering Medicaid. Each agency’s contract is based on average unit costs, which take into account the parameters set by that agency for types of trips and trip length. Each agency sets its parameters by either statute, as with the Medical Assistance Agency, or by local policy, as is the case with the AAA. ACCESS then uses its average unit cost information to establish an annual contract fee based on the number and types of trips that will be provided. ACCESS has worked closely with local agencies to implement strict ADA eligibility guidelines to control costs. As part of that process, ACCESS provides eligibility screening services for the AAA, while the Medical Assistance Agency pre-screens for eligibility before referring riders to ACCESS.

The relationships between ACCESS and the contracting agencies have proven pivotal in maintaining the feasibility of the current system. Prior to ADA, ACCESS provided service to areas that were also being served by fixed-route transit. When the ADA was passed, customers did not want to be mainstreamed onto those fixed-routes. However, the Pennsylvania State Department of Public Welfare, responsible for Medicaid, drafted a mandate that required local agencies administering Medicaid to pay only the ADA fare, which would represent a large decrease in revenues to ACCESS. ACCESS worked to start a process of building consensus among its customers and local health and human service agencies in order to make an argument against passing the state mandate. The key to their ultimate success turned out to be local health and human service agencies’ willingness to provide more of the cost of ride versus the required ADA fare, a result of an understanding of the true costs of the service gained from the annual contracting process with ACCESS.