Beth Osborne, Author at Transportation For America

How to use performance measures: Performance measures for members, part IV

Beth Osborne

Performance measures should be used to prioritize and account for public spending and, through this, demonstrate to the public that their dollars are being used wisely. There are four ways to deploy performance measures: 1) create a dashboard, 2) prioritize projects, 3) optimize investments and 4) check on the performance of past investments.

This is the fourth post in a series on performance measures by Beth Osborne, T4America Senior Policy Advisor. Catch up on the series with a high-level overview of the concept, find out how and why you should go beyond the federal requirements and learn more about choosing the best measures for addressing your priorities. -Ed.

A dashboard

A visualization of data or metrics known as a “dashboard” can be a useful snapshot of current conditions or the direction an area is moving. Dashboards are useful because they are visual and quickly lay out in one glance the region’s goals and the current status. Two examples are Salt Lake City and the Virginia Department of Transportation.

Sample dashboard from VDOT

The development of a dashboard can help determine what the public wants to prioritize — the public has an easy-to-digest summary of core statistics about your transportation network to help them choose what’s important. But this tool has a notable limitation: it’s not responsive to those public priorities if it isn’t also connected to decisions about public spending. After all, you could have an elegant dashboard that’s easy for anyone to understand rating the safety of the roadway…while making no investments in roadway safety.

A dashboard doesn’t automatically make it clear that any investment decisions are being made to impact the statistics being measured or displayed.

Prioritizing projects

The State of Virginia recently decided to take steps to ensure that the public’s priorities were better reflected in their process of selecting projects. Their legislature passed a bill in 2014 requiring all new capacity projects to be weighed against each other based on five priority areas: congestion reduction, economic competitiveness, access to opportunity, safety and environmental protection. They created a process — now available to the public —for ranking these outcomes based on significant public input.

Optimize investments

Spurred on in part by funding uncertainty, The State of Tennessee took a step back and analyzed their entire list of planned projects with a critical eye. They looked closely at the expected outcomes of their planned projects and then searched for other less expensive ways to accomplish the same things. The results were quite impressive, with multi-million dollar projects replaced by projects that cost just hundreds of thousands, saving millions and still bringing 80-90 percent of the benefits of the more expensive project. (A presentation on their new process can be found here.)

How have past investments performed?

In the world of transportation planning where everyone is always looking forward, forward, forward, it can be incredibly useful to take a look backwards on a regular basis and see if past choices have met projections and brought the benefits promised — and recalibrate the process if they have not.

Transportation agencies do their best to project the outcomes of each investment, but no one can get it right all the time. The agencies that are making the most of their dollars are the ones that check back on their investments to see if they performed as expected and, if they did not, make changes.

To do this, the goals of each project must be clearly stated and then reviewed at major milestones to see if the expected results materialized. The public notices when we make promises that don’t come true, and being right there with them and explaining what will change with the process is an important way to build credibility. We must check our own work and continually improve our performance, especially if we want the public to continue to invest in our programs.

Using performance measurement to chase the right goal: Performance measures for members, part III.

If states and metro areas merely take up conventional measures without question, they may move in the wrong direction toward the wrong outcomes in the end. Instead, start from your ideal goal and work backwards to choose the appropriate measure to get you there.

This is the third post in a series on performance measures by Beth Osborne, T4America Senior Policy Advisor. Read the rest of the series anddive into a high-level overview of the concept, find out how and why you should go beyond the federal requirements, and learn how to demonstrate to the public that their dollars are being used wisely. -Ed.

What do you mean “conventional” measures? What sort of commonly accepted measures can send us in the wrong direction?

Beth Osborne, T4America

A fantastic, timely example can be seen in the latest release of the oft-cited but flawed Texas Transportation Institute’s (TTI) Urban Mobility Report which purports to rank metro areas by congestion, but is a case study in how a conventional, yet flawed, measure can lead you in the wrong direction.

There are numerous issues with TTI’s measure of congestion and we don’t cover them all here. Read our detailed public critique of the TTI report on the blog. -Ed.

When actual drivers talk about congestion, they tend to mean one of two things: bumper-to-bumper traffic or unpredictability. The latter really makes commuting a hassle — the lack of certainty about just how long today’s commute may take, with no way to adequately plan for it. (“Do I know how much time I need to allow today so I’m not late for work?”) It turns out that commuters understand that traffic will slow during rush hour, just like they know they will wait for a table at a popular restaurant on Saturday night.

TTI’s travel time index identifies congestion as any delay compared to the speed of traffic when the road is essentially empty. In other words, if people travel at 65 miles-per-hour on a particular highway where there is no traffic, any time the speed drops is considered delay, even if a slowdown to 55 mph is simply down to the posted speed limit. Yet, if you knew that you could travel at a reliable 55 mph each day for a rush hour commute (or 45 mph or 35 mph, even), wouldn’t you appreciate that predictability?

Most delay measures also only consider the experience of those driving. When the TTI report speaks of the experience of commuters universally, they’re actually only talking about driving, leaving out millions of others who commute each day by other methods.

Why is that distinction important? Let’s say the roadways are equally congested in two cities. In one city only half of the commuters experience it because they commute by transit, bike, foot or opt out entirely by telecommuting. In the other city, almost every commuter is traveling by car and is stuck in traffic each day. Do these two cities have the same congestion problem? According to the way TTI measures it, yes they do.

This is precisely how the wrong measure can put you on a path to the wrong outcome.

By defining and measuring congestion as only the change in speed at rush hour compared to the middle of the night and ignoring all other commuters, the “solution” becomes as limited as it is expensive: build enough roads and lanes so commuters in cars can drive as fast as they want — even above the speed limit — at all times of day no matter how many vehicles are on the road.

That narrow focus also means the community will be missing ways to encourage shorter trips, commuting by other less costly modes, and opting out of commuting entirely. If the community has economic development goals of attracting the young talent that is increasingly looking to locate in places with a wide range of transportation options, using a measure that focuses solely on driver delay will run at cross-purposes to those (and other) goals.

So how can we make sure we’re measuring congestion in a more beneficial way; in a way that reflects our residents’ concerns?

First, listen to your residents and stakeholders explain what it is about congestion that frustrates them. Understanding the source of their frustration will ensure your community is tracking the real problem.

Second, think about the point of view of all of your residents — roadway congestion may not be a frustration for all commuters. This means getting a more accurate picture of who is actually experiencing congestion day-to-day. After all, one way to address congestion is to remove drivers from the road, through carpooling, telecommuting, alternative work hours, helping people live closer to work or supplying affordable and convenient transportation options. Many of these solutions are cheaper than building more road capacity.

Congestion is not the only area where states and metro areas attempting to measure performance can mess up. One smart way to avoid these mistakes is to think deeply about the preferred, ideal outcome you’re seeking. Is it free flow conditions on every road at all times of day? Is it giving your commuters a predictable commute? Is it giving the highest possible number of people access to jobs within a 45-minute commute? Identify the end goal and find a measure that leads toward that goal.

We have more information about ways to do this in our report, Measuring What We Value. And let us know if we can help you explore these issues more deeply.

Don’t settle for the limited things Congress could agree on: Performance measures for members, part II

If states and metro areas don’t act now to establish their own priorities for their transportation system, they’ll end up only measuring what Congress deemed important in MAP-21. The time is now to start the conversation of what else also matters to the leaders and citizens in your area.

This is the second post in a series on performance measures by Beth Osborne, T4America Senior Policy Advisor. Read the rest of the series and dive into a high-level overview of the concept, learn more about choosing the best measures for addressing your priorities, and learn how to demonstrate to the public that their dollars are being used wisely. -Ed.

With federal performance measures rolling out, what happens next?

Beth Osborne, T4America

The transition to a new system of “performance measurement” represents an attempt by Congress to get a better sense of how our transportation system is performing nationally, to allow states and regions to be compared with against another, and to communicate with the public about what they are getting for their tax dollars.

With USDOT nearing the end of the rulemaking process for establishing new performance measures for our transportation dollars, attention will turn to state DOTs and MPOs which will soon need to establish accompanying goals for their transportation system in these limited priority areas set by Congress, including safety, infrastructure condition, air pollution and congestion.

Each state DOT and MPO will set a target for each of USDOT’s measures. For example, State A may currently have 700 highway fatalities a year and want to bring this down to 650. The state would announce that goal, describe which projects will help them attain it and then report back to USDOT and the public about whether they hit their target.

In the case of the safety and infrastructure conditions measures, if a state or MPO fails to hit their target then they’ll have to spend a minimum amount of funding in that area. In the case of the other measures, there is no specific implication or consequence if targets are missed. But the process should still help improve accountability and transparency for priorities and spending: the public will have the chance to help set those targets, scrutinize whether or not the projects being chosen are likely to help meet the state and/or regional goals and to hold leadership responsible for the results.

Should we do more? Why won’t these federal measures alone be enough for our state or MPO?

If you have other important priorities and big picture goals for your transportation dollars outside of the limited set of measures agreed to by Congress — and many of you do — you need to begin work now to establish your own system. If not, with all the time and attention going into Congress’ limited measures, they could easily overwhelm your other priorities not addressed by them.

If your other priorities are to get the same emphasis, they need to receive the same treatment, including a system for measuring and setting goals for those priorities. Failing to have your other measures in place could easily lead to a system where projects get funded to satisfy federal measures but neglect certain regional priorities or even do damage to them. For example, building a highway expansion to address auto congestion (federal measure) that cuts off local access to jobs in a commercial center (metro priority).

One example of how to ensure your priorities are in the mix

The Salt Lake City region has conducted extensive outreach to the public and stakeholders to identify goals for the region with excellent results. Through a (widely admired and emulated) visioning process called “Envision Utah” that engaged thousands of citizens in its feedback process, the booming Salt Lake City region looked at future challenges and considered different ways to grow, including the infrastructure needs associated with each vision. They developed several approaches and evaluated them against their valued priority outcomes, like protection of open space, household transportation cost, and disaster resilience — all measures that the federal performance measure system will not take into account.

By doing this a decade ago, the region chose a growth pattern that saved $4.5 billion in avoided infrastructure costs over 10 years. And the public involvement led to strong support and excitement for the eventual projects selected based on this process. Citizens see their views reflected in the vision, and feel included in the process — which, incidentally, makes it easier to raise new revenue to invest in transportation, as the state recently did.

Transportation is just a mechanism to reach your shared vision and goals, so focus on the goals first

What is particularly exciting about Utah’s approach is that it isn’t rooted in the notion that transportation is a separate thing; an end unto itself. Their analysis of transportation needs flow from the shared vision for the region overall and aren’t simply reactive to current traffic conditions. The region believes that transportation should be planned to support economic development in the area and that traffic flow alone does not equal economic development.

Remember almost no one travels just to travel. There’s always a destination in mind. The goal is to get to work in order to earn money or to get to school to pick up your kids or to get to the doctor for medical care. The end goal isn’t just to drive the designed speed of the roadway or never spend a minute in congestion, though that is often where traditional engineering standards can take you. On the other hand, it is not the job of the engineers to decide our values or choose the community’s broader goals and outcomes. That is the job of the political and civic leadership.

And performance measures are where that process happens.

Now is the time to start the conversations with stakeholders and the public to ensure all regional priorities are being considered and measures are chosen to address those goal areas. And if you want your regional and local priorities to be reflected in the state DOT’s performance management system, a discussion about how to align those priorities should occur before the federal rules are pushing states to implement the new system.

For more information, feel free to check out our report on performance measures, Measuring What We Value.

Credit where it’s due: With repair rule, the feds listened to public comment

In developing new standards for ensuring our roads and bridges are kept in good condition, officials at the U.S. DOT did something skeptics would find surprising: They really listened to public comment, and reflected it in the newly released rule.

T4America’s Beth Osborne

As we have noted here often, the 2012 transportation law (MAP-21) requires transportation agencies to begin using performance measures to govern how federal dollars are spent. The U.S. DOT is working to establish those metrics for safety, the state of repair, congestion, air emissions and other aspects of our transportation system.

State DOTs and metropolitan planning organizations (MPOs) will then set their own targets for areas. They then must show how their investment plans will help them reach the targets and report on the results. If they fail to make enough progress on say, road and bridge conditions, they would be expected to spend more in those areas.

Creating this brand new system from scratch is a challenge. DOT officials have to figure out which sets of data are truly valid measures and where the data come from; how much time and wiggle room to give states and MPOs in setting and meeting targets; and what happens when they don’t.

We at T4America and many of our allies howled last year when USDOT’s first proposed performance measure, on safety, allowed states and MPOs to fail outright on half of the measures, making the targets for states virtually meaningless. T4A and the Complete Streets Coalition responded with 1500 public comments saying that this was not good enough.

We are still waiting for the full rule on safety, but with the release of this second proposed rule on system conditions (i.e. bridge and pavement maintenance) USDOT has shown that they heard us on the question of how agencies will be held to account. The new rule proposes that MPOs and states must hit all of the required targets — 50 percent success is no longer a passing grade. And states must either beat the trends or, if their target is not as good as the trend line, they must hit their target. This is a substantial improvement. Considering the current condition of the country’s infrastructure, holding states’ and metros’ feet to the fire on state of repair is critically important. As Smart Growth America’s 2014 Repair Priorities report made clear, most states are still spending billions on new roads or expanding existing ones — while neglecting their growing repair backlogs.

Between 2009 and 2011, the latest year with available data, states collectively spent $20.4 billion annually to build new roadways and add lanes to existing roads. America’s state-owned road network grew by 8,822 lane-miles of road during that time, accounting for less than 1 percent of the total in 2011.

During that same time, states spent just $16.5 billion annually repairing and preserving the other 99 percent of the system. … [In 2011], just 37 percent of roads were in good condition that year—down from 41 percent in 2008.

Under the new rule, that kind of investment decisions and resulting diminishing performance should fail to pass muster in the future.

As someone who has worked for USDOT and is accustomed to the dense documents we sometimes produced, I was struck by the clarity and tone of this (still long and technical) rule. The impact of the public comments — including those provided by T4America and our partners — was clear. USDOT explains each issue they had to grapple with, what they heard from stakeholders on each issue, the principles they used to evaluate options, how each option performed in that evaluation and then their final choice. Reading the rule felt like having a frank conversation with the experts at FHWA writing the rule.

This is especially encouraging because the third rule on congestion (and other measures) undoubtedly will be the hardest and, in many ways, the most impactful. It includes measures that are newer to the federal program and can be defined many different ways. For example, is the goal of highway performance to keep traffic moving at the speed limit no matter how many cars are on it? Or is it to know that your trip today will take the amount of time you budgeted for it? If it is the former, we will have to spend a lot of money paving over a lot of places at marginal benefit to ensure a safe and efficient commute or delivery. If it is the latter, we can address the issue with a mix of more affordable operational improvements, emergency response and new capacity. In congestion, are we only interested in the speed of cars or do we give communities credit for letting their residents opt out of congestion entirely by taking transit, walking or biking?

One thing we now know for sure: USDOT is listening to the public, so we need to engage. We thank USDOT for the improvements and for listening. It is a heavy responsibility, and one the folks at the U.S. Department of Transportation executed very nicely.

There are a couple more ways they can improve the rule further, like making more of the process available to the public. I encourage everyone to comment on the current draft.

Author Archives: Beth Osborne