Let's get three things out of the way right off the bat:
Now that that's out of the way, what is the likely net result of signing Jason Johnson to replace Scott Elarton in the role of fifth starter?
Well, it probably makes sense to say a few words about the role of the fifth starter for a contending team, which I consider Cleveland in 2006. He is to eat innings, protect the bullpen, and provide something as close to league-average performance as he can. Generally speaking, league-average performance is out of reach of any fifth starters. That is why they are fifth starters. As Earl Weaver is credited for noting, it is easier to find four good starters than five.
As a commodity, fifth starters are pretty fungible. You ought to be able to get a fifth starter pretty much off the street. Fifth starters can be young guys cutting their teeth, or injury reclamation projects, or old guys providing "leadership," but for the most part, fifth starter is a rollover commodity, and depending on him is a sucker bet. You get what you get. You invest in the other guys, not him.
Which is pretty much why Jason Johnson is here and Scott Elarton is not: anything more than a one-year commitment to your fifth starter is really overkill, especially if a franchise has any sort of pitching in the minor leagues. Once you call a guy up to start, he ought to get a regular turn: since it's hard to predict exactly when that guy is ready, it behooves a team to retain as much flexibility as possible, and that means one-year contracts for the fifth man. Options are fine. Elarton got two years guaranteed from Kansas City. Good bye, Scott.
Now look, this isn't to say that Scott Elarton did a bad job in 2005. On the contrary, he did a wonderful job. I mean, he wasn't an Ace or anything, but the man made 31 starts and broke the 180 IP mark, all with an ERA around league average (4.61). Sure, it was nothing to write home about, but this is the fifth starter, guys: we got 11-9 with a 4.61 in 180+ IP out of the Weak Link in the rotation. I'll take that any year. I'd take it this year, for example.
Am I likely to get it?
Well, let's take a look at how likely Scott Elarton was to PROVIDE it in 2006. Here are three guys to consider in 2003-2005, where any minor-league stats are translated into major-league equivalents, and the ERA is the "fair ERA" defined by Baseball Prospectus based on peripheral stats:
Guy One (ERA, WHIP/9, BB/9, K/9, HR/9, GB/FB)
2003: 4.76, 12.9, 3.2, 3.8, 0.7, 0.89
2004: 5.27, 14.6, 1.7, 4.5, 1.7, 1.24
2005: 5.73, 14.5, 3.2, 4.1, 1.5, 0.90
2003: 5.60, 14.4, 3.4, 4.6, 1.8, 1.05
2004: 5.50, 11.8, 3.0, 5.4, 1.6, 0.71
2005: 5.15, 12.4, 2.5, 5.1, 1.5, 0.67
2003: 4.85, 13.3, 3.7, 5.4, 1.0, 1.31
2004: 4.35, 12.0, 2.5, 5.3, 0.9, 1.67
2005: 4.63, 12.0, 2.1, 4.0, 0.9, 1.74
Now, none of these guys is going to set the world on fire. Guy Three is cool in that has made 32, 33, and 33 starts, racking up 189 2/3, 196 2/3, and 210 IP. Since his numbers barely move, one might say he is pretty consistent.
The other guys are pretty consistent too: consistently bad. In the PECOTA projections of these guys, they show a graph of where the guy's percentile ranking stacks up against the (projected) league-average pitcher and where his chances cross over the mythical "replacement player," who represents a guy you can get pretty much anywhere: AAA veteran, waiver wire, raw kid, Brett Tomko. League-average: not bad. Replacement player: bad. Guys One and Two have projection curves a run to a run-and-a-half above the league average hurler, making their replacement player crossover happen before the 50% mark. In other words, if they perform better than their projected level, they will be just about as good as a guy you can get off waivers.
It's not hard to see why: both guys are flyball pitchers with negative "stuff" who don't strike enough guys out to get away with extreme flyball tendencies. Guy One was lucky in 2004, holding hitters to a .274 batting average on balls in play (BABIP), while unlucky in 2005 (.320 BABIP). There's really not much difference between the 13-5 pitcher of 2004 in a pitcher's park and the 5-16 pitcher of 2005. Likewise, Guy Two had an preposterously low .241 BABIP at his primary stop in 2004, and a still-low .274 in 2005. Once he stops getting lucky, he'll probably turn into Guy One. (BTW, Guy One had a 0.74 GB/FB ratio in 2002, so it's 2004 that's the outlier.)
In other words, Guy Two (Scott Elarton) is very comparable to Guy One (Jose Lima). Do you want Jose Lima anywhere close to the Tribe rotation in 2006? (Hint: no.)
Guy Three (Jason Johnson) doesn't strike anyone out, ether, but his BABIP those three seasons are .311, .314, and .300, which is just on the high side of "normal." He's cut his WHIP/9 down each year by cutting his walk rate, and when he gives up hits, they're less likely to leave the park (a HR/9 of 1.5 for Elarton is truly bad). For those fans of trend analysis, Johnson's GB/FB ratios from way back: 0.69, 0.83, 0.92, 1.09, 1.18, 1.31, 1.67, 1.74. I'm not sure if that's especially meaningful, but to turn from an extreme flyball pitcher to a significant groundball pitcher is probably pretty weird.
Now, it's been floated that Johnson's stamina is in question, and that he slows down in the back half of the year. First of all, the man throws 200 innings a year and never misses a start, averaging about 6 1/3 IP per start last season. How much stamina are you asking for? Second, if Johnson falls apart (which I consider unlikely), then bring up Sowers for a few starts. I see no big problem here.
For what it's worth, Johnson's PECOTA percentile graph pretty much sits on top of the league average projection. He is Average Man.
And for a fifth starter, that's awfully useful.