You won't see me cite Game Score very often, but for our purposes here, it's perfectly suitable.
A.J. Burnett, Career: 54.6 average game score, 17.2 standard deviation
Andy Pettitte, Career: 51.4, 17.2
A.J. Burnett, Yankee: 53.3, 17.0
Andy Pettitte, Yankee, 2007-2009: 49.8, 16.3
And, for the hell of it:
C.C. Sabathia, Yankee: 58.1, 17.8
For a little perspective, Felix's numbers this year came out to be 60.8 and 16.5. Felix in 2007, meanwhile, came in at 52.1 and 18.7.
A.J. Burnett has a reputation for being wildly inconsistent, throwing three ugly innings in one start and then spinning a shutout the next. That angle has, naturally, been played up in the postseason, as the"can't be sure what they'll get" out of one of their more shiny investments from game to game.
The people saying that last bit are correct. The point they're missing, though, is that the same thing could be said for pretty much every other pitcher in baseball. Starters are inconsistent. When so much depends on the weather, the opposing lineup, and the defense, it isn't realistic to expect a guy to go out and have the same game every turn. There will be variation. And though Burnett is far from being the most consistent pitcher in the league - Roy Halladay, for one, is more reliable - people need to get over the notion that his body is some sort of exceptional binary system that on any given day will either suck or excel. He made two starts lasting fewer than five innings this year, to Pettitte's three. He made five starts in which he allowed 6+ runs, to Pettitte's four. And 71% of his starts this year have generated a game score between 40 and 70, to Pettitte's 62%. Pettitte, of course, is just standing in as an example, as someone most "baseball people" will tell you is consistent. As it turns out, it seems he's not really much more consistent than the guy who used to wear rings in his nipples.
I hate the word "consistency" and all of its sister terms as people relate them to baseball players. A bit of inconsistency isn't a flaw, because consistency the way people talk about it doesn't exist. Some players will fluctuate around good results while some players will fluctuate around worse results. The sooner people can understand that, the better I'm going to sleep.
Note: I realize that standard deviation isn't the best way of looking at this. Other measures are being attempted!