It looks to me like it's the age of the story that's considered, not the age of the vote. Apparently, the stories only have their rank adjusted when a vote is made, so the solution was to have something in addition to voting drive the adjustment - in this case, randomly selecting a story.