I figured it out. It turns out I didn't know how to tell the script which object to apply it to (the var in the first line).

Here's the correct script, in case anyone is interested:

myTextFrame = app.selection[0];

with(myTextFrame){

parentStory.storyPreferences.storyDirection =

StoryDirectionOptions.rightToLeftDirection;

}