Jacoby Ellsbury is refusing to read the writing on wall