“First off” is another way of saying “first of all” or “in the first place.” It’s considered informal — and there’s nothing wrong with using informal terms or expressions. See Macmillan and Merriam-Webster. If those two sources acknowledge “first off” as an acceptable phrase (they do), well, that’s good enough for me. In sum, there’s nothing “hideous” about this perfectly acceptable phrase/expression.