Regardless of what it is - star, planet, comet... I'd say if it is too small to see with the naked eye, it won't have the mass/gravity to make a difference.
I know I've gotten on some other people in other posts about this, but honestly I think your mistaken here Rob.
If this thing were somewhere between 1 AU -2 AU out in space, and is a brown dwarf (a black dot in a black sky) that is creating a gravitational lensing effect then only the very best INFRARED telescopes would be able to see it. No matter if it is the size of Pluto, Jupiter, or 100x the size of Jupiter. Now I don't know how many stars you see in the night sky when you go out and stare at the night sky, but obviously you see millions in some places on Earth (if your away from city lights). How many planets can you see with the naked eye though? Those things are big, Jupiter can be seen, but only because it REFLECTS sunlight, and how big is that to the naked eye in the night sky? Planet X, Y, Z, whatever you call it, would not reflect any light, from any star, especially that far away from any star. Additionally think of things like Halley's comet. We know it to be on a 76 year orbit, and we sure as shit can't see that with a naked eye, but it's out there buzzing around, and sooner or later...it too will again be visible to the naked eye. But only for a few weeks, not months, not years, weeks. It flies in, and it flies back out of our solar system.
Honestly guys, this thing could be 10x the size of Jupiter, be hanging out on the edge of the solar system in the Oort cloud amongst millions of other objects, and we would never ever see it unless we were looking really really hard, and only with the very best of telescopes, probably infrared only, until it gets really close. Obviously if it's going to affect us here on Earth in any major ways though it'll need to come in much closer, and sooner or later it will be visible to the naked eye (if it is out there and coming in). But it would need to be a LOT closer for it become visible to the naked eye, as it would have to reflect some visible light in order to be seen without infrared.
Also remember, objects appear larger as they get closer, so....the farther it is, the smaller it is, and therefore the harder it would be to see with the human eye.
Now just because it's 100,000,000,000 miles away right now, and it sure looks really really small doesn't mean that by the time it flies in close it doesn't look really really big, and do some serious damage.