on grove street north of the path station (a block above tania's previously recommended by the alpha hound) there is a bakery that also serves west indian food. (Actually the owners are from Antigua...)
they make an excellent coconut tart (I big thing that looks like an apple turnover with a coconut filling and a scone like (very short) exterior). I think the crust may have cinnamon in it - it has just a hint... Warning: I also got a stale one on another day - so be sure to ask what was made today... I also had some tasty coconut buns - like a scone - very short - with coconut.
They also make excellent "French" bread in the Carribean style. Its actually really terrible French bread if you expect French bread from France. But its exactly like the bread I've eaten in the Carribean in many places - extremely airy with a not too chewey crust. When you eat this bread you realize why the Cubans press the bread for Cuban sandwiches and Cuban toast.
I haven't been there around lunch - so I haven't checked out the full carribean menu - but I spied oxtail, curried goat and the usual suspects... Also patties come out around 11 - so I haven't been able to try a fresh one - I've only been there on the way to work.
Any other jersey city west indian recommendations? If I'm around one day during the day I'll check out the hot food here and report back.