chalky grape sherbet

My concord grape and rosemary sherbet had a chalky, slightly gritty mouthfeel after I churned it and ripened it. I used 1 cup of fresh grapes that had been cooked down into a sauce, 2 cups of whole milk, 1/3 cup of sugar, 2 tablespoons honey, and about a teaspoon of fresh rosemary leaves. I heated the milk, sugar, honey, and a dash of salt on the stove until everything was dissolved, then added the grape sauce and rosemary leaves. The milk curdled slightly, but some recipe notes I found online said that this was ok and wouldn't affect the final product. I let the mixture sit overnight, then strained out the grape solids before churning. The sherbet had a great texture right after it was churned (I used a Cuisinart 1.5 qt machine), but became chalky by the next day (but was still soft, scoopable, and otherwise pretty and tasty). Any idea where I went wrong, and how I can prevent this from happening in the future?