We hear a lot about the benefits of whole grains—and not just from the people who sell them to us. Getting people to eat more whole grains is also a major pillar of the Dietary Guidelines for Americans. We’re told that people who eat more whole grains are healthier—they have lower rates of heart disease, cancer, diabetes, obesity, and many other diseases. But this isn’t quite true.

People who eat whole grain products—such as whole wheat bread—instead of refined grain products—such as white bread—do appear to be healthier. But most of the benefit comes from the fact that they have reduced their consumption of refined grains. When people simply eat more whole grains without eating less refined grain, they don’t get the same benefits.

The fact is that most people do not need more grain-based foods in their diets. And while whole grains are definitely a better choice than refined grains, don’t let that “health halo” blind you to the following realities:

1. Whole grain foods are not that much lower in calories.

Many people assume that whole grain foods are significantly lower in calories than refined grains, but this is not the case. A slice of 100 percent whole wheat bread has approximately the same number of calories as a piece of white bread, for example. A cup of brown rice actually has a few more calories than a cup of white rice. The differences between whole grain pasta and white pasta are also minimal. The good news is that, even though it might not be much lower in calories, the whole grain option may help you feel fuller for longer—primarily because of the extra fiber.

2. Whole grain products are only slightly easier on your blood sugar than refined grains.

As a general rule, you want to avoid foods that cause a rapid rise and fall in your blood sugar levels. That’s one big reason that I suggest you limit your intake of foods that contain a lot of sugar. Starch, which is the main component of grains, can also cause a fairly rapid rise in blood sugar. And while whole grains cause a smaller rise than refined grains, the difference is not as great as you might imagine.

3. Whole-grain foods are not nutrient-dense.

We also hear a lot about how nutritious whole grain foods are. Although they are higher in some nutrients than their refined counterparts, the amount of vitamins, minerals, and fiber they provide is still fairly modest.

4. What is glycemic load?

Glycemic load refers to how a given food affects your blood sugar level. It’s related to the glycemic index, but I find glycemic load more useful because it also takes into account how much of that food you eat—which, as you’ll see in a moment, makes a really big difference.

The glycemic load of a small plate of regular (white) pasta as about 23, which is considered high. The glycemic load of a small plate of whole wheat pasta is only 15, which is moderate. A large plate of whole wheat pasta, however, has a glycemic load of 30, or very high. Here’s the point that I want to make: in terms of your blood sugar, a small plate of white pasta is better than a large plate of whole wheat pasta.

Choosing a whole grain option does not give you license to have a larger helping. It’s important to watch portion size with all grain-based foods—even whole grain foods.

5. What is nutrient density?

Nutrient density refers to how much nutrition a food provides for the calories. If one food provides the same nutritional value as another but has only half the calories, we say it is twice as nutrient-dense. For all the talk about the nutritional value of whole grains, you’d probably think that they are much more nutrient dense than refined grains—but they’re really not.

On a scale of 0 to 5, with 5 representing the most nutrition for the calories and 0 representing the least, most refined grain products, such as white bread, pasta, and rice are in the 2.5 to 2.8 range. The whole grain alternatives range from 2.9 to 3.3 or so—a bit better. Most vegetables, on the other hand, are way up in the 4.5 to 5.0 range.

Replacing a refined grain with a whole grain alternative offers a slight nutritional upgrade but nowhere near as big an upgrade as replacing it with an extra serving of vegetables.