To eliminate label background:

Layer.Params.Labels.Pattern := bsClear;
Layer.Params.Labels.OutlineWidth := 0;

To eliminate the color shadow around the label text (the halo effect), use:

Layer.Params.Labels.Font.Color = Layer.Params.Labels.Color

or just set the same color for both items, like this:

Layer.Params.Labels.Font.Color = GisSafeColor( mycolor )
Layer.Params.Labels.Color = GisSafeColor( mycolor )

GisSafeColor() will guarantee that color will be properly mapped, i.e., potential transparent color values will translated to non-transparent.