Well, anyone on screen was possible to work and draft with. And unless you were at the very end of the E group, odds are the people you were around were from the E group.
So you were working with and benefiting from the draft, even if you were surrounded by D riders.
There was at least 2:1 and often 4:1 in E:D, so that group always had plenty of people to roll with. The only way you were solo was if you were truly separated on screen.
For reference, you can readily identify the group of any rider by the color and tag ID next to theirs name in the rider list. Then you can also see the color of the rider dot in the top right map section.