Knitting Math makes my head hurt

When we last left the Rosemarkie Waistcoat - my Alice Starmore colorwork knitting project - we were just 6 rounds from the top.  But these six rounds are complex.  I had just dropped off the stitches for the back of the neck (on holder - marker is the center back).

So let me show what I'm having to keep track of right now.  Starting from the center front:

Front Steek (half)
V neck decrease  slanting / as you are wearing the finished vest - every third round
Front Right Shoulder in pattern
Right arm steek (no decrease, thank heavens)
Back Right Shoulder in pattern
Neck line decrease / - every round
Back neck mini steek  (just 6 rounds deep)
Neck line decrease \ - every round
Back Left Shoulder in pattern
Left arm steek
Front Left Shoulder in pattern
V neck decrease  slanting \ as you are wearing the finished vest - every third round
and the final half of the Front Steek. 

AND at the end of all of that, all of the shoulder bits HAVE to have the same number of stitches when I hit round 6 or I'll have to rip it all out and do it again.

Is it any wonder that this project has been SO slow?