@GreatLor
Woah! Take it easy with the shifting! I take it that you're doing all the shifting to try to optimize your 2D/3D routine? Lets see if we can figure out why things are going wrong.
So, we have a 2D point (or vector if you prefer) at <10,10>. We want to rotate it clockwise 30 degrees (i.e. about the Z axis). To do this we need to multiply the x component by cos 30 and the y component by sin 30:
x' = x * cos(θ) - y * sin(θ)
y' = y * cos(θ) + x * sin(θ)
Now, the real problem here is that most Amigas don't do floating point math very quickly and sin() and cos() always return a value less than 1 so when converted to an integer they are always 0, hence x and y are always 0! This is not good! So how do we get around that?
One way is to use floating point math, but this is both difficult in assembly and slow in practice. Another way, and what I think you were trying to do, is to use fixed point math. This method scales all values by some constant so that the sin() and cos() values are not rounded off to zero. The formulas can be rewritten as:
x' = ((x * cos(θ) * SCALE) - (y * sin(θ) * SCALE)) / SCALE
y' = ((y * cos(θ) * SCALE) + (x * sin(θ) * SCALE)) / SCALE
Using a shift left is a quick way to scale a number by a power of 2. Shifting 7 bits scales it by 128 and is the same as multiplying the number by 128. If we choose a good scale factor and scale all of our operations by it then things will come out fine.
So lets look at our vector and try to write a routine in AMOS to rotate it 30 degrees:
Degree
Rem never change the originals!
X = 10
Y = 10
Rem Our scale factor
SCALE = 128
Rem Rotate THETA degrees
THETA = 30
Rem Scale the sin() and cos() values
_SIN = Sin(THETA) * SCALE
_COS = Cos(THETA) * SCALE
Rem Compute the new (scaled) values
_X = X * _COS - Y * _SIN
_Y = Y * _COS + X * _SIN
Rem Unscale the coordinates
_X = _X / SCALE
_Y = _Y / SCALE
Rem Here is the new point
Plot _X, _Y
Now, in assembly:
move.l X,d0 X-coord in d0
move.l Y,d1 Y-coord in d1
muls.w COS30,d0 X * cos(30)
muls.w SIN30,d1 Y * sin(30)
sub.l d1,d0 (X * cos(30)) - (Y * sin(30))
lsr.l #7,d0 Unscale X
move.l d0,_X New x-coord
move.l X,d0 X-coord in d0
move.l Y,d1 Y-coord in d1
muls.w COS30,d1 Y * cos(30)
muls.w SIN30,d0 X * sin(30)
add.l d0,d1 (Y * cos(30)) + (X * sin(30))
lsr.l #7,d1 Unscale Y
move.l d1,_Y New y-coord
rts
_X ds.l 1 Storage for x-coord
_Y ds.l 1 Storage for y-coord
X dc.l 10 Initial x-coord
Y dc.l 10 Initial y-coord
SIN30 dc.w 64 sin(30)*128
COS30 dc.w 110 cos(30)*128
Notice that since we've scaled the sine and cosine values, we don't need to scale the x and y coordinates. Doing so will cause problems. Also, everything must use the same scale factor. Trying to shift the sines and cosines by 14 bits and the coordinates by 8 bits will not work!
Anyway, I hope this helps. Let me know how it works for you.