Woohoo! Progress!!
Yes, take FDD out. Then take lid off the drive (you dont need to take any other parts off). Some drive lids held on with 1,2 or maybe 3 ver small screws, but some amiga drive lids are taped at the sides, if its one if those just peal that tape / silver foil off and youre in.
Use a damp cotton bud with alchohol and gently rub the heads from side to side around 10 times each. The bottom head is easy, top one usually tricker - dont bend it upwards, just lift it gently until you get cotton bud onto it.
Also clean any hair or dust out of the assembly while u are there.
If it was taped at the sides u can replace with selotape.