Tricky situation, dude!
I've been in a similar situation myself, although that girl wasn't my boss.
We started out working together, and suddenly, we were flirting openly. I got myself together, and decided to stick with my girlfriend (whom I'm now married to).
Before I took that descicion, I decided that I'd never cheat on my girlfriend, and that I'd break up with her before getting into a new relationship. It'd been unfair to do otherwise.
As it turned out, I stopped flirting with the girl at work, and things returned to normal. We remained good friends until I got another job.
Not much advice, I know, but one thing is certain: You need to decide on how to proceed. Any action not thought through might have disastrous effects (especially since it's your boss we're talking about).
If it's just a sexual attraction, then it's probably just something that will pass (the grass is always greener on the other side, and all that). If it's emotional, then perhaps you should take a look at your relationship. Is there a reason behind this?
Anyway, I hope it all works out for you.
