Knifing Paste Filler

I am trying to finish a box which had counter sunk screw holes that hadn't been filled up.
So I bought this knifing paste filler (grey colour) to fill up the holes and other scratches.
The filler hardened and smoothed perfectly after a bit of sanding. Problem now is that it dosn't take the stain and the entire surface of the box sides look blotchy. I guess I screwed up pretty badly somewhere. Would be grateful for any ideas on how to fix this - preferably without having to gouge out the filler which has set pretty good. Should I have done something else for the holes? Thanks.

Filler always will take up the stain more than the wood, you could glue in some contrasting wood to make it a highlight instead.

I'd probably just cover the side with veneer.

Thanks. That's a good idea. Will try and cut out small pieces of veneer to go over the filler.