4 is great. And it is a start. But honestly it can take considerably more fulls than that to sign. The GREAT news is you know your query is working. The quirkier the book--the less it fits into a cookie-cutter market niche (like cozy mystery or whatever) the more an agent has to be willing to stick his/her neck out to place it (after all they don't get paid until we do--before that they work for free). That does not, however, mean you won't find an agent. I also GET what you are saying--you know the audience and you just have to reach them. And either way--traditionally published or indie--you will have to work to reach that audience. Just make sure you talk to other writers who have done both and weigh your options. Again best of luck!

p.s. you said of the "non-form" rejections but I am sorry to tell you the line you quoted IS one of the standard form rejection lines. Remember unless you talk to an agent on the phone (rare but it can happen before an offer) or get a very specific rejection chances are what you are seeing is their standard language. That's because they are super busy and because what they don't want to have is someone contacting them for more insight (people do it, seriously) when that contact has not been solicited by them.