You will not be allowed on H4 if H1(primary) is not residing in the US. H4 is a dependent visa, you will be turned away at the border and asked to get a B1/B2 visa. You might luck out once or twice but at some point they will deny your entry
Reason- H4 is a dependent visa, if Primary doesnt...