I spent a solid hour looking for this chest because I had it's location from Intel. You can't find it unless you finish the pirate leader hunt. I was on my way to the last pirate leader but this location was "on the way" so I thought I would knock it out on the way. Got frustrated. Eventually left after at least an hour of "I got this, I'm not looking it up." Looked it up and it showed it was where I originally thought the entrance would be. Only it wasn't there! Long story, longer; don't bother unless you finish the pirate killing. Wait, was it the 100 pirates or the leaders? Whichever you turn into the biwa master unlocks the location.
