For the first time, we assess episodic simulation in a patient with visual memory deficit amnesia, following damage to visual association cortices. Compared to control participants, the patient with visual memory deficit amnesia shows severely restricted responses when asked to simulate different types of future episodic scenarios. Surprisingly, the patient’s responses are more limited in cases where the scenarios require less reliance on visual information. We explain this counterintuitive finding through discussing how the severe retrograde amnesia in visual memory deficit amnesia limits the patient’s access to episodic memories in which vision has not been a focus of their life. As a result, we argue that the deficits in visual memory deficit amnesia continue to distinguish it from amnesia after direct damage to the hippocampus.