He became a recluse  and built his home in Gu hill （孤山）in Hangzhou Westlake.  He had no wife and offspring. He amused himself by planting plum and feeding cranes.  Hence, he acquired the reputation of  having plum as his wife and the crane as his son (梅妻鹤子）. 

His love of plum, Tao Yuanming love of Chrysanthemum, Wang Xizhi love of goose and Zhou Dunyi love of lotus are frequently illustrated in chinese arts and known as the four love (四爱）.