Those close to Prince worried about his use of painkillers for years