I think Potter is problematic. He did a magnificent job at Brighton because he was given time and support to do it. No Premier League club will do that as they'll want pretty immediate results, something he struggled to deliver at Chelsea with ridiculous resources at his disposal; a well resourced Champ club would be ideal but he probably doesn't want to drop to that level.
His ideal is probably a club like Forest or Leeds, clubs with the resources to let him build and realistic ambitions to be top half of the Prem. It's going to be very difficult to find one of those that will give him time to deliver though - Brighton really was a case of perfect club, perfect time, perfect circumstances. Maybe he'll go back there when De Zerbi gets poached by a top European club next season.