we don't want to go out of our way to find a non white male just to tick a box
See, they should have gone out of their way to do this. Not just to tick a box, but because ticking that box is a kind of due diligence towards building a more inclusive community. Unless they're ready to claim that there's no interesting non-white, non-male speakers able or willing to speak at their conference, then the failure is theirs to make any effort at all. But if they did make the effort and failed to attract someone, then they can say "we actually tried, and couldn't do it." The effort need be no more than to send a bunch of emails, and if that fails, it demonstrates a deeper problem in their community than they can fix.
See, they should have gone out of their way to do this. Not just to tick a box, but because ticking that box is a kind of due diligence towards building a more inclusive community. Unless they're ready to claim that there's no interesting non-white, non-male speakers able or willing to speak at their conference, then the failure is theirs to make any effort at all. But if they did make the effort and failed to attract someone, then they can say "we actually tried, and couldn't do it." The effort need be no more than to send a bunch of emails, and if that fails, it demonstrates a deeper problem in their community than they can fix.