Setting Recurrence Rules to Conflict with One-Time-Dates
Create a "decoy" holidays set, with one single "arbitrary dates" holiday in it. Then just enter all the "one-time" ocurrences into that holiday and have the recurring holidays conflict rule point to that single holiday.