6 months ago I would have swapped with you! Although mine were piano black. Always loved black but OCD could never handle the microscratches.
But the cars gone now so...
Few options I guess...
1. More than a handful of times in even just the short-time I owned the SQ5, there were carbon and piano black trims posted on ebay.
2. Depending on what you think, you could just get them wrapped (professionally!)
3. I've also seen them on sale on the various Audi forums but this is more rare
4. Depending on the look you are wanting to get, you can even get "stick-ons" but obviously more than enough negatives here
PS: Not wanting you to spend more money and unless you change your wheel too, it might look odd but that gear selector "head" can be very easily removed (just clipped on)
I changed mine for the perforated ones found in the slightly newer SQ5 (but in the same shape as yours). Again OCD maybe but I couldnt handle the hand-oil and leather ding marks on it.
This was another thing I found on the bay and ended up spending about £70 for the whole selector and then selling it back with the old cap getting back 70% of what I spent.